Azure DevOps API Continuation Tokens / Graph API not returning all users

I was doing some work last week with Azure DevOps user and security management Graph APIs. The Graph APIs let you manage users, and groups, and group memberships and I needed to make some changes for a couple thousand user accounts. It was definitely worth automating.

When I was working with the API call that gets the list of users, I was getting odd results. The problem was that I was only getting the first 499 users. At first, since I didn’t know how many users I had, I didn’t think too much about that 499 number but then I kept getting all these errors saying that users couldn’t be found by my script. I’d go and look at the admin website and those users would be there so what the heck was going on?

Azure DevOps API Continuation Tokens

The API that was giving me the most trouble was the one that returns a list of all known users in Azure DevOps. Like I mentioned above, I’d get back 499 users and that was it. When I looked at the documentation it said that it would return a continuation token if there were too many users in the result set. “Since the list of users may be large, results are returned in pages of users. If there are more results than can be returned in a single page, the result set will contain a continuation token for retrieval of the next set of results.”

A continuation token is a little chunk of data that you pass back to the server to say “ok…give me the next chunk of data for my query”. But I wasn’t getting a continuation token in the JSON result set. At least I didn’t think I was getting a continuation token.

It turns out that I actually was getting a continuation token but it just wasn’t where I expected to find it. I’d been expected it in the JSON result from the service call.

NOPE!

When I went and looked at the raw network traffic for that service call using Edge’s developer tools, I found that the continuation token was stored in the response header. The header name for the continuation token is x-ms-continuationtoken.

I needed to grab that header value and put it on to the next HTTP GET request for that service call.

Some Code to Get the Continuation Token

Here’s some sample code that calls that Azure DevOps Graph API to get the users. Notice that it checks for the header value and extracts it if it’s there.

private async Task GetUsers()
{
    var onlyCount = ArgNameExists(UtilityConstants.CommandArg_OnlyCount);

    if (ArgNameExists(UtilityConstants.CommandArg_FilterContains) == true)
    {
        _hasFilter = true;
        _filter = Arguments[UtilityConstants.CommandArg_FilterContains];
    }

    using var client = GetHttpClientInstance(_tpcBaseUrl, _tokenAsBase64);

    var requestUrl = $"https://vssps.dev.azure.com/{OrganizationName}/_apis/graph/users?api-version=6.0-preview.1";

    var response = await client.GetAsync(requestUrl);

    if (response.IsSuccessStatusCode == false)
    {
        throw new InvalidOperationException($"Problem with server call to {requestUrl}. {response.StatusCode} {response.ReasonPhrase}");
    }

    var headers = response.Headers;

    string continuationToken = null;

    if (headers.Contains(Header_ContinuationToken) == true)
    {
        Logger.LogInfo("** CONTINUATION TOKEN **");
        continuationToken = response.Headers.GetValues(Header_ContinuationToken).FirstOrDefault();
        Logger.LogInfo($"Header --> {Header_ContinuationToken}: {continuationToken}");
    }

    var responseContent = await response.Content.ReadAsStringAsync();

...
}

Some Code to Pass the Continuation Token Back to the API

This next method uses the continuation token from the previous call in order to get the next set of results. If you think that this method is darned similar, you’re right. Pretty much the only difference is that this API call passes the token using the continuationtoken query string variable.

private async Task<string> GetUsers(string continuationToken)
        {
            var onlyCount = ArgNameExists(UtilityConstants.CommandArg_OnlyCount);

            using var client = GetHttpClientInstance(_tpcBaseUrl, _tokenAsBase64);

            var requestUrl = $"https://vssps.dev.azure.com/{OrganizationName}/_apis/graph/users?api-version=6.0-preview.1&continuationToken={continuationToken}";

            var response = await client.GetAsync(requestUrl);

            if (response.IsSuccessStatusCode == false)
            {
                throw new InvalidOperationException($"Problem with server call to {requestUrl}. {response.StatusCode} {response.ReasonPhrase}");
            }

            var headers = response.Headers;

            if (headers.Contains(Header_ContinuationToken) == true)
            {
                Logger.LogInfo("** CONTINUATION TOKEN **");
                continuationToken = response.Headers.GetValues(Header_ContinuationToken).FirstOrDefault();
                Logger.LogInfo($"Header --> {Header_ContinuationToken}: {continuationToken}");
            }
            else
            {
                continuationToken = null;
            }

            var responseContent = await response.Content.ReadAsStringAsync();

            var typedResponse = GetJsonValueAsType<GetUsersResponse>(responseContent);

            Users.AddRange(typedResponse.Values);

            if (onlyCount == true)
            {
                Console.WriteLine($"Current batch count: {typedResponse.Count}");
                Console.WriteLine($"Total count: {Users.Count}");
            }

            return continuationToken;
        }

Summary

So. If you’re finding that your Azure DevOps API calls aren’t returning all the records that you expected, check to see if you have an x-ms-continuationtoken value in your HTTP response headers. If you do, that means that there are more results waiting for you on the server and you need to make another call.

I hope this helps.

-Ben

— Need help migrating your on-premise Azure DevOps or Team Foundation Server to the cloud? Looking for help working with the Azure DevOps REST APIs? Not sure where to start with your version control problems or with continuous integration / continuous deploy pipelines? We can help. Drop us a line at info@benday.com.