Using the Graph API with large data sets

Use Paging to retrieve more records

For this article I’ll use the convention /me when I mean https://graph.microsoft.com/v1.0/me
Sometimes there will be a need to retrieve blocks of data that are too large for a single API call.  For example, the query /users in most organizations will return more data than a single call can accept. This query would return all the users in the current Active Directory. Here are some strategies to use in this situation.

With a query like /users the API will return the default page size number of items, and then include a field called @odata.nextLink. The @odata.nextLink field contains a url to the query that will return the next page of data.
You can set the page size with $top=xx where xx is the desired page size. So the query /users?$top=200 will return the first 200 users and a @odata.nextLink url to the next page.

You have reached the last page when the query returns less than the page size. If the number of pages * page size = the item count, the last request will return an empty set.

For the users collection the default page size is 100, the max page size is 999, if you try $top=1000 you’ll get an error for invalid page size.
The response to /users/$top=2 looks like this.

With this knowledge you can construct a loop that loads the entire data set one page at a time.

Use Select to limit the data returned

One way to reduce the size of the return payload is using the $select query parameter to request only the fields you need.  For example /users?$select=displayName,mail would return only those two fields for each user.
The response to this query looks like this.

If you don’t specify the $select the default set of fields is returned, there may be other fields that can only be returned when explicitly requested or when requesting a single item.
 

Be aware of throttling

If the response to a request is a 429 error, this indicates that throttling has occurred. The response header will contain a value called Retry-After which indicates the number of seconds to wait before retrying the call. Wait the requested number of seconds, then try the call again, if it fails again on the 429 error, get the new value for Retry-After and continue to wait. More details can be found in the article https://docs.microsoft.com/en-us/graph/throttling.

Use the Microsoft Graph Explorer to quickly browse the Graph API to discover the urls and properties available. https://developer.microsoft.com/en-us/graph/graph-explorer