API Rate Limiting

This page describes rate limits imposed by DataSift's API to ensure fair usage of the platform.

General API rate limit

Regardless of the product you are using, any calls to the REST API are subject to the general API rate limit.

DataSift gives you 10,000 credits and replenishes them every hour. For example, if the Rate Limit Cost of validating a filter is 5 credits and you validate it four times, the total cost is 20 credits, leaving you with a balance of 9,980 credits for the current hour.

The table below summarizes the cost of each REST API endpoint.

If this limit causes your application any problems, please contact DataSift Support.

The DataSift 'hour' starts when you first hit the API. For example, if you compile a rule at 8:05 am and consume all of your 10,000 credits by 8:50 am, DataSift will give you another 10,000 credits at 9:05 am. These credits are free of charge; there is no dollar equivalent. We've documented the Rate Limit Cost for each endpoint in a summary box that looks like this:

Rate limiting is based on the type and the number of requests. You can determine the cost of an API request by inspecting the X-RateLimit-Cost header.

Response Headers

Every time you make a request to the API we will return the current rate status in the response header. Look for the following keys:

X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 9980
X-RateLimit-Cost: 5
X-RateLimit-Reset-Ttl: 1729
X-RateLimit-Reset: <timestamp>

If the X-RateLimit-Remaining falls to 0, you are rate limited. Wait up to an 1 hour for the next 10,000 credits to reach your account.

If you hit your API limit, all subsequent requests will fail with a suitable error message and HTTP response code of 403 or 429.

X-RateLimit-Reset-Ttl gives the number of seconds until the next reset.

X-RateLimit-Reset gives the Unix timestamp when the next reset will occur.

API Rate Limit Summary

Account Api
Endpoint Rate Limit Cost
GET /account/usage 5
Endpoint Rate Limit Cost
validate 5
compile 5
usage 25
dpu 5
balance 25
Endpoint Rate Limit Cost
push/get 1
push/validate 5
push/create 5
push/pause 1
push/resume 1
push/update 5
push/stop 1
push/delete 1
push/log 1
pull 0
Managed Sources
Endpoint Rate Limit Cost
source/get 5
source/create 25
source/start 5
source/stop 5
source/delete 25
source/update 25
source/log 5
source/auth/add 25
source/auth/remove 25
source/resource/add 25
source/resource/remove 25
source/category/get 5