Platform Allowances

Interaction filters

You are limited to 1MB of text and/or 999 logical operators (AND, OR) in each filter definition.

You can use up to 10,000 tags in each of your filters. You can define them locally or include them from external definitions; the limit is the same in both cases. For further information see our classification documentation.


Results of interaction filtering are placed in the index. They expire after 32 days in the index.

Your package specifies the number of concurrent recordings you can run at any time.

Your package also specifies your allowed index capacity per month **** for your account. This translates to a daily, account-level interaction limit based on the following formula:

Daily account limit* = Index capacity / 30 days

*The daily account limit is always rounded up to the nearest 1 million. For example, if your index capacity is 12 million, your daily account limit would be 1 million.

Your index capacity can be used as needed across filters subject to an additional limit of 1 million interactions per filter per day.

For limiting purposes, a day is defined as a 24-hour window beginning at 00:00:00 Pacific Standard Time. If you hit your index limit, it will reset at midnight Pacific Standard Time. So if you hit the limit at 23:00:00, it will reset an hour later. But if you hit the limit at 02:00:00, it will reset at midnight the following night, 22 hours later.

No limits are enforced on the complexity of interaction filters.

Analysis queries

The total number of analysis queries per hour for your account will depend on the package you subscribe to.

We currently limit query filtering in CSDL to 30 logical operators, each with a maximum of 100 arguments.

The /pylon/analyze endpoint allows you to include up to 200 elements returned in a freqDist analysis.

In order to make use of your entire hourly analysis query allowance, your queries need to be distributed evenly across each hour.

API rate limit

To ensure fair usage, each API call has a rate limit cost. Take a look at our API Rate Limiting page for full details. The main thing you need to know is that you have an allowance of 10,000 credits per hour and each API endpoint (except the /pylon/analyze endpoint) consumes a small piece of that allowance. If you are rate limited, your API calls will no longer be accepted and you will need to wait until the next hour begins.

You can find the cost of calling each endpoint in the Resource Information box on its documentation page. For example, the cost of each call to /pylon/compile is 5 credits.

What you need to know:

  • The hourly analysis query limit depends on your package.
  • The rate limit costs of all the other PYLON endpoints count against the allowance of 10,000 that DataSift gives you each hour. For example, if you hit /pylon/validate and then /pylon/compile, each of which have a rate limit cost of 5, your initial allowance of 10,000 will fall to 9,990.
  • That allowance of 10,000 credits is used by the endpoints in DataSift's PYLON product and those in DataSift's Stream platform. If you are using both sets of endpoints at the same time, in the same DataSift account, you will use up your allowance more quickly.
  • The 10,000 credits is issued hourly. If your first call is at 5 minutes past the hour and you hit your rate limit, we will reset it when the time next reaches 5 minutes past the hour.

Response Headers

Every time you make a request to the API we will return the current rate status in the response header. Look for the following keys:

X-RateLimit-Limit: 10000
X-RateLimit-Remaining: 9980
X-RateLimit-Cost: 5
X-RateLimit-Reset-Ttl: 1729
X-RateLimit-Reset: 1435852586

If the X-RateLimit-Remaining falls to 0, you are rate limited. Wait up to an 1 hour for the next 10,000 credits to reach your account.

If you hit your API limit, all subsequent requests will fail with a suitable error message and HTTP response code of 403.

API rate limit for /pylon/analyze

The Rate Limit Cost for this endpoint is 25. However, this cost is not taken from your regular allowance of credits. The /pylon/analyze endpoint has its own allowance of credits. The number of credits you can use per hour depends on your customer agreement.

Note that if you view analysis results in the DataSift dashboard this analysis is performed via the API and therefore consumes your allowance for the the /pylon/analyze endpoint.

API Rate Limit Summary

Pylon For Facebook Topic Data
Endpoint Rate Limit Cost
pylon/get 25
pylon/compile 5
pylon/validate 5
pylon/start 25
pylon/stop 5
pylon/update 25
pylon/analyze 25
pylon/sample 25
pylon/tags 25
Account Api
Endpoint Rate Limit Cost
GET /account/identity 5
POST /account/identity 5
GET /account/identity/{id} 5
PUT /account/identity/{id} 5
GET /account/identity/limit/{service} 25
GET /account/identity/{identity_id}/token 5
POST /account/identity/{identity_id}/limit 25
POST /account/identity/{identity_id}/token 25
GET /account/identity/{identity_id}/limit/{service} 25
PUT /account/identity/{identity_id}/limit/{service} 25
GET /account/identity/{identity_id}/token/{service} 5
PUT /account/identity/{identity_id}/token/{service} 5
DELETE /account/identity/{identity_id}/limit/{service} 25
DELETE /account/identity/{identity_id}/token/{service} 5
GET /account/usage 5
Endpoint Rate Limit Cost
validate 5
compile 5
usage 25
dpu 5
balance 25
Endpoint Rate Limit Cost
push/get 1
push/validate 5
push/create 5
push/pause 1
push/resume 1
push/update 5
push/stop 1
push/delete 1
push/log 1
pull 0
Managed Sources
Endpoint Rate Limit Cost
source/get 5
source/create 25
source/start 5
source/stop 5
source/delete 25
source/update 25
source/log 5
source/auth/add 25
source/auth/remove 25
source/resource/add 25
source/resource/remove 25
Endpoint Rate Limit Cost
historics/get 25
historics/prepare 25
historics/start 25
historics/stop 5
historics/pause 5
historics/resume 5
historics/update 5
historics/delete 25
historics/status 5
Historics Preview
Endpoint Rate Limit Cost
preview/get 5
preview/create 25