- How do I determine the total cost of running a stream?
- How do I determine the total cost of running a Historics query?
- What are licensing fees?
- What's a DPU?
- What is the cost of running a 1 DPU stream for 1 hour?
- How long does it take to run a Historics query?
- What if I stop a Historics query midway? Will I still be charged for it?
- What's the difference between credits and DPUs?
- Why do you charge using DPUs?
- What are the benefits of this flexible pricing?
- How do I optimize my streams to use fewer DPUs?
- How can I check my billing?
- Why is my DPU cost higher than I expected?
- Why can't I pay a fixed price?
- What is the charge for a Historics Preview?
The total cost of running a stream depends on the complexity of the stream (which we measure in DPUs), how long you run the stream for, and the number of output objects it produces.
For example, each Tweet that we deliver to you costs $0.0001. If you create a stream that costs 1 DPU and run it for 6 hours, and receive 4,000 Tweets, the cost will be:
1 * 6 * 0.20 + 0.0001 * 4000 = $1.60
You cannot predict the license fee in advance because it is impossible to predict how many messages users will post. News stories, by their very nature, often come as complete surprises. However, you might decide to make an estimate of the traffic a stream will generate by running a test, either via DataSift's API or GUI, for a few minutes and extrapolating those results.
The cost of running a Historics query depends on data processing usage plus the licensing costs. Data processing usage is calculated based on the duration of the Historics query and the sample size of the output data; it is deducted from the monthly DPU usage. Licensing costs depend on the volume of data retrieved.
For example, you created a Historics Query of a simple stream of Tweets that costs 0.1 DPU, for the timeframe of one month, with 10 percent as the sample output data size. The data processing usage for this Historics query is calculated to be 288 DPU and it is deducted from your monthly DPU allowance. As per the usage statistics, the volume of data retrieved is a total of 1,212,194 augmentations and sources. Since Twitter charges $0.10 for every 1000 Tweets that we retrieve for the query, hence the licensing costs of the Historics Query will be $121.21 approximately.
DataSift charges licensing fees on behalf of our partner sites such as Twitter. The license fee that you pay is exactly proportional to the number of objects your stream produces. Or in case of Historics queries, the number of objects retrieved by your query.
If want to create very highly targeted streams, you should typically expect to receive a low volume of data and so your license fees will be very low. For example, a filter for Tweets about hippopotamuses sent by authors with an unusually high Klout score within a radius of 20 miles of San Diego zoo, isn't going to generate very much output, even on days when the hippos do something exceptional.
On the other hand, a filter that looks for any mention of, say, music will probably generate substantially more output and cost more in license fees.
A DPU is a Data Processing Unit, a reflection of the computational complexity for the processing that you perform on the DataSift platform. A higher number represents a more complex stream. We measure DPUs on a per-hour basis because running a stream for five hours costs five times as much as running it for one hour.
It costs 20 US cents. If you purchase a subscription, you will benefit from a discount.
It depends on the capacity of our data archive at the time of creating a Historics query, as well as the timeframe and sample size of your Historics query. The timeframe of the query is the duration between the start date and time, and the end date and time of the query. The sample size of the output data can be either 100 percent or 10 percent of all the available data.
When you create a Historics query, it needs to access our data archive in order to retrieve output data for a selected timeframe. Our data archive could be very busy when multiple Historics queries are running at the same time. If the data archive is running over capacity, your query will be queued; that is, it will have to wait for access until other queries accessing the data archive have been executed. Although the queuing process takes a little time, keep in mind that Historics queries, once they are running, retrieve data 30 times faster than a real-time filter.
Once your Historics query has access to our data archive, it then depends on the timeframe of your query and the sample size of the output data. A Historics query with a shorter timeframe and a sample size of 10 percent is likely to execute more quickly than a query with a longer timeframe and a sample size of 100 percent.
Yes, you will be charged even if you stop a Historics query midway. You will be charged for the licensing costs of the volume of data retrieved until you stopped the query. The data processing usage until you stopped the query will also be deducted from your monthly DPU usage.
You purchase credits on the DataSift platform. Credits are priced in US dollars.
A credit costs $1. One credit is equivalent to 5 DPUs so the effective price of one DPU is 20 cents.
DataSift believes customers should only pay for what they consume. DataSift is a cloud platform, allowing you to consume only what you need and retain the flexibility to scale, either up or down, whenever necessary. The DPU amount that you pay is determined by the complexity of the rules you create.
Applications need to handle dynamic loads to survive. DataSift provides dynamic vertical scaling to handle unexpected data spikes as well as horizontal build out to support application growth over time.
Via the REST API:
For Streams, hit the /dpu endpoint to find details of DPUs for a stream.
For Streams, hit the /usage endpoint to find how many objects DataSift has delivered to you.
For Historics queries, hit the historics/prepare endpoint to get details of DPUs for your Historics query.
Via the DataSift GUI, visit our Billing page.
The minumum DPU charge is currently 1 DPU per hour, no matter how simple your stream is.
If you run just one stream that costs just 0.1 DPU, the total charge is 1 DPU per hour, which equates to 20 cents. In other words, the minimum DPU cost to use the platform is 20 cents per hour.
However, if you run ten streams, and they all cost 0.1 DPU, DataSift will still charge only 1 DPU per hour for all ten.
You can pay a fixed price for the processing but not for the licensing. We offer a range of subscriptions which include prepaid DPUs. The license cost of the content is variable and depends on the number of objects your stream returns. Clients who choose to prepay for DPUs benefit from a discount.
A Historics Preview is charged at a fixed cost of 20 DPUs per request. 20 DPUs are deducted from your account after a complete and successful execution of your request.
Applications succeed or fail based on performance. DataSift dynamically distributes workload into under-utilized CPU and memory resources to provide best-in-class service delivery with very low latency. DataSift consistently performs faster than competitors. A new Tweet, for example, is likely to be available on our platform 1 to 2 seconds after it appears on Twitter.
Yes! We are able to offer the most reliable and comprehensive SLA to our customers because we host our own dedicated cloud infrastructure. It is built on a massive-scale Service Orientated Architecture giving our customers peace of mind that we will continue running when others fail.
For more details, please take a look at our Terms and Conditions.