Push is a simple and robust mechanism for periodically delivering your data directly to a wide range of Data Destinations.
What's the difference between Push and Streaming?
Live streaming delivers interactions individually over a long-living HTTP or websocket connection that you maintain with us. This means you receive data with sub-second latency. In truly time-critical situations it is a good option.
With the Push mechanism, we buffer up to 60 minutes worth of data for you. Periodically, we connect to your Data Destination to deliver accumulated data in a short burst. You control the time interval between these bursts.
Push removes the need for you to maintain an online connection with us over long periods of time, it can deliver higher throughput streams more reliably, and it adapts to a range of widely supported, standard, stable protocols, such as HTTP POST, FTP, and Amazon AWS DynamoDB.
Unless you need to receive your data with very low latency, we recommend Push.
What does DataSift do if the Data Destination goes down?
We keep your data safe by buffering it. In the event of a problem, we'll try to resend up to five times immediately. Then, we'll retry every minute for 60 minutes. Once the connection is restored, we'll revert to the original delivery frequency.
When can I use Push?
If you're running streams, we recommend Push but you do have the option of live streaming instead.
If you're running Historics queries through the Historics API, you must use Push.
Benefits of Push
- Never lose data because we guarantee delivery.
- We buffer your data for up to 60 minutes. If you experience temporary problems with your systems, we'll hold onto your data for up to one hour and attempt to resend it as soon as your systems come back online.
- You can pause and resume. If you need downtime, we can delay delivery.
- We can deliver to any number of endpoints simultaneously.
- You control the way you receive data and the way it is stored.
- You control where the data goes. We already have several connectors available and others are close to release for data warehouses, popular databases, document stores and search engines.
- You no longer need to write middleware code to store raw data to your favorite database. We can store it automatically to any data destination for which there is a connector. Let us know if there's one you'd like that doesn't already exist.