Deprecating Historics "volume_info" Output Field

On December 2nd, 2013, we plan to remove the "volume_info" field from the DataSift Historics API call response. Please ensure that your application does not expect to receive this field from Historics API calls by this date. If you are using one of the official DataSift API client libraries, support for this has already been implemented in the following versions of the libraries: Java - 2.2.1+ Python - 0.5.4+ Ruby - 2.0.3+ PHP - 2.1.4+ .NET - 0.5.0+

Read Deprecating Historics "volume_info" Output Fieldimageimage

New delivered_at meta field for Push

DataSift is adding a new metadata field to each JSON object delivered via Push in the json_meta output format - a delivered_at timestamp. This new timestamp represents the time DataSift delivered this particular object. An example of a json_meta formatted Push delivery containing this new field can be seen below:   {"count":3, "hash":"4ede6111534c5e29145f", "hash_type":"historic", "id":"58802d124916ed826a08d58d791f85c5...

Read New delivered_at meta field for Pushimageimage

Using Managed Sources

I've noticed some questions from clients who are using Managed Sources for the first time. In this blog I'm going to go through the steps to run a DataSift filter on a Managed Source: Create a token Create a Managed Source Create a CSDL filter for that Managed Source Start recording the output of the filter Start the Managed Source I'll use Facebook in my examples, but the process is similar for all the Managed Sources the platform offers. Suppose that...

Read Using Managed Sourcesimageimage

Pulling Data with the Pull (Push) Connector

The Pull Connector is the latest addition to our growing family of Push connectors. This new Push connector takes its name after the mechanism used to deliver the interactions you filter for: you pull data from our platform instead of us pushing it to you. Even though the name of this connector might seem to be out of place for a Push connector, it makes sense to classify it as another Push connector, because it uses the same robust Push subsystem that powers other DataSift Push Connectors....

Read Pulling Data with the Pull (Push) Connectorimageimage

Platform Performance Gains with Arista Switches

In late 2012 I wrote about the migration of DataSift's Hadoop cluster to Arista switches but what I didn't mention was that we also moved our real-time systems over to Arista too. Within the LAN During our fact-finding trek through the Cisco portfolio we acquired a bunch of 4948 and 3750 switches which were re-purposed into the live platform. Unfortunately, the live platform (as opposed to Hadoop-sourced historical data) would occasionally experience performance issues due to...

Read Platform Performance Gains with Arista Switchesimageimage