historics/prepare

Create a new Historics query and return its id.

An HTTP POST request sent to:

https://api.datasift.com/v1.3/historics/prepare

A successful call to this endpoint returns: 202 Accepted plus a JSON object.

Parameters

Parameter Description
hash
required

The hash of the CSDL for your Historics query.

Example values: 2459b03a13577579bca76471778a5c3d

start
required

Unix timestamp for the start time.

Example values: 1325548800

end
required

Unix timestamp for the end time. Must be at least 1 hour in the past.

Example values: 1325548800

name
required

The name you assign to your Historics query.

Example values: Football

sources
required

Comma-separated list of data sources to include. In the future, you will be able to choose any source listed as a valid value that you would use in the interaction.type target. Currently, you can use any of the sources shown here.

Example values: twitter, facebook, bitly, tumblr

sample
optional

You can sample 100 percent or 10 percent of the data.

If you omit this parameter, the sample defaults to 100.

Example values: 10

Examples

  1. The /v1.3/historics/prepare endpoint returns an id for your Historics query. This id is important because you'll use it later with all the other Historics endpoints in the REST API:

curl -X POST https://api.datasift.com/v1.3/historics/prepare
    -d 'hash=1d1be27f4a07bf1e9503f3c71ddd77a8'
    -d 'name=mydata'
    -d 'start=1341673200'
    -d 'end=1341759600'
    -d 'sources=twitter'
    -H 'Authorization:username:apikey'

Here's an example of the JSON output that you receive when you call this endpoint. The id returned is the Historics id that you will need when you hit the /push/create and /historics/start endpoints:

{
    "dpus": 234.6916666667,
    "id": "4ef7c852a96d6352764f",
    "availability": {
        "start": 1341673200,
        "end": 1341759600,
        "sources": {
            "twitter": {
                "status": 99,
                "versions": [
                    3
                ],
                "augmentations": {
                    "klout": 50,
                    "links": 100
                }
            },
            "facebook": {
                "status": 99,
                "versions": [
                    2,
                    3
                ],
                "augmentations": {
                    "links": 95
                }
            }
        }
    }
}

Property: Description:
dpus The cost in DPUs to run this Historics query. Remember that some data sources charge a license cost, which we pass on to you in addition to the DPU cost.
id The Historics id, a unique identifier for this Historics query.
start

Unix timestamp defining the start of a day in our Historics archive. A day starts at midnight and ends one second before midnight 24 hours later.

In your call to /historics/prepare, you can specify any start time to one second of accuracy. However, the 'start' returned in the JSON is always midnight on the day defined by the start value you supplied in your API call. The status element in the JSON (see below) gives you the percentage data coverage available in the archive for the interval between the JSON start and end.

end

Unix timestamp defining the end of a day in our Historics archive. A day starts at midnight and ends one second before midnight 24 hours later.

In your call to /historics/prepare, you can specify any end time to one second of accuracy. However, the 'end' returned in the JSON is always one second before midnight on the day defined by the end value you supplied in your API call. The status element in the JSON (see below) gives you the percentage data coverage available in the archive for the interval between the JSON start and end.

We frequently find that calls to /historics/prepare specify their start to be midnight on one day and their end to be midnight on another day. Technically, this means that the request ends one second after the end of a day as defined in our archive. We overlook this second since it makes no sense to consider data coverage for an additional day just for the sake of one extra second.

status

The status element tells you how complete our data coverage is for the time interval you requested and for the sources your query accesses. Usually we have 100 percent of the data but you can call the /historics/prepare endpoint to check.

Your Historics query can run from any start and end timestamps; you might require data from a portion of a day or, for example, 10am on January 24 to 2:15pm on January 27. However, the figures returned in status are based on an examination of one or more entire days. See the description of the start and end JSON elements above.

versions This element is reserved for future use.
links For each Augmentation (Klout, Links, and so on) the JSON shows the percentage of coverage available for the time interval defined in start and end.

Notes

  1. Be sure to read our introduction to the Historics API to get started with these endpoints.
  2. Also take a look at the endpoint dependencies.
  3. There is a known problem with the end parameter returned in the JSON. It should return a timestamp that represents the end of the day you specified in your end parameter in the call to this endpoint; that is, one second before midnight. Instead it returns a timestamp that represents midnight at the start of that day.
  4. All calls to the API must be properly authenticated with a DataSift username and API key.
  5. All calls to the API must be versioned. The current version is v1.3.

Resource information

Rate limit cost: 25

Requires authentication: Yes

Response formats: JSON