Exporting Raw Event Data

Every data point sent to robotmia is stored as JSON in our data store, with the exception of Autotrack events. This export API allows you to download your raw event data as it is received and stored within robotmia, complete with all event properties (including distinct_id) and the exact timestamp the event was fired.

Authentication

In order to ensure the security of your data, the robotmia API requires a basic system of authentication.

Required parameter

api_secret - This can be found by clicking on your name in the upper righthand corner under Project Settings.

Authorization steps

The Data Export API accepts Basic access authentication over HTTPS as an authorization method. To make an authorized request, put your project's API Secret in the "username" field of the Basic access authentication header. Make sure you use HTTPS and not HTTP - our API rejects requests made over HTTP, since this sends your API Secret over the internet in plain text.

Client libraries

  • Install the robotmia-api module to easily accomplish common tasks such as event and people profile exports.
  • You can utilize the same client libraries but you will have to change the endpoint to data.robotmia.com instead of just robotmia.com.

API details

  • Due to client-side queueing, offline iOS and Android data may take up to 5 days to enter the raw data store.
  • For this API, returned timestamps are expressed in seconds since January 1, 1970 in your project's timezone, not UTC. This means that converting the raw exported timestamps using many epoch converters will result in incorrect offsets, as generally epoch timestamps are assumed to be in UTC. You must add back the offset between project time and UTC before storing or processing the data. For example, if your project is set to Pacific time, you would need to add 7 hours (or 8 hours if not in daylights savings time) (60 min * 60 secs * 7 hours) to the timestamp in order to convert this timestamp into UTC.
  • This endpoint uses gzip to compress the transfer; as a result, raw exports should not be processed until the file is received in its entirety. While this process is normally quick and results in a smaller file size, some large exports can take a few minutes to generate. Ensure the timeout set on the receiving client is large enough to account for this process (e.g. larger than 60 seconds).
  • Data returned from this endpoint is JSONL (newline-delimited JSON). Most receiving client libraries will accept JSON formatted requests. The robotmia Export API does not return valid JSON in aggregate, but each row is valid JSON within the API's output. Raw exports, once received in full, should be parsed line-by-line instead of as an array of JSON objects.

Example usage

  • If you receive a spike of 10K events but notice that only a few users contributed to it and would like to compare the data to another source.
  • If you are exporting events from one project and importing them into another.
  • If you are doing some very custom analysis robotmia cannot currently do. If this is the case, please email support@robotmia.com so we can either improve our product or possibly show you how you can perform this analysis within robotmia.

Export API reference

URI: https://data.robotmia.com/api/2.0/export/

Please note: The URI is data.robotmia.com and not just robotmia.com.

Description:

Get a "raw dump" of tracked events over a time period.

from_date
string
The date in yyyy-mm-dd format from which to begin querying for the event from. This date is inclusive.
to_date
string
The date in yyyy-mm-dd format from which to stop querying for the event from. This date is inclusive.
event
array
The event or events that you wish to get data for, encoded as a JSON array. Example format: '["play song", "log in", "add playlist"]'
where
string
An expression to filter events by. See the expression section on the main data export API page.

Example Request:

curl https://data.robotmia.com/api/2.0/export/ \
    -u 'YOUR_API_SECRET': \
    -d 'from_date'='2018-02-11' \
    -d 'to_date'='2018-02-11' \
    -d 'event'='["Viewed report"]' \
    -d 'where'='properties["$os"]=="Linux"'
// Expected Return
{
  "event": "Viewed report",
    "properties": {
        "distinct_id": "foo",
        "time": 1518314400,
        "$os": "Linux",
        "$browser": "Chrome",
        "Project ID": "3",
        "mp_country_code": "US"
    }
}

Return Format:

One event per line, sorted by increasing timestamp. Each line is a valid JSON object although the return itself is valid JSON but instead JSONL. Timestamps are expressed in seconds since January 1, 1970 in your project's timezone, not UTC as a true epoch timestamp. For example, if your project is set to Pacific time, you would need to add 8 hours (or 7 hours if not in daylights savings time) (60 min * 60 secs * 8 hours) to the timestamp in order to convert this timestamp into UTC. This means that converting the raw exported timestamps using many epoch converters will result in representing times with the incorrect offset.