Documentation

  1. Home
  2. Documentation

The Basics

About

This document details functionality of v1.0 of the Information Machine API.

Information Machine’s API is designed to be REST-ful; its URLs follow predictable patterns and responses carry appropriate HTTP response codes, among other features that characterize the REST architectural style.

All requests made to the Information Machine API must be URL encoded.

All responses from this API, including error messages, will be returned in JSON format.

API Key & Secret

Before you write any code, make sure you create your free Information Machine account to retrieve your API credentials , i.e., your API key and API secret.

Be careful never to publish your API secret key as it uniquely identifies your account and will allow anyone with your API key to gain access to your account. If your API details are compromised, please contact us immediately.

API Requests

API requests are made up of four components:

  • The API Endpoint to which your requests should be directed, e.g., https://www.iamdata.co/v1/
  • An API Resource with which you wish to interact, e.g., products, categories, stores, etc.
  • Query & Filter Parameters that define how you interact with these API resources, e.g., ?name=Milk&page=1&per_page=15&full_resp=false
  • Authorization which consists of client_id (your API Key) and client_secret (your API secret) which you can get from your dashboard.

API Responses

Best way to learn an API is from an example and in this example we will see how response looks like when calling /v1/products endpoint.

The below url will get products which have matching UPC/EAN number, this is represented as product_identifier=014100044208 query parameter.

https://www.iamdata.co/v1/products?product_identifier=014100044208&page=1&per_page=10&full_resp=false&client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

If you call the above url link given the proper API key and secret then the response should look something like this:

{
    "result": [
    {
        "serving_size_in_grams": 30,
        "serving_size_unit": "g",
        "servings_per_container": "5.5",
        "serving_size": "30.0",
        "ingredients": "Whole grain yellow corn flour, corn flour, vegetable oils (canola, sunflower and/or soybean), soft red wheat, brown sugar, cornstarch, corn maltodextrin, hulled barley, rye, triticale, contains 2 percent or less of: sugar, salt, leavening (baking soda, monocalcium phosphate, sodium acid pyrophosphate), dehydrated onions, citric acid, modified whey*, molasses, nonfat milk*, yeast extract, malted barley flour, dehydrated garlic, dehydrated tomatoes, dextrose, wheat flour, soy lecithin, autolyzed yeast, paprika, spices (contains celery), natural flavors, paprika extract, sesame oil, vinegar and natural smoke flavoring.\n*adds a trivial amount of cholesterol.",
        "weight": "6.0 OZ",
        "description": "Pepperidge farm classic bbq cracker chips:/deliciously baked/ridiculously crunchy/70% less fat than leading potato chip/no preservatives/16g whole grain per serving//",
        "brand": "Pepperidge Farm",
        "upc": "014100044208",
        "tags": ["Cereals","Corn","Gluten","Lactose","Milk","SesameSeeds","Soy","Sulfites","Wheat"],
        "name": "Pepperidge Farm Classic Bbq Cracker Chips, 6 Oz",
        "id": 380312,
        ....
    }
    ],
    "meta": {
        "page": 0,
        "per_page": 0,
        "total_count": 0,
        "next_page": null,
        "last_page": null,
        "max_number_of_requests_per_day": 10000,
        "remaining_number_of_request": 9999,
        "time_in_epoch_second_till_reset": 1441159200
    }
}

The above response does not show all properties. There are more properties in this response which you can explore using the Interactive Docs page. From the response above you can see two major data properties, "Result" and "Meta"

The "Result" property contains the object from the resource that you were retrieving, the "Meta" property contains information about your api request such as:

  • page if result data is an array of objects.
  • per_page if the request returns an array of objects then this property contains the number elements per page in the given array.
  • total_count if the request returns an array of objects then this property contains the total number of elements.
  • next_page and last_page are url links to the next page and last page given the current request.
  • max_number_of_requests_per_day contains maximum number of requests per day.
  • remaining_number_of_request contains remaining number of request that you can make in the next time_in_epoch_second_till_reset seconds.
  • time_in_epoch_second_till_reset time till the next request counter reset.

Libraries

The easiest way to make requests to the API is to use one of our many client libraries. We recommend that you use these libraries since they absolve developers of the need to explicitly handle authentication and request construction.

controller = LookupController(client_id, client_secret)
response = controller.lookup_get_product_alternative_types()
print response.result[0].name
Python library on Github 
ProductsController productsController = new ProductsController(clientId, clientSecret);
ProductData productFull = productsController.productsGetProduct("380728", true).getResult();
List kaleProducts = productsController.productsGetProducts("Kale", null, 1, 25, true).getResult();
Android library on Github 
products_controller = InformationMachineApi::ProductsController.new client_id, client_secret
product_full = products_controller.products_get_product("380728", full_resp:true)["result"];
Ruby library on Github 
ProductsController productsController = new ProductsController(clientId, clientSecret);
ProductData productFull = productsController.ProductsGetProduct("380728", true).Result;
List kaleProducts = productsController.ProductsGetProducts("Kale", null, 1, 25, true).Result;
C# library on Github 
$productsController = new ctrl\ProductsController($clientId, $clientSecret);
$kaleProducts = $productsController->productsSearchProducts("Kale", NULL, 1, 25, NULL, true)->result;
PHP library on Github 
ProductsController productsController = new ProductsController(clientId, clientSecret);
ProductData productFull = productsController.productsGetProduct("380728", true).getResult();
List kaleProducts = productsController.productsGetProducts("Kale", null, 1, 25, true).getResult();
Java library on Github

The GitHub repositories for these libraries provide detailed explanations for making your first request.

Errors

Information Machine uses conventional HTTP status codes to indicate success or failure of API requests. In general, codes in the 2xx range indicate success, codes in the 4xx range indicate an error that resulted from the provided information (e.g. a required parameter was missing), and codes in the 5xx range indicate a network or server error.

The response text for error conditions typically contains four fields in the supplied JSON response, message, max_number_of_requests_per_day, remaining_number_of_request, time_in_epoch_second_till_reset, which describe in further detail the error that has occurred and how many requests are left on your account.

{
    "message": "No stores found for given criteria",
    "max_number_of_requests_per_day": "10",
    "remaining_number_of_request" : "5340",
    "time_in_epoch_second_till_reset" : "561"
}

Interactive Docs

The best place to learn the Information Machine API is to use our Interactive API playground to construct requests and explore the data.

IM User

About

IM user is entity referred by other IM API artifacts (store connections, bar code scans, etc.). Pair of client_id/client_secret keys, can create as many as needed IM API Users. It is must have for majority of "user" based actions.

Create IM User

In order to created IM User, you'll have to specify unique "user_id". Only alphanumeric characters, underscore, hyphen and dot are allowed in "user_id".

request sample (POST)

https://api.iamdata.co/v1/users?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

POST data

{
  "email": "mail@example.com",
  "zip": "21000",
  "user_id": "exampleID123456"
}

Edit IM User

In order to update IM User, you'll have to specify unique "user_id". Only alphanumeric characters, underscore, hyphen and dot are allowed in "user_id".

request sample (PUT)

https://api.iamdata.co/v1/users/exampleID12345?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

PUT data

{
  "email": "mail@example.com",
  "zip": "21000" 
}

List all IM User(s)

You can always list your user(s) by using a get method on users resource.

request sample (GET)

https://api.iamdata.co/v1/users?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Get single IM User

You can get user information by providing user_id.

request sample (GET)

https://api.iamdata.co/v1/users/USER_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Delete IM User

Delete user and all her/his store connections and associated data.

request sample (DELETE)

https://api.iamdata.co/v1/users?id=USER_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

User Stores

About

User store connection provides continuous stream of purchase data (historical and future). Connection is created via valid set of credentials (username/password). Check "Lookup/Store" section, in order to get list of stores available (only those with "can_scrape" = true).

Create

Store connection is created through API with username/password provided in request. On successfully created connection, you'll receive ID that uniquely identify your connection. In order to check credentials (store connection) status, you'll have to use GET method explained in "Find" section.

request sample (POST)

https://api.iamdata.co/v1/users/USER_ID/stores?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

POST data

{
  "store_id": 3,
  "username": "USERNAME",
  "password": "PASSWORD"
}

In order to find out what are possible values for store_id please check Lookup stores section

Find

Using user ID you can get all store connections for a specific user.

request sample (GET)

https://api.iamdata.co/v1/users/USER_ID/stores?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Using user ID and store connection ID, obtained during store connection creation, you can find store connection and get detailed information about it ("Not defined", "Verified", "Invalid", "Unknown" & "Checking") and scrape status ("Not defined", "Pending", "Scraping", "Done" & "DoneWithWarning" ).

Store connection information also contains two important aspects of a store connection, whether it is locked and a url to unlock the connection.

request sample (GET)

https://api.iamdata.co/v1/users/USER_ID/stores/STORE_CONNECTION_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Delete

You can delete you store connection, specifying store connection ID, obtained during store connection creation.

request sample (DELETE)

https://api.iamdata.co/v1/users/USER_ID/stores/STORE_CONNECTION_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Edit

Update of existing store connection is also possible. You can change both username and password.

request sample (PUT)

https://api.iamdata.co/v1/users/USER_ID/stores/STORE_CONNECTION_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

PUT data

{
  "username": "USERNAME",
  "password": "PASSWORD"
}

Unlocking the store connection

Sometimes the username and password are not enough. Some stores (Amazon, Walgreens...) could lock the account and display a security question, image captcha or something else.

From this moment on, we cannot get your purchase history until a user unlock its account.

For this purpose, store connection information also contains three important properties, whether it is locked (account_locked), the reason why it is locked (account_lock_code) and a url to unlock the store connection (unlock_url).

First check if the account_locked is set to true, if so present the user a url that is stored in unlock_url property of store connection.

Also, the user might be interesed in why the account is locked. You can check for a reason inside account_lock_code property which may contain one of these values:

  1. "Security Question" - this means our data bot has encountered a security question during login and connot go further without an answer to a security question. Once answered, the answer will be stored and the account will never be locked again beacuse of security question.
  2. "Image Captcha" - our data bot can not process image captcha so a user must assist in answering the image captcha.
  3. "Incomplete User Profile" - Site is asking a user to confirm something (new terms of service etc.) or complete the profile such as entering a missing zip code, reset the password etc. Please note that we do not support unlocking of incomplete user profile. The user should go to the website of the particular store and complete its profile. Once this is done the bot can scrape purchase history.
  4. "User Account Deactivated" - User's account has been deactivated by a website that we scrape. User needs to go to a particular store website and see further instructions on how to re-activate the account.
  5. "Access Denied" - Our API can not access the website at the moment
  6. "Password Reset Required" - User needs to reset his password on the website that we scrape, and then update the password on our API

When a user visits this url it will be presented with a view which asks for an answer to a security question (or image captcha, sms verification code etc.)

Once the user answers security question the account is unlocked and will be scraped shortly by our bots.

Adding receipt for User Store

We can get data from receipts provided only receipt number and receipt purchase date.

Currently this functionality is only supported for Walmart and H-E-B stores.

request sample (POST)

https://api.iamdata.co/v1/users/USER_ID/stores/USER_STORE_ID/receipts?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

POST data

{
  "receipt_id": "12345678901234567890",
  "receipt_date": "2016-11-10T15:14:46.025Z"
}

receipt_id should not contain any spaces in between characters.

Please note that receipt_date is required by Walmart store, and it should not be older than 7 days.

For H-E-B store if receipt_date is not specified then we will use current date as purchase date for that receipt.

Processing H-E-B receipt is usually a few minutes. But processing Walmart receipts can take up to several days.

You can get information about all receipts for a specified user and user store id by calling:

request sample (GET)

https://api.iamdata.co/v1/users/USER_ID/stores/USER_STORE_ID/receipts?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Response for this call looks something like this:

{
  "result": [
    {
      "id": 1,
      "user_store_id": 52800,
      "purchase_id": 123456,
      "identifier": "12345678901234567890",
      "status": "Done",
      "created_at": "2016-11-07 16:50:08",
      "receipt_date": null
    },
    {
      "id": 2,
      "user_store_id": 52800,
      "purchase_id": null,
      "identifier": "10298372070213183800555",
      "status": "Failed",
      "created_at": "2016-11-09 10:29:36",
      "receipt_date": null
    }
  ],
  "meta": {
    "max_number_of_requests_per_day": 10000,
    "remaining_number_of_request": 9997,
    "time_in_epoch_second_till_reset": 1478822400
  }
}

Notice that each receipt has it's status which can be one of the following:

  1. "Processing" - This means our data bot will try to get the data in the near future. For Walmart this status can be present for several days. It really depends on Walmart.
  2. "Done" - Receipt data is fetched and stored in our database. Also purchase_id is populated.
  3. "Failed" - There were one or more errors while trying to get the receipt data. This might include: Invalid receipt date or receipt id, databot has timed out when fetching the data (the store server is down for some reason or currently is not processing receipt requests), other bad requests made to the store server. We will try to scrape these "Failed" receipts whenever databot for this user store is activated for scraping event though the status is marked as "Failed".

You can also get individual user store receipts by calling:

request sample (GET)

https://api.iamdata.co/v1/users/USER_ID/stores/USER_STORE_ID/receipts/RECEIPT_ID?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Stores Supporting Online Purchase Streams

One store connection can pull "online", "loyalty" or "online & loyalty" purchase stream. It varies from store to store (marked in list below).
E.g., Walmart store connection (store id: 3) pulls both online and loyalty purchase stream. On the other hand, Safeway (online) and Safeway (loyalty) are separate infrastructures, i.e., separate store ids, 50 & 1041 respectively. In order to pull both "online" and "loyalty" purchase streams, two connections are required.

Stores not listed below are supported through paper receipt images & bar code scans.

User Scans

Barcode Scan

It is possible to associate barcode to IM user account. API response contains product data for provided barcode. Execution might take up to 15 seconds, depending on whether barcode exists in database or IM services must gather data for uploaded barcode.

request sample (POST)

https://api.iamdata.co/v1/users/USER_ID/barcode?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

POST data

{
  "bar_code": "021130126026",
  "bar_code_type": "UPC-A"
}

Receipt Upload

Another way (besides scrape bots) of associating purchases with user. If store does not have online shopping section, you can take a photo of your receipt and upload it via receipt upload engine. Receipt is typically transcribed within 24h time frame.

If receipt is long, split it, and upload in multiple upload requests, each having same "receipt_id". Receipt image can be splitted in up to 10 components.

request sample (POST)

https://api.iamdata.co/v1/users/USER_ID/receipt?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

POST data

{
  "receipt_id": "UNIQUE_RECEIPT_ID",
  "image": "BASE64_ENCODED_IMAGE"
}

Products & Purchases

About Purchases

Purchase represents entity that is equivalent to invoice, delivered into IM database via harvesting bots. It is composed of basic invoice properties (purchase date, store [where purchase is made], total & tax) and individual purchase items (name, quantity, unit price, image, UPC, etc). Purchase item properties may vary from store to store. However, purchase item name & price are pretty much always captured by harvesting bots.

About Products

Product is entity non related to actual purchase items, i.e., harvesting bots. Each is composed out of many properties. Name, UPC/EAN/ISBN, image, ingredients, nutrition card, etc.. Often, same product has different (purchase) name in various stores. Information Machine works hard on bringing all these purchase items under the same parent, i.e., Product. Each Purchase Item is connected to a Product, which expands its rudimentary set of properties delivered via scrape bots.

Purchase Query & Filter Fields

Check our Swagger documentation, "UserPurchases" section, for available purchase (purchase item) query parameters and filters.

Product Query & Filter Fields

Check our Swagger documentation, "Products" section, for available product query parameters and filters.

Lookup

About

This set of API methods expose lookup based (id -> name) tables.

Alternative Types

List all alternative type IDs available.

request sample (GET)

https://api.iamdata.co/v1/product_alternative_types?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Units of Measurement

List all unit of measurement IDs available.

request sample (GET)

https://api.iamdata.co/v1/units_of_measurement?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Categories

List all category IDs with parent/child structure.

request sample (GET)

https://api.iamdata.co/v1/categories?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Nutrients

List all available nutrients in IM infrastructure.

request sample (GET)

https://api.iamdata.co/v1/nutrients?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Stores

List all stores in IM infrastructure. Those with "can_scrape=true" are eligible for purchase harvesting via scrape bots. See "UserStores" swagger section.

request sample (GET)

https://api.iamdata.co/v1/stores?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Tags

List all supported tags (organic, gluten free, vegan, etc.).

request sample (GET)

https://api.iamdata.co/v1/tags?client_id=YOUR_API_KEY&client_secret=YOUR_API_SECRET

Webhooks

User Purchases Webhook

If you do not want to "poll" our API on a regular basis to get the newest purchase data from your users, then you can use our Webhook and we will notify you when new purchase data is available.

So, as soon our data bots finish scraping the users purchase history you will be notified with scraped data.

To configure this webhook go to your Dashboard page, select Webhooks tab, enable "User Purchases Webhook" and enter your URL where would you like us to send the purchases to. Finally click Save button and that's it.

You will recieve the model as seen below

"idempotency_key" uniquely identifies every webhook sent from Information Machine servers.

POST data model

{
  "user_store_id": 0,
  "user_id": "string",
  "idempotency_key": "string",
  "purchase_data": {
    "purchased_items": [
      {
        "product_id": 0,
        "name": "string",
        "purchase_history": [
          {
            "invoice_id": 0,
            "store_id": 0,
            "store_name": "string",
            "harvested_name": "string",
            "quantity": 0,
            "price": 0,
            "unit_price": 0,
            "current_price": 0,
            "purchase_date": "2017-03-10 15:41:58",
            "recorded_at": "2017-03-10 15:41:58"
          }
        ],
        "product_timestamps": {
          "upc_resolved_at": "2017-03-10 15:41:58",
          "recorded_at": "2017-03-10 15:41:58"
        },
        "product_identifiers": {
          "upcs": [
            "string"
          ],
          "plus": [
            "string"
          ]
        },
        "product_details": {
          "nutrients": [
            {
              "dvp": 0,
              "value": 0,
              "name": "string",
              "id": 0
            }
          ],
          "recipes": [
            "string"
          ],
          "plus": [
            "string"
          ],
          "visibility_count": 0,
          "score": 0,
          "amazon_link": "string",
          "manufacturer": "string",
          "ingredients_count": 0,
          "large_image": "string",
          "small_image": "string",
          "serving_size_in_grams": 0,
          "serving_size_unit": "string",
          "servings_per_container": "string",
          "serving_size": "string",
          "ingredients": "string",
          "weight": "string",
          "description": "string",
          "brand": "string",
          "upc": "string",
          "tags": [
            "string"
          ],
          "category": "string",
          "category_id": 0,
          "name": "string",
          "size_info": {
            "size": 0,
            "size_uom": "string",
            "package_count": 0,
            "package_count_uom": "string"
          },
          "id": 0
        }
      }
    ],
    "invoices": [
      {
        "id": 0,
        "store_id": 0,
        "store_name": "string",
        "total": 0,
        "total_without_tax": 0,
        "tax": 0,
        "purchase_date": "2017-03-10 15:41:58",
        "recorded_at": "2017-03-10 15:41:58",
        "order_number": "string",
        "receipt_id": "string",
        "receipt_image_url": "string",
        "receipt_image_urls": [
          "string"
        ],
        "user_store_receipt": {
          "id": 0,
          "user_store_id": 0,
          "created_at": "2017-03-10 15:41:58"
        }
      }
    ]
  }
}

Do note that property purchase_data contains the same model you would get if you called our API endpoint /v1/users/{user_id}/purchases_product_based


Store Connection Credentials Webhook

If you do not want to "poll" our API for credentials status, you can subscribe to webhook. As soon our data bots resolve credentials status, you will be notified. Also, we'll notify on every future store connection status update.

Credentials status webhook will be triggered, as soon as IM Webhook infra is aware of credentials status value (“Verified”, “Invalid” or “Unknown”). Webhook is HTTP POST request, with “text/json” content type, targeting address defined in IM API Web Dashboard. POST payload is object composed of following properties:

  • store_connection_id (type:long) – ID of store connection (returned by IM API).
  • credentials_status (type:string) – Status of credentials. “Verified”, “Invalid” or “Unknown”.
  • account_locked (type:bool) – Lock status of store connection.
  • user_id (type:string) – ID of user in IM system.
  • account_lock_code (type:short, NULL-able) – Lock code type (details below).
  • unlock_url (type:string) – URL you should open to unlock account.
  • username (type:string) - Store connection username.
  • store_id (type:long) - Store ID
  • idempotency_key (type:string) - Unique request ID.
account_lock_code property indicates type of lock enforced by retailer. Should be available if account_locked property is set to true. List of available codes below.

codedescription
2Security Question
3Image Captcha
4Incomplete User Profile
5User Account Deactivated
6Access Denied
7Password Reset Required
IM Webhook service will try to reach destination three times, before final abort.

You will recieve the model as seen below

POST data model sample

{
	"store_connection_id":113216,
	"credentials_status":"Verified",
	"account_locked":false,
	"user_id":"id_34234_22",
	"account_lock_code":NULL,
	"unlock_url":"https://api.iamdata.co/mfa?uside=userid",
	"username":"someemail@hotmail.com",
	"store_id":4,
	"idempotency_key":"5487-4",
}

OnDemand Refresh Result Webhook

If you do not want to "poll" our API for result of OnDemand refresh, you can subscribe to webhook. As soon as requested connection is refreshed, you will be notified with results.

OnDemand refresh webhook will be triggered as soon as refresh is completed to notify you how with harvested purchase count. Webhook is HTTP POST request, with “text/json” content type, targeting address defined in IM API Web Dashboard. POST payload is object composed of following properties:

  • store_connection_id (type:long) – ID of store connection (returned by IM API).
  • refresh_request_id (type:string - GUID) – OnDemand refresh request id.
  • purchase_count (type:int) – Count of harvested purchases.
  • idempotency_key (type:string) - Unique request ID.
IM Webhook service will try to reach destination three times, before final abort.

You will recieve the model as seen below

POST data model sample

{
	"user_store_id": 2108076,
	"refresh_request_id": "59feb8ac-9373-46c3-ad03-dec9381f7f14",
	"purchase_count": 0,
	"idempotency_key":"2108076-1"
}