Syncing your Data using a JSON Feed

Setup your JSON Feed to sync your data with Clerk.io.
Platforms:

Introduction

This guide explains how to set up your JSON Feeds to sync your data with Clerk.io.

If you are using one of our supported platform, you do not need to set up a JSON Feed.

Our older JSON Feed is still supported, but we recommend using the new JSON Feed for new integrations.

The new JSON Feed makes it easier and more efficient for you to sync your data with Clerk.io.

This is done by being able to sync your data in separate files/endpoints.

And supporting pagination and incremental updates for large data sets. Data Sync Settings panel for JSON Feed V2

Setup

To get started, go to my.clerk.io and click “Data” in the left-side menu.

Scroll down to “Data Sync Settings”, and choose “Clerk.io JSON Feed V2” from the Sync Method dropdown in the top-right corner.

Insert the URL’s for your JSON Feeds for your Products, Categories, Orders, Customers and Pages, in Data Sync Settings.
See Here for how to organize the JSON Feeds for each entity.

We support content-type: application/x-ndjson or application/json.

Security

⚠️ Your must be kept secure, so the endpoints you implement must require authentication before they are deployed!

For this reason, HTTPS is forced on all URLs.

We recommend that the JSON feed only accepts an SSL encrypted connection and uses HTTP Authentication if possible.

When querying your endpoint, the Clerk platform will send an authorisation header on every HTTP request, that you need to verify:

X-Clerk-Authorization : Bearer THE_TOKEN

You can verify the token with a POST request to the token/verify endpoint https://api.clerk.io/v2/token/verify:

{ "token": "THE_TOKEN", "key": "your_store_public_key"}

See our api documentation for more details

Incremental Updates

Products, Categories, Customers and Pages are deleted if not present in the JSON Feed. Use the modified_after query parameter to enable incremental updates and keep existing data that has not been modified. See Below

Orders are always retained

By default Clerk.io will delete Products, Categories, Customers and Pages that are not present in the JSON Feed.
When having big data sets, this can be a problem, as it can take a long time to sync all data.
In this case, you can send only the data that has bene modified after a given date, which can be done by using the modified_after query parameter. See Below

Orders are always retained. These can be deleted by sending a DELETE request to the orders endpoint.

Query Parameters

To support pagination and incremental updates for large data sets, we support the following query parameters:

limit={{limit}}
offset={{offset}}
modified_after={{modified_after}}

Paging size with {{limit}} and {{offset}}

The limit query parameter can be used to limit the amount of entities returned in each request.
The default value is 25 and the maximum value is 1000.\

When using the limit query parameter, you must also use the offset query parameter.
As we increase the offset by the limit until no more data is returned, either by returning an empty array or by an empty response.

Using the modified_after query parameter

Retaining only works if {{modified_after}} is present in the URL and a value in Incremental Time.

To retain existing data, you can use the modified_after query parameter.

{{modified_after}} needs to be present in the URL as well as a value in Incremental Time in Data Sync Settings for this to take effect.

This only effects each URL individually, so if you have multiple URLs, you need to add {{modified_after}} to each URL.

To delete Products, Categories, Customers and Pages that are not present in the JSON Feed, you can send a DELETE request to the products endpoint, categories endpoint, customers endpoint and pages endpoint.

JSON Feed Structure

We only support attributes of the types: int, float, str, array and bool.

Each JSON Feed should be an array of objects, where each object represents a single entity (product, category, order, customer or page). Objects in the Products URL should represent products, objects in the Categories URL should represent categories, etc.

Each object must contain the mandatory fields for the entity type, and can optionally contain any extra attributes.

Mandatory fields are marked with a * in the tables below.

Products

The JSON Feed for products should be an array of objects, where each object represents a single product.

Table of mandatory and some examples of optional fields for a product object:

AttributeRequiredTypeDescription
id*int/strThe product ID, which should be unique for each product
name*strThe product name.
description*strThe product description.
price*floatThe product current selling price.
list_pricefloatThe product original list price. Useful to show discounts.
image*strThe full URL for the product image. When used for thumbnails we recommend a maximum image size of 200x200px.
url*strThe product URL.
categories*arrayAn array of category IDs that the product belongs to.
created_at*intThe UNIX timestamp of when the product was created.
brandstrThe product brand.
color_namesarrayAn array of color names for the product.
color_codesarrayAn array of color codes for the product.
reviews_amountintThe amount of reviews for the product.
reviews_avgfloatThe average review score for the product.

Example JSON Feed for products

[
  {
    "id": 135,
    "name": "Lightsaber",
    "description": "Antique Rebel Lightsaber",
    "price": 99995.95,
    "image": "https://galactic-empire-merch.com/images/a-r-lightsaber.jpg",
    "url": "https://galactic-empire-merch.com/antique-rebel-lightsaber",
    "brand": "Je'daii",
    "categories": [987, 654],
    "created_at": 1199145600,
    "color_names": ["Green","Red"],
    "color_codes": ["#7CFC00","#FF3131"],
    "reviews_amount": 164,
    "reviews_avg": 4.8
  },
  {
    "id": 261,
    "name": "Death Star Deluxe",
    "description": "Death Star - Guaranteed idiot proof",
    "price": 99999999999999.95,
    "image": "https://galactic-empire-merch.com/images/death-star.jpg",
    "url": "https://galactic-empire-merch.com/death-star",
    "brand": "Imperial Inc.",
    "categories": [345678],
    "created_at": 1197565600
  }
]

Categories

The JSON Feed for Categories should be an array of objects, where each object represents a single category.

Table of mandatory and some examples of optional fields for a category object:

AttributeTypeDescription
id*int/strThe category ID, this should be unique for each category.
name*strThe category name.
url*strThe category URL.
subcategories*arrayAn array of category IDs that are subcategories of this category.
imagestrThe full URL for the category image.
descriptionstrThe category description.

Example JSON Feed for Categories

[
  {
    "id": 1,
    "name": "Imperial Goods",
    "subcategories": [42, 25],
    "url": "https://galactic-empire-merch.com/imperial-goods"
  },
  {
    "id": 42,
    "name": "Tatooine",
    "subcategories": [],
    "url": "https://galactic-empire-merch.com/imperial-goods/tatooine"
  },
  {
    "id": 25,
    "name": "Coruscant",
    "subcategories": [],
    "url": "https://galactic-empire-merch.com/imperial-goods/coruscant"
  }
]

Orders

Orders are stored as a log and thus never deleted. If you want to delete an order, you can do so by sending a DELETE request to the orders endpoint.

The JSON Feed for Orders should be an array of objects, where each object represents a single order.

Table of mandatory and some examples of optional fields for a order object:

Example JSON Feed for Orders

[
  {
    "id": 123458,
    "customer": 789,
    "email": "vader@the-death-star.com",
    "products": [{"id":456,"quantity":1,"price":200.00}, {"id":789,"quantity":2,"price":120.00}],
    "time": 1389871120
  },
  {
    "id": 123456,
    "customer": 456,
    "email": "obi.wan@kenobi.me",
    "products": [{"id":456,"quantity":1,"price":200.00}, {"id":789,"quantity":2,"price":120.00},{"id":123,"quantity":2,"price":60.00}],
    "time": 1389870977
  },
  {
    "id": 123457,
    "customer": "",
    "products": [{"id":789,"quantity":2,"price":120.00}],
    "time": 1389871090
  }
]

Customers

The JSON Feed for Customers should be an array of objects, where each object represents a single customer.

Table of mandatory and some examples of optional fields for a customer object:

AttributeTypeDescription
id*int/strThe customer ID, this should be unique for each customer.
name*strThe customers full name.
email*strThe customer email.
subscribed*boolBoolean indicating whether the customer has subscribed to newsletters. This must be true for Clerk.io to send marketing emails to this customer.
zipstrThe customers zip code.
genderstrThe customers gender
ageintThe customers age.
is_b2bboolBoolean indicating whether the customer is a business customer.

Example JSON Feed for Customers

[
  {
    "id": 135,
    "name": "Luke Skywalker",
    "email": "luke@rebels.org",
    "subscribed": true,
    "gender": "male",
    "zip": "1134",
    "is_b2b": "false"
  },
  {
    "id": 165,
    "name": "Darth Vader",
    "email": "vader@empire.com",
    "subscribed": false,
    "gender": "male",
    "age": 45,
    "interests": ["lightsaber", "force"],
    "is_b2b": true
  }
]

Pages

The JSON Feed for Pages should be an array of objects, where each object represents a single page.

Table of mandatory and some examples of optional fields for a page object:

AttributeTypeDescription
id*int/strThe page ID, this should be unique for each page.
type*strThe type of the content. Used to separate different types of pages such as CMP pages, blog posts and landing pages.custom.
url*strThe URL of the content.
title*strThe title of the content.
text*strThe text of the content.
imagestrThe full URL for the content image.

Example JSON Feed for Pages

[
  {
    "id": 135,
    "type": "cms",
    "url": "https://galactic-empire-merch.com/imperial-goods/tatooine",
    "title": "Open Hours",
    "text": "The main text about our opening hours..."
  },
  {
    "id": 1354,
    "type": "blog",
    "url": "https://galactic-empire-merch.com/imperial-goods/tatooine",
    "title": "New Blog Post",
    "text": "The main text about our opening hours...",
    "keywords": ["blog", "post", "new"]
  }
]

Multi-language feed

Please note that multi-language feed is not generally recommended. We do not support all features e.g. Pages import. We do not generally advice this due to how easy it is to mess up the indexation if the requirements are not met.

Example JSON Feed for products

[
  {
    "id": 135,
    "name": {"english":"Lightsaber","spanish":"Sable de luz", "italian":"Spada laser"},
    "description": {"english":"Antique Rebel Lightsaber","spanish":"Sable de luz rebelde antiguo","italian":"Antica spada laser ribelle"},
    "price": 99995.95,
    "image": {"english":"https://galactic-empire-merch.com/images/a-r-lightsaber.jpg","spanish":"https://galactic-empire-merch.com/es/images/a-r-lightsaber.jpg","italian":"https://galactic-empire-merch.com/it/images/a-r-lightsaber.jpg"},
    "url": {"english":"https://galactic-empire-merch.com/antique-rebel-lightsaber","spanish":"https://galactic-empire-merch.com/es/antique-rebel-lightsaber","italian":"https://galactic-empire-merch.com/it/antique-rebel-lightsaber"},
    "brand": "Je'daii",
    "categories": [987, 654],
    "created_at": 1199145600,
    "color_names": ["Green","Red"],
    "color_codes": ["#7CFC00","#FF3131"],
    "reviews_amount": 164,
    "reviews_avg": 4.8
  },
  {
    "id": 261,
    "name": {"english":"Death Star Deluxe","spanish":"Estrella de la Muerte de lujo", "italian":"La Morte Nera Deluxe"},
    "description": {"english":"Death Star - Guaranteed idiot proof","spanish":"Estrella de la Muerte: a prueba de idiotas garantizada","italian":"Morte Nera - A prova di idiota garantita"},
    "price": 99999999999999.95,
    "image": {"english":"https://galactic-empire-merch.com/images/death-star.jpg","spanish":"https://galactic-empire-merch.com/es/images/death-star.jpg","italian":"https://galactic-empire-merch.com/it/images/death-star.jpg"},
    "url": {"english":"https://galactic-empire-merch.com/death-star","spanish":"https://galactic-empire-merch.com/es/death-star","italian":"https://galactic-empire-merch.com/it/death-star"},
    "brand": "Imperial Inc.",
    "categories": [345678],
    "created_at": 1197565600
  }
]