Platform Sync (previously Hosted GET)

Created by Anthony Young, Modified on Mon, 20 Jun, 2022 at 6:13 PM by Anthony Young

The Platform Sync, previously known as the Hosted GET connector lets you expose Data Source rows from your own systems, for direct access by your app.

The key advantage of the Platform Sync connector is that you can control the data rows being exposed to the app, letting you filter rows down to the individual app user level as desired. For ERP, CRM, and other enterprise systems this is important as the security/permissions around "who gets to see what" is critical.

A secondary benefit is that your data does not need to be pushed our platform, so any data sovereignty concerns you may have are resolved by this connector.

One final benefit is that Data Sources that run off of Platform Sync connectors are not subject to the standard 50,000-row limit, since the rows are delivered directly to the device.

NOTE: There are still performance limits on the app side when the rows become large - this varies across Android and IOS, and also depends heavily on the device specifications.

As a rule of thumb, performance is affected by the overall file size of the data source more than the number of rows.

For example, 5,000 rows containing say 100k of text data per row can be expected to run slower than 50,000 rows that have a total of 1k of data per row (e.g. columns containing say just numbers and a few short text strings).

Additionally, the use of "heavy" data sources in many screens that can be navigated sequentially by the user will result in a compounding performance decrease, as those big data sources can get progressively accumulated in device memory.

So with Platform Sync, you really need to test on the app to find what is the best row count and screen mix for your specific data set.

It is assumed that you have software development resources to create the web service needed by the Platform Sync connector.

If not, we can put you in contact with a consulting partner on request.

Setting Up a GET Connector
How the GET Connector Works
GET Request
GET Response
API Usage Example
Example Code
Troubleshooting / Testing

Setting Up a GET Connector

For test purposes, we recommend you use a REST API development service like - https://www.mockable.io

Create a Data Source as normal or select an existing Data Source for use.
On the Rows page of your Data Source, manually define and organize your Data Source's columns to match the output of your web service.
The order of the columns must match up with what your web service returns in order for any column references to work.
If you change column orders on your web service response at any stage, you will need to update the columns on the Rows page to match.
On the Settings page of your Data Source, make sure that you set the External Id field with some value for identifying this Data Source.
Remember this External Id value, since you will need it when you work through the GET web service details later in this document.
Also on the Settings page, add a Platform Sync connector via the "Add Connector" button.

And, specify the web address of your web service endpoint in the provided Target GET URL field.
You can also set a Global DataSource GET URL on your organization and specify that the connector uses it - this allows the reuse of the same service endpoint across multiple Data Sources.

How the Platform Sync Connector Works

When the app discovers that a Data Source has a Platform Sync connector, it performs an HTTP GET request against the target web service URL.

The GET request contains the External Ids of the Data Source's being requested along with other identifying information.
Included in the request is are default query-string parameters, namely your account's Private Token (useful for authenticating the request), along with the logged in-app user's email address (useful for filtering the rows based on the user's access).

The app will issue GETs to your service whenever it attempts a data synchronisation.

Requests are issued by the app in an itemized manner - one request per Data Source.

IMPORTANT!
You MUST expose your web service endpoint as SSL secured HTTPS to ensure that app requests to your service are protected.

Attached to the bottom of this document is a C# file that outlines the request and response structures as simple classes.

Even if you are not using C#, reviewing the properties and structure in code form can be very helpful in getting started with your own Platform Sync service.

GET Request

GET requests are issued by the app to your service, passing the values below as query string parameters.

Most of the settings for the integration with your Platform Sync API can be obtained from the Organization Setup screen. This includes the Integration Key (Private Token) and Provider ID (Company ID).

The parameters passed on the request are as follows:

Param Name	Data Type	Required	Description
ids	String	Yes	Comma-separated list of External Ids for each Data Source that the app wants to synchronise. You set the External Id on your Data Sources via the App Builder>Data Sources>Settings page in our platform. The image below shows where the external ID for the data source is set.
providerId	Integer	Yes	Your unique Provider Id found on your Organisation Setup page (Organisation&Users>Organisation Setup) Use this with the IntegrationKey and Email to authenticate the request. ProviderID is the same as Company ID.
integrationkey	String	Yes	Your unique Private Token found on your Organisation Setup page (Organisation&Users>Organisation Setup). Use this with the Provider and Email to authenticate the request.
email	String	Yes	Email address of the logged in-app user making the request. Use this to identify the user in your system as well as to apply any user-specific filtering you wish against the returned Data Source rows.
deviceid	GUID	Yes	The unique identifier of this user's device registered on our platform. Useful for tracking which device the request originated from.
lastupdated	DateTime (YYYY-MM-DDTHH:MI:SS)	Yes	The last time that the app checked for Data Source updates. Use this to implement incremental row updates, passing back just the new/updated and delete rows since the time the app checked in. This is recommended for large Data Sources to save server resources and prevent exhausting mobile data allocations. You can also use this value to determine whether there is anything at all to update since the last time the app pinged your service. If nothing has changed, then you can simply return a response with no DataSources property defined. The first time the app checks for rows, the LastUpdated value will be at least 1 year in the past. We suggest that you monitor for this and return the full set of data in the Rows property of your response.
lastupdateds	String	No	Comma-separated list of DateTime values in the format "YYYY-MM-DDTHH:MI:SS" that represents the last time the app requested an update for the corresponding data source. The values are listed in the same order as the external ids in the Ids parameter. Each date in the list represents the last time that the app checked for Data Source updates for the specific data source. Use this to implement incremental row updates, passing back just the new/updated and delete rows since the time the app checked in. This is recommended for large Data Sources to save server resources and prevent exhausting mobile data allocations. You can also use these values to determine whether there is anything at all to update since the last time the app pinged your service. If nothing has changed for the related data sources, then you can simply return a response with no DataSources property defined. The first time the app checks for rows for a specific data source, the related last updated value will be left blank in its position in the comma-separated list. We suggest that you return the full set of data in the Rows property of your response in this case.

GET Response

IMPORTANT!
Your web service MUST return a response in the following structure.  Failure to comply with this format will result in failure to download rows on your app.
Your response MUST be in JSON format.

We also support gzip and deflate compression on responses.
This is recommended for large data sets since text data compresses very well and thus will conserve mobile data allocations and massively reduce download time.
Our support for compression is identified through the "Accept-Encoding" header being set by the app on its REST requests.

Your web service/server should encode the entire response stream using gzip or deflate, which will then be recognised by our app code.
This is usually an automatic behaviour at the webserver level, though you should check that your server configuration has enabled compression for JSON content.

In terms of how to apply response encoding, this is beyond our support scope - you'll need to confirm this with your software developers.
If you are using Microsoft IIS as your web server.

Rows Response (<DRowsResponse>)

Input Name

Data Type

Required

Description

DataSources

Collection of DataSource

Collection containing the DataSources that have been requested.

If nothing has changed since the LastUpdated received from the app request, then you can simply return a response with no DataSources property defined or, if returning multiple data sources, then no Rows, NewRows and DeletedRows properties set on those you do not wish to update.

LastUpdated

DateTime
(YYYY-MM-DDTHH:MI:SS)

Yes

The last time that the rows for these Data Sources were updated. Generally, we recommend that this should be the current date and time on your host server at the time of processing the request.

This value is stored by the app and sent exactly as received in the LastUpdated and LastUpdateds fields of the next GET request made by the app.

This gives a central point of time for incremental updates in particular since the app will echo your server times rather than use its own unreliable device time.

The “T” in the required format is a delimiter.

Data Source (<DataSource>)

Below is an example of a functioning GET Request URL, populated from hidden fields where the integration key; providerID; data source name are dynamically populated.

https://www.************.com/api/users?email={{userEmail}}&integrationkey={{privateToken}}&providerid={{companyProviderID}}&ids={{dataSourceExternalIdsCommaSeparated}}

Input Name	Data Type	Required	Description
Id	String	Yes	The External Id of the DataSource for which the associated Row data applies. Must match the External Id configured in the Data Source -> Settings page on the platform.
Rows	Collection of Row	No	A collection of Rows that will completely replace all current Data Source rows on the app. If you specify Rows, then any values in NewRows and DeletedRows will be ignored.
NewRows	Collection of Row	No	A collection of Rows that should be added/updated onto the existing Data Source rows on the app. Use this field to perform incremental inserts/updates of your Data Source.
DeletedRows	Collection of Row	No	A collection of Row Items that should be deleted from the existing Data Source rows on the app. Use this field to perform incremental deletes of your Data Source. Each Row should contain a single Val element that is your unique identifier for the Row to delete.

Row (<Row>)

Each Row is a collection of Val (<Val>) items, which are the individual column values for that Row.
Every Row must contain at least two Vals and must have as many Vals as there are Headers (columns).

The first Val should be the unique identifier or key that will be used as the answer value should this Data Source row be selected in a Form. The first Val is also critical as the key for performing incremental deletes through the DeletedRows field mentioned above.
The second Val should be the default displayable label text of the row. You can override this default by configuring the Display Options field in the Data Source "Settings" page.

Input Name	Data Type	Required	Description
Val	String	Yes	A column value for this row can contain any string value. If you want this column to display an icon/image in the mobile app, then specify the Val as an HTTP URL to the image in question. The image linked must be in PNG or JPG/JPEG format.

If the GET is unsuccessful, your web service should return a 400 level HTTP status.

API Usage Example

Given that this connector is REST-based, you can test your web service directly via your web browser using a REST plugin like the Postman plugin for Google Chrome.

In this example, we assume you have set the "Target GET URL" on your Platform Sync Connector to be https://yourdomain.com/xx/rowsearch.
We also assume that the Data Source has an External Id of "JOBS"

GET Request (made by the app, with user@somewhere.com as the logged-in user)

GET https://yourdomain.com/xx/rowsearch?email=user@somewhere.com&integrationkey=xxxxxxxxxxxx&providerid=1&ids=JOBS,JOBS-DISS&lastupdated=2013-02-12T14:24:09

GET Response

{   "DataSources":[      {         "Id":"JOBS",         "Rows":[            [               "10011",               "10011 - Test Generic Project",               "TEST",               "12345AA",               "19 Nov 2013",               "12 Somewhere Road"            ],            [               "ZDS654",               "ZDS654 - Maintenance Costs to Excavator",               "TEST",               "",               "27 Nov 2013",               "24 Somewhere Else Road"            ],            [               "ZDF662",               "ZDF662 - Maintenance Costs for Hitachi Excavator",               "TEST2",               "",               "28 Nov 2013",               "182 Another Place, Somewhere"            ]         ],         "NewRows":[          ],         "DeletedRows":[          ]      },      {         "Id":"JOB-DISS",         "Rows":[            [               "10011-0201",               "Project Management",               "10011",               "Y"            ],            [               "10011-0202",               "Quality Control",               "10011",               "Y"            ],            [               "10011-0251",               "Electrical Design",               "10011",               "Y"            ]         ],         "NewRows":[          ],         "DeletedRows":[          ]      }   ],   "LastUpdated":"2014-03-11T15:45:50.4818098+10:00" }

Example Code

Attached to the bottom of this document is a C# file that outlines the request and response structures as simple classes.

Even if you are not using C#, reviewing the properties and structure in code form can be very helpful in getting started with your own Platform Sync service.

Troubleshooting / Testing

Once you have set up a Platform Sync connector on your Data Source, the Rows page for that Data Source will automatically attempt to pull back rows from the given GET URL.

The Rows page makes the same request as the app would, using your current logged-in user as part of the request.

This is a quick and easy way to see if your Platform Sync web service is understanding the GET request and is returning rows in the correct format.

What to do when a Platform Sync works on my IOS app, but not on Android?

This is almost certainly due to the way that Android treats SSL certificates - essentially Android is more strict about SSL certificate chains.

SSL certificate chains involve the primary SSL certificate for your web service endpoint domain, along with any intermediate certificates issued by your Certificate Authority.

If you attempt to connect to a web address that features an incomplete SSL chain, then Android will fail the connection.

This is likely what is happening with requests by your Android devices to your Platform Sync web domain.

To test what we mean, attempt to browse to your Platform Sync service URL from your default Android web browser.

You will likely get a connection error with text such as "the certificate authority is invalid".

You can further verify your SSL configuration using a testing service like SSL Labs:

https://www.ssllabs.com/ssltest/

If your SSL installation has issues, you will see warnings such as "chain incomplete" on the SSL Labs test results.

To resolve the issue, you need to ensure that the server running the Platform Sync web service has the intermediate certificates from your Certificate Authority installed.

For example, if you use GoDaddy, the best place to start would be here:

https://support.godaddy.com/help/article/868/what-is-an-intermediate-certificate?countrysite=au

Once you have the correct certificate chain in place, the Platform Sync data sources will sync to your Android devices, since they can now connect to your web service.

You can confirm the certificate chain by simply checking the Platform Sync URL in your Android web browser - it should no longer show certificate errors when resolved.

HostedGET.cs

(4.4 KB)