iFormBuilder Data Feeds

What's covered:

• What are Data Feeds?
• What is metadata?
• What info is included in metadata?
• Location Data Overview
• Excel Data Feed
• XML Data Feed
• JSON Data Feed
• RSS Data Feed
• ATOM Date Feed
• Data Feed Parameters
• Complex Data Structures
• Data Feed Integration

What are Data Feeds?

iFormBuilder Data Feeds is a method of exporting your data which includes metadata that will not show in the Data Views. You can use Data Feeds to consume data in other backend systems and reporting platforms.

The different Data Feeds include:

• Excel
• XML
• JSON
• RSS
• ATOM

What is metadata?

Metadata is information automatically captured in the background by your device. Data such as CREATED_DY, CREATED_DATE, CREATED_LOCATION and more are all recorded and can be accessed via any of the iFormBuilder Data Feeds.

What info is included with metadata?

The following information is captured automatically by your device in the metadata.

• ID: This is the ID given to the record by the database server upon upload.
• PARENT_RECORD_ID: This is the Record ID of the Parent Form.
  • PLEASE NOTE: This will show as zero unless it has associated Subform records.
• PARENT_PAGE_ID: This is the Page ID of the Parent Form.
  • PLEASE NOTE: This will show as zero unless it has associated Subform records.
• PARENT_ELEMENT_ID: This is the Subform element that leads to the particular subform. 
  • PLEASE NOTE: This will show as zero unless it has associated Subform records.
• CREATED_DATE: This is the date the record was created either the device or on the server.
• CREATED_BY: This is the user that created the record either the device or on the server.
• CREATED_LOCATION: This is the location where the record was created.
  • PLEASE NOTE: If created on the server, the field will say "Server".
• CREATED_DEVICE_ID: This is the Device ID of the device where the record was created.
• MODIFIED_DATE: This is the last time the record was modified either on the device or on the server.
• MODIFIED_BY: This is the user that last edited the record either the device or on the server.
• MODIFIED_LOCATION: This is the location where the record was last modified.
  • PLEASE NOTE: If created on the server, the field will say "Server"
• MODIFIED_DEVICE_ID: This is the Device ID of the device where the record was last modified.
• SERVER_MODIFIED_DATE: This is the date the record was last uploaded/updated on the server. 

Location Data Overview

While most of the metadata above it pretty straightforward, the CREATED_LOCATION and MODIFIED_LOCATION represent a number of different measurements in a single column. Please reference the example below to help parse the data.

Parameters:

latitude:longitude:altitude:horizontalAccuracy:verticalAccuracy:speed:course:timestamp

Data:

30.398019:-97.737039:0.000000:80.000000:-1.000000:-1.000000:-1.000000:1282938543.81998

PLEASE NOTE: You can use Unix Time Converters to convert the timestamp to a date and time.

Excel Data Feed

The Excel Data Feed is an Excel export of your data that includes all the metadata (such as location, user, created date, etc) captured in the background of your device. This metadata will be displayed in columns A through M on your Excel spreadsheet while user-defined columns will begin in column N.

The data returned will show the data column names that you defined for each element in the Form Builder as well as the key values used for options in your Option Lists. XLS+ will show you the sort order or index value of the options in your Option Lists.

PLEASE NOTE: Microsoft Excel limits the number of columns to 256 in an XLS file. If your data exceeds that, please export with XLSX. This Customer Success Center article will show you how to do so.

Click here for instructions on how to export your data via the Excel Data Feed.

XML Data Feed

The XML Data Feed is an XML representation of your data that can be integrated with other backend Systems. XML stands for eXtensible Markup Language. It was designed to store and transport data and be both human-readable and machine-readable. 

Click here for instructions on how to export your data via the XML Data Feed.

JSON Data Feed

JavaScript Object Notation or JSON is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute-value pairs and array data types. The JSON Data Feed is a representation of your data in this format.

Click here for instructions on how to export your data via the JSON Date Feed.

RSS Data Feed

RSS is a type of web feed which allows users to access updates to online content in a standardized, computer-readable format. 

The RSS Data Feed can be consumed in backend systems much like the other iFormBuilder Data Feeds. To utilize the RSS feed you must open up the RSS feed from iFormBuilder.com and copy the URL from the address bar. When the backend system you are using asks for the link to the RSS feed, paste in the URL and append the username and password at the end of the URL String:

&USERNAME=test&PASSWORD=testpassword

Click here for instructions on how to export your data via the RSS Date Feed.

ATOM Data Feed

The name Atom applies to a pair of related Web standards. The Atom Syndication Format is an XML language used for web feeds, while the Atom Publishing Protocol is a simple HTTP-based protocol for creating and updating web resources.

The Atom Data Feed can be consumed in Backend systems in the same way as RSS Feed. When the backend system you are using asks for the Data Feed URL, go into the ATOM Feed in in your iFormBuilder server and copy the URL and paste it in the backend System with the Username and Password and append it to the end of the URL String:

&USERNAME=test&PASSWORD=testpassword

Data Feed Parameters

The XML and JSON Data Feeds enable the use of three different parameters. You can (and should) use these parameters to run smarter queries when pulling data from the server.

PLEASE NOTE: To use these options, right-click the XML or JSON feed icon and copy the link into the browser. Use the three different parameters together to run different requests TYPE, SINCE_DATE, SINCE_ID.

Ex. &TABLE_NAME=_data99999_data_feed_test&TYPE=TOP&SINCE_DATE=2011-01-10&SINCE_ID=221&USERNAME=XXXXXX&PASSWORD=XXXXXX

TYPE: The “TYPE” parameter controls what type of request you are sending to the server. 

DEEP: By default, your request is “TYPE=DEEP” which means that if you have a form that contains Subforms, the request is going to find all related Subform records and return them. To change the type, you need to manually pass the parameter and the property into the URL.

TOP: When you pass the “TOP” property in the URL, the request will only return the data for the table you are running the request on. No sub-form records will be returned with this request.

ID: Passing the “ID” property will only return the record IDs. This option is effective for complex data structures and identifies the records that need to be fetched. Knowing the records that are new, you can now grab each record one at a time. 

SINCE_DATE: This parameter accepts a value of “yyyy-mm-dd” and will return records that have a CREATED_DATE greater than the date you specify in the URL.

Ex. SINCE_DATE=2011-01-01(Records that were created on or after 2011-01-02 will be returned.)

SINCE_ID: Similar to SINCE_DATE, this parameter accepts a numerical value and returns records that have an ID greater than the one specified.

Ex. SINCE_ID=250 (Records starting from 251 and up will be returned.)

Complex Data Structures

For complex structures with lots of sub-forms, you should manage your downloads in two different steps. You should create a script that looks first for the new record IDs to download, and then download each record one at a time.

To figure out which records to download, you can use TYPE=ID and either SINCE_ID, or SINCE_DATE to determine which records you need to pull down. Once you have the list of IDs then you can go about downloading each one.

STEP 1: This produces a list of all the new records that you will want to download. Based on the URL below, the results returned 4 records (101, 102, 103, 104)

&TABLE_NAME=_data99999_data_feed_test&TYPE=ID&SINCE_ID=100

STEP 2: Now you can run queries on each new record from your list one at a time.

• &TABLE_NAME=_data99999_data_feed_test&ID=101
• &TABLE_NAME=_data99999_data_feed_test&ID=102
• &TABLE_NAME=_data99999_data_feed_test&ID=103
• &TABLE_NAME=_data99999_data_feed_test&ID=104

Data Feed Integration

You can use the data feed URL to pull data at any given time by passing in the USERNAME and PASSWORD parameters in the URL that you want data from.

PLEASE NOTE: This is not able to be used if your server upgraded to the new identity management system (ZIM-enabled).

There are a few other ways you can access data without logging into the iFormBuilder Database.

• OPTION 1. One thing you can do is to set up an email notification so iFormBuilder will email any new data to a particular address(s). You can then either process the email or use it as a trigger to wake up a script that pulls data into another database.
• OPTION 2. To integrate the data you collect into other backend systems please view this blog post on backend integration using the JSON Data Feed.

For security reason, we recommend that you create a specific user that is only able to VIEW data for that one page you want access to. Please do not link you admin user to an external system.