Snap4City supports the connection with CKAN Open/Private Data Portals. The CKAN integration is a scenario in which Snap4city solution is used to realize automated solutions to:
- automating the Import data from CKAN to Snap4City
- automating the Upload Public Data from Snap4City to CKAN
- Data Harvesting
- automating the Creation of Snap4City Dashboards and Mobile/Web Apps in which can be viewed with advanced tools: real time data, historical data, heatmaps, etc. and on which are available decision support services, as the what-if analysis, and remote control services.
| Test Case Title |
TC9.17 – CKAN vs Snap4City Integration and Interaction |
| Goals |
Interact with CKAN tool to:
|
| Prerequisites |
Using a PC or Mobile with a web browser. You need a Snap4City account. If you want to create a Resource or a Datyaset on CKAN (and not only read them) you must have an account on CKAN. The following functionalities are available only for specific Snap4city users with specific privileges. |
| Expected successful result | Generation of flows of data to interact with CKAN. |
Snap4City can interact with external CKAN portal in various ways via IOT applications. In the next, some real examples of possible interaction between Snap4City and CKAN are explained.
- Automating Read Dataset Info from CKAN
The following IOTApp flow shows how to read a dataset (called package in CKAN) on the basis of its name or identifier. A package is a container of a set of real data.
The function node, uses the CKAN API by indicating the CKAN package of interest. As shown in the following image, if the resource is private, you have also to include the CKAN user APIkey to access to it. Please note that CKAN API are explained here: https://docs.ckan.org/en/2.8/api/
In the example above, the resource of interest has the id “=from_snap4city_to_ckan”.
The node http request sends HTTP requests and returns the response on the basis of the URL defined in the function node.
The node json simply allows to make more readable the results in the debug area.
- Automating Read Resource info from CKAN
The following flow allows to recover a single resource information from the CKAN tool on the basis of its name or identifier.
In this example, the function node uses the CKAN API to search a resources matching a query by using the resource id. Please note that CKAN API are explained here: https://docs.ckan.org/en/2.8/api/
As in the previous example, the node http request sends HTTP requests and returns the response on the basis of the URL defined in the function node.
The node json simply allows to make more readable the results in the debug area.
- Automating Create a Dataset on CKAN
In this example, the flow allows to create a dataset into the DISIT organization in CKAN. The dataset is created according to the fields reported in the json in the function node.
The function node is shown in the following image. In this case the code specifies also the organization ID and the user API key, in case the resource is not public.
The data_json reports the list of fields useful to create the dataset.
The http request node specifies the CKAN API to be used. Please note that CKAN API are explained here: https://docs.ckan.org/en/2.8/api/
- Automating Create a static resource in CKAN
This following flow creates a static resource containing a link to a resource in Snap4City and public this link directly on CKAN as private resource of the user.
The first function node has the aim to read the url of a resource in Snap4City that in the example below is related to a weather sensor in Tuscany.
The http request node contains the CKAN API call to public the resource into the CKAN tool. Please note that CKAN API are explained here: https://docs.ckan.org/en/2.8/api/
As in the previous examples, the node json simply allows to make more readable the results in the debug area.
- Automating Create a Dynamic Resource in CKAN
This example has the aim to public on CKAN a resource containing an url to recover the last hour data of a weather sensor in Tuscany. In this case the resource is dynamic since allows to recover the last hour data depending on when the process is launched.
The first function node has the aim to define the correct format of the query by defining a starting date and an ending date, that in this example is related to the last hour.
The second function node has the aim to compose the correct message to be sent via the http request node that calls the CKAN API.
- Automating Send a json to CKAN from a query to Snap4City
In this last example, a json recovered on Snap4city is published a resource in CKAN. The numered nodes in the following image are explained above.
The first function node (1) contains the query to Snap4City to request the data in json;
Then, the next http request node called weather_ow (2), takes the query as input (url as parameter) and executes http post, that means: sends the url and recover the json;
The function node (3) then has the aim to select only the data of interest from the received json;
The next Edit file node (4) has the aim to take the output of the node (2), i.e. the json, and save it in a file. This file is overwritten every time in order to not accumulate useless data;
The delay node (5) has the aim to wait the end of the process of writing the data in the file.
Then, the Edit file node /data/weather_ow (6) read the data from the previously saved file;
Finally, the last function node (7), creates the message to be sent to CKAN, in which the “Content-type” has to be a “multipart/form-data” type. In the following the code.
APIkey_user='*****'
msg.headers = {};
msg.headers['Content-type'] = 'multipart/form-data'
msg.headers['Authorization'] = APIkey_user;
data_json = {
'package_id':'21bdb2a0-48b2-4d12-b42d-bd18722cb407',//'from_snap4city_to_ckan',
//'url': url,
'description':'JSON - Real Time weather in a Tuscany Sensor. Data coming from Snap4City portal (https://servicemap.snap4city.org) in connection with OpenWheather',
'name': 'Real time weather in a Tuscany weather sensor',
'format':'json',
}
msg.headers = {
"Content-Type": "multipart/form-data; boundary=------------------------d74496d66958873e",
"Authorization": APIkey_user
}
msg.payload = '--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="package_id"\r\n'+
'\r\n'+
'21bdb2a0-48b2-4d12-b42d-bd18722cb407\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="description"\r\n'+'\r\n'+
'JSON - Real Time weather in a Tuscany Sensor. Data coming from Snap4City portal (https://servicemap.snap4city.org) in connection with OpenWheather\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="name"\r\n'+
'\r\n'+
'Real time weather in a Tuscany weather sensor\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="format"\r\n'+
'\r\n'+
'json\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="upload"; filename="weather_ow.json"\r\n'+
'Content-Type: application/json\r\n'+
'\r\n'+
msg.payload+'\r\n'+
'--------------------------d74496d66958873e--\r\n';
return msg; 