It will be shown how to leverage Connect IoT extensibility for a more dynamic implementation. This will show a more advanced feature of Connect IoT.
Topics
- Overview
- Use Case IoT
- Use Case MES (extra)
Overview
Connect IoT is not just customizable, but it is also extendable. This means that you have entry points that are already given out of the box, for a more advanced implementation that can leverage the system. For example, in the automation controller the code task, provides a point of extensibility where you can create a task with your own set of code. This is a rather straightforward point of extensibility, but also the driver is extensible and the use of both together is what is going to be shown today.
The driver is traditionally described in the Automation Driver Definition, where you will complete all the required communication interface with a particular equipment in a given protocol. Nevertheless, it may be the case where you may not want to have such a static definition, and may want to resolve what to subscribe or get by logic outside of the normal driver definition.
This is an advanced scenario as you are skipping the normal system behavior and are replacing this mechanism with your own.
The use case is on Track In, depending on the attached resource instrument, we want to dynamically resolve which properties to retrieve and post to a Data Collection. So, in this case we don’t know before hand what are the properties we want to retrieve. The business logic is the owner of what the system must retrieve and is even orchestrated outside the Automation Controller.
The implementation will be done using OPC-UA as protocol, but should be more or less generic to every protocol.
Use Case IoT
We will try to focus first on the use case from a Connect IoT perspective and will then explain a bit more what was the full feature, using the MES. From an IoT perspective the use case will be that we require on Track In to dynamically Get Properties and post to a DataCollectionInstance opened for a Material.
Create a DEE in Track In
We will have a DEE that will publish to IoT on the topic Cmf.Trackin.LineClearance with an object like:
{
"materialName": "materialExample",
"parameterTag":
{ "Temperature": {
"tag": "ns=3;s=AirConditioner_1.Temperature",
"deviceType": "Long",
"dataType": "Double",
},
"Pressure": {
"tag": "ns=3;s=AirConditioner_1.Pressure",
"deviceType": "Long",
"dataType": "Double",
},
}
}
Then the DEE must be something like this (here we show the snippet pertinent to IoT), it will collect the material name and then for each parameter collect all the necessary information to be able to get the property and post to a datacollection.
(...)
var parameters = dc.DataCollectionParameters.Select(dcp => dcp.TargetEntity.Name);
// This is the object that will hold the parameter and as value, everything required to perform a get
JObject parameterTag = new JObject();
foreach (DataRow row in ds.Tables[0].Rows)
{
var rowParam = row["Parameter"].ToString();
if (parameters.Contains(rowParam))
{
JObject tagInfo = new JObject();
tagInfo["tag"] = row["DeviceID"].ToString();
tagInfo["deviceType"] = row["DeviceType"].ToString();
tagInfo["dataType"] = row["DataType"].ToString();
parameterTag[row["Parameter"].ToString()] = tagInfo;
}
}
// This code implies there is an Instance associated between the Resource, the Controller and Driver
var instance = resource.GetAutomationControllerInstance();
if (instance == null)
throw new Exception("Resource not connected to any IoT instance");
else
{
JObject data = new JObject();
data["materialName"] = material.Name;
data["parameterTag"] = parameterTag;
string topic = "Cmf.Trackin.LineClearance";
// This Code will publish a message bus broadcast for the specified topic, with the specified data
instance.Publish(topic, data);
}
Note: The payloads being sent in the MessageBus should be the least required for the implementation. Beware of sending big payloads, creating traffic, without a very strong reason, if you fall in this case, you are probably doing something wrong in terms of component coupling architecture.
Looking at the Automation Controller
The implementation in Connect IoT, will now be split in to four parts: Receiving Messages, Transform Data, Dynamically Get Properties, Post to Data Collection.
Receiving Messages
Now we have a DEE that publishes a message with topic Cmf.Trackin.LineClearance on TrackIn with the information regarding the material tracked in and what parameters to collect and post. Now we need to add a subscriber for that topic by using a On System Event Documentation, this task will be notified whenever a message is published in the MessageBus for this topic.
We have created a link between the information in the DEE and our workflow.
Transform Data
We will use two Code tasks to transform the data from the parameterTags provided by the DEE into the input content the Send Notification To Driver is expecting. Then we will transform that information into what is needed to successfully post the data.
Create the content object to send to the driver
In this case we are not worried about the extended data, so we will just use the default.
import { Framework } from 'framework';
export default class {
(/** Allows accessing external functions */
private framework: Framework;
constructor(framework: Framework) {
this.framework = framework;
}
/*
* Entry point of the class (IMPORTANT: don't change the signature of this method)
* Should return an object containing the values for each output to emit
* If necessary, use the parameter "outputs" to emit data while running the code.
*/)
public async main(inputs: any, outputs: any): Promise<any> {
const values = new Map<string, Tag>(Object.entries(inputs.parameterTag));
let content = [];
for (const [key, value] of values.entries()) {
const prop = {
name: value.tag,
deviceId: value.tag,
deviceType: value.deviceType,
dataType: value.dataType,
extendedData: value?.extendedData ?? JSON.parse("{\"$id\":\"1\",\"customDataType\":\"\",\"arrayType\":\"Scalar\",\"dimensions\":\"\",\"path\":\"\",\"isPersisted\":\"false\",\"defaultValue\":\"\",\"minimumSamplingInterval\":\"1000\"}")
};
content.push(prop);
}
outputs.content.emit(content);
outputs.parameterTag.emit(inputs.parameterTag);
}
}
export interface Tag {
tag: string;
deviceType: string;
dataType: string;
extendedData: any;
}
Create the rawData object to send to the DataCollection Task
The translation from what we receive from the Send Notification To Driver into rawData of the task Data Collection Documentation.
import { Framework } from 'framework';
export default class {
/** Allows accessing external functions */
private framework: Framework;
constructor(framework: Framework) {
this.framework = framework;
}
/*
* Entry point of the class (IMPORTANT: don't change the signature of this method)
* Should return an object containing the values for each output to emit
* If necessary, use the parameter "outputs" to emit data while running the code.
*/
public async main(inputs: any, outputs: any): Promise<any> {
const values = new Map<string, Tag>(Object.entries(inputs.parameterTag));
const reply = Array.from(inputs.reply) as Array<TagReply>;
let dcp = new Map<string, DataCollectionPoint>();
for (const [key, value] of values.entries()) {
for (const replyValue of reply) {
if (value.tag === replyValue.propertyName) {
const point = {
value: replyValue.value,
sampleId: "Sample 1",
};
dcp.set(key, point);
break;
}
}
}
outputs.dataCollectionPoints.emit(dcp);
}
}
export interface Tag {
tag: string;
deviceType: string;
dataType: string;
extendedData: any;
}
export interface TagReply {
propertyName: string;
originalValue: any;
value: string;
}
export interface DataCollectionPoint {
value: string;
sampleId: any;
}
Dynamically Get Properties
We will now dynamically invoke the Send Notification to Driver Documentation. We can use the extensibility to use in runtime, without having to pre declare in the driver definition all the information regarding events, commands and properties. In this case, we will dynamically GetProperties.
Extensibility
If we check the documentation for OPC-UA Documentation, in the section under methods, it describes how we can use the extensibility feature. In this case we want to get properties, to collect OPC-UA tags. In order to do that, we need to send a Request for the driver with the topic connect.iot.driver.opcua.getPropertiesValues and as the documentation explains, it requires a name, deviceId, dataType, deviceType.
In order to use the extension entry point we could invoke directly from the Code Task, in this case we want to separate the business logic, so we will use the default task for this Send Notification To Driver. We will connect the data from the On System Event to the Code Task retrieving the object parameterTag. Notice these are the keys of the object created in the DEE.
Using Only the Code Task
Post to Data Collection
Now we have everything we need to perform a post to the material DataCollectionInstance, we use a Synchronize task, to make sure we have everything ready when we post. We use the converter Get Object Property path: materialName, type string, to retrieve the material name from the data output and then the converter Create System Entity Instance, with entityType Material and identifier Name to create a shell object of the Material. The Data Collection task will receive a material and the rawData corresponding the to dataCollectionPoints emitted by the code task.
This was to show how you can dynamically get properties, similar variations exist for all actions, set properties, execute commands and also notify and register events.
Use Case MES
We will now take some time to explain the full use case.
In this example we will create a generic table that relates Resource <-> Parameter <-> Device Id <-> DataType <-> DeviceType. In the TrackIn, we will retrieve the resource instruments attached to the resource and resolve the table, in order to retrieve the Device Ids to collect from the OPC-UA server. Then invoke Connect IoT to collect and post to the DataCollectionInstance that was opened in the TrackIn for the material.
Create a Generic Table
We created a Generic Table with the Resource (Name) and Parameter (Name) as keys, the DeviceId and DeviceType as text fields and the DataType as the Enum Cmf.Foundation.BusinessObjects.AutomationDataType. The DeviceType is part of the extension data of the Protocol, so we have no way of providing that information for the table, without being protocol specific.
We can add some data to the table matching resource instruments, parameters and OPC-UA tags.
Create a DEE in Track In
We will now create a DEE that will, resolve the instruments attached to the resource, and correlate the CurrentDataCollectionInstance of the Material with the parameters defined in the table for the attached instruments.
Here we are using some specific project utilities, but bellow the pertinent code is shown.
UseReference("System.Data.dll", "System.Data");
UseReference("Cmf.Foundation.BusinessObjects.dll", "Cmf.Foundation.BusinessObjects");
UseReference("Cmf.Foundation.BusinessOrchestration.dll", "");
UseReference("", "Cmf.Foundation.Common.Exceptions");
UseReference("", "Cmf.Foundation.Common");
UseReference("Cmf.Navigo.BusinessObjects.dll", "Cmf.Navigo.BusinessObjects");
UseReference("Cmf.Navigo.BusinessOrchestration.dll", "");
UseReference("Cmf.Navigo.BusinessOrchestration.dll", "Cmf.Navigo.BusinessOrchestration.MaterialManagement.InputObjects");
UseReference("Newtonsoft.Json.dll", "Newtonsoft.Json.Linq");
UseReference("%MicrosoftNetPath%Microsoft.CSharp.dll", "");
UseReference("Cmf.Custom.Project.Common.dll", "Cmf.Custom.Project.Common");
UseReference("Cmf.Common.CustomActionUtilities.dll", "Cmf.Common.CustomActionUtilities");
UseReference("Cmf.Custom.Project.Common.dll", "Cmf.Custom.Project.Common");
IMaterial material = DeeContextHelper.GetContextParameter("material") as IMaterial;
IResource resource = DeeContextHelper.GetContextParameter("resource") as IResource;
material.Load();
resource.Load();
// Retrieve Resource Instruments
if(resource.RelationCollection == null || !resource.RelationCollection.ContainsKey("ResourceInstrument")) {
resource.LoadRelations("ResourceInstrument");
}
var filters = new List<IFilter>();
if(resource.RelationCollection != null && resource.RelationCollection.ContainsKey("ResourceInstrument")) {
foreach(ResourceInstrument item in resource.RelationCollection["ResourceInstrument"]) {
filters.Add(new Foundation.BusinessObjects.QueryObject.Filter()
{
Name = "Resource",
Operator = Cmf.Foundation.Common.FieldOperator.IsEqualTo,
Value = item.TargetEntity.Name,
LogicalOperator = Cmf.Foundation.Common.LogicalOperator.OR
});
}
}
DataSet ds = Cmf.Custom.Project.Common.ProjectUtilities.GetGenericTableData("CustomIoTDataCollectionContext", filters.ToArray());
if (ds.HasData())
{
IDataCollectionInstance dataCollectionInstance = material.CurrentDataCollectionInstance;
dataCollectionInstance.Load();
IDataCollection dc = dataCollectionInstance.DataCollection as IDataCollection;
dc.LoadRelations("DataCollectionParameter");
if(!dc.DataCollectionParameters.Any()){
throw new Exception("No Parameters found");
}
var parameters = dc.DataCollectionParameters.Select(dcp => dcp.TargetEntity.Name);
// This is the object that will hold the parameter and as value, everything required to perform a get
JObject parameterTag = new JObject();
foreach (DataRow row in ds.Tables[0].Rows)
{
var rowParam = row["Parameter"].ToString();
if (parameters.Contains(rowParam))
{
JObject tagInfo = new JObject();
tagInfo["tag"] = row["DeviceID"].ToString();
tagInfo["deviceType"] = row["DeviceType"].ToString();
tagInfo["dataType"] = row["DataType"].ToString();
parameterTag[row["Parameter"].ToString()] = tagInfo;
}
}
// This code implies there is an Instance associated between the Resource, the Controller and Driver
var instance = resource.GetAutomationControllerInstance();
if (instance == null)
throw new Exception("Resource not connected to any IoT instance");
else
{
JObject data = new JObject();
data["materialName"] = material.Name;
data["parameterTag"] = parameterTag;
string topic = "Cmf.Trackin.LineClearance";
// This Code will publish a message bus broadcast for the specified topic, with the specified data
instance.Publish(topic, data);
}
}
This is the code pertaining to Cmf.Custom.Project.Common.ProjectUtilities.GetGenericTableData, invoked in the DEE.
/// <summary>
/// Returns a dataset with the filtered data from a GenericTable
/// </summary>
/// <param name="table">The Generic Table to filter</param>
/// <param name="filters">The filters to apply</param>
/// <returns>A DataSet with the filtered content, null otherwise</returns>
public static DataSet GetGenericTableData(IGenericTable table, params IFilter[] filters)
{
DataSet output = null;
if (table != null && table.Id > 0)
{
IFilterCollection tableFilters = new FilterCollection();
if (filters != null)
{
tableFilters.AddRange(filters);
table.LoadData(tableFilters);
}
else
{
table.LoadData();
}
if (table.Data != null)
{
output = NgpDataSet.ToDataSet(table.Data);
}
}
return output;
}
/// <summary>
/// Returns a dataset with the filtered data from a GenericTable
/// </summary>
/// <param name="tableName">The name of the Generic Table</param>
/// <param name="filters">The filters to apply</param>
/// <returns></returns>
public static DataSet GetGenericTableData(string tableName, params IFilter[] filters)
{
IGenericTable table = new GenericTable();
table.Load(tableName);
if (table != null)
{
return GetGenericTableData(table, filters);
}
return null;
}
After the message is published IoT will take care of the Data Collection Post.
Author
Hello 馃憦 , my name is Jo茫o Roque 鉁岋笍
I’ve been working for some years at Critical Manufacturing. I split my time working in the IoT Team and working with the project’s teams. You can visit me at https://j-roque.com/ or check me at LinkedIn
Skills: Connect IoT / DevOps