Configure Dataflow notifications in Onesait Platform
Dataflow is a tool that allows the definition and execution of data flows. In this post we are going to see how we can configure alerts using Webhooks. For a description of Dataflow functionalities in the Platform, we recommend you to review the documentation we have prepared in the Development Portal.
Specifically, we are going to see the example using Microsoft Teams to send notifications, but you could use any other system that allows sending messages via HTTP.
Configuring a channel in Microsoft Teams
The first thing to do is to configure the channel in Teams to create a “Connector“. This step is explained in detail in the Microsoft Teams documentation, but as a summary, suffice it to say that you have to open the options of the channel in which you want to write the notifications, go to the “Connectors” option, and create an “Incoming Webhook” type connector.
Once the connector is configured, it will provide us with a URL:
Pipeline configuration
This URL will allow you to send messages to the channel. Simply copy it and go to the notification settings of the Onesait Platform pipeline that you want to send notifications to the Teams channel.
Copy the URL in the “Webhook URL” box. As you can see, simply set the “Content-Type” header to “application/json” and use the POST method to send the message to Teams. Within the notifications you can indicate the status that you want to generate a notification.
The message format must follow Teams specifications. The following JSON is an example of a valid message:
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"themeColor": "0076D7",
"title": "Alerta entorno PRO, cambio de estado en pipeline '{{PIPELINE_TITLE}}'",
"summary": "Pipeline '{{PIPELINE_TITLE}}' state changed to {{PIPELINE_STATE}} at {{TIME}}",
"sections": [{
"activityTitle": "Pipeline '{{PIPELINE_TITLE}}' state changed to {{PIPELINE_STATE}} at {{TIME}}",
"activitySubtitle": "Cambio de estado en pipeline",
"activityImage": "https://indra365.sharepoint.com/:i:/r/sites/Grp_T_OnesaitPlatformCloud2-TestingDataflowWebhook/Documentos%20compartidos/Testing%20Dataflow%20Webhook/warning.png?csf=1&web=1&e=6VvXZL",
"facts": [{
"name": "Status Message",
"value": "{{PIPELINE_STATE_MESSAGE}}"
}, {
"name": "Input Records Count",
"value": "{{PIPELINE_INPUT_RECORDS_COUNT}}"
}, {
"name": "Output Records Count",
"value": "{{PIPELINE_OUTPUT_RECORDS_COUNT}}"
}, {
"name": "Error Records Count",
"value": "{{PIPELINE_ERROR_RECORDS_COUNT}}"
}, {
"name": "Error Messages Count",
"value": "{{PIPELINE_ERROR_MESSAGES_COUNT}}"
}, {
"name": "Runtime Parameters",
"value": "{{PIPELINE_RUNTIME_PARAMETERS}}"
}],
"markdown": true
}],
"potentialAction": [{
"@type": "OpenUri",
"name": "Go to ControlPanel",
"targets": [{
"os": "default",
"uri": "https://development.onesaitplatform.com/controlpanel"
}]
}]
}The following example shows a message when the pipeline stops:
Summary
In this case we have seen how to generate notifications in Teams notifications about the status of Onesait Platform Dataflow with very little effort. In addition, if necessary, notifications can also be created based on rules about metrics of the data flows and about values of the data itself.
Header image: Microsoft
✍🏻 Author(s)
