February-1st-2021, 08:07 AM
(This post was last modified: February-2nd-2021, 04:14 PM by Armin@netPI.)
What I can definitively say is that you need indeep Node-RED programming skills to program a good working Node-RED flow for this specific customer demand.
Here in this forum you will not find programmers who can help you with programming a good working Node-RED flow. You need to do this on your own since it is not really a netPI related question but an application related one. Even Hilscher HQ is not able to program such alike applications. Here in HQ we forward such requests to local engineering companies who have programming skills.
Based on the azure Node-RED documentation https://flows.nodered.org/node/node-red-...re-iot-hub we can assume the azure Node-RED node returns an information when there is no connectivity to azure at all or the internet connection is lost. So when it indicates a fail for sending the data to azure the data needs to be backuped in influxdb database instead. And then of course you need to program an algorithm to read back the data stored from the influxdb database and send it to azure when it is back online.
My personal opinion: to be 100% reliable means to me that Node-RED should cache/buffer all fieldbus data influxdb DB always and then as a second task forward the data to azure node so that you get the max chance of the cache/buffer being set before detecting whether the connection works or not . You also need some kind of feedback to detect a failed send. As the send happens over TCP/IP, that could take an appreciable amount of time - TCP timeouts can be up to minutes though. So in any case writing the data stream to influxDB always first is a safe way to me. This is only possible with 32GB high endurance card. The 8GB card will get destroyed too soon for this kind of use case.
One word to Influxdb in general: when creating a database in influxdb you can specify enter a rentention time for the data stored. Influxdb will delete then data stored that is older than a certain amount of time e.g. 24 hours or 1 week (minimum is 1 hour) .... This prevents the database from getting too big after years. You should consider this in your pogramming
My opinion about a possible Node-RED flow:
1. The "fieldbus input" node has to inject the fieldbus data to the "influxdb out" node always which in turn stores all the data in the related influxdb database.
2. With an inject node create a cyclic node. This node connects to "influxdb in" node which reads stored data from the influxdb cyclically. For each read you can specify the time you want to read data from the database somewhere in the past. So you get back exactly the data from a specific time written in the past. The output of the influxdb out node is then connected to the azure out node and hence sent to azure cloud then.
3. If now the azure output node reports a successful write to azure cloud then you are sure that data is in the cloud and you can increase the time for the next influxdb read cycle. How to detect a message could be sent to the azure cloud successfully the documentation says https://flows.nodered.org/node/node-red-...re-iot-hub that the azure debug output message is "Message sent.". So you can check this value and you know about the status.
4. If now the azure node reports a fail when sending the data to azure you know the data is not in the cloud, then do not increase the influxdb time and in the next cycle it will read the same data from influxdb and hence tries to send the data again that failed before.
Thx
Armin
Here in this forum you will not find programmers who can help you with programming a good working Node-RED flow. You need to do this on your own since it is not really a netPI related question but an application related one. Even Hilscher HQ is not able to program such alike applications. Here in HQ we forward such requests to local engineering companies who have programming skills.
Based on the azure Node-RED documentation https://flows.nodered.org/node/node-red-...re-iot-hub we can assume the azure Node-RED node returns an information when there is no connectivity to azure at all or the internet connection is lost. So when it indicates a fail for sending the data to azure the data needs to be backuped in influxdb database instead. And then of course you need to program an algorithm to read back the data stored from the influxdb database and send it to azure when it is back online.
My personal opinion: to be 100% reliable means to me that Node-RED should cache/buffer all fieldbus data influxdb DB always and then as a second task forward the data to azure node so that you get the max chance of the cache/buffer being set before detecting whether the connection works or not . You also need some kind of feedback to detect a failed send. As the send happens over TCP/IP, that could take an appreciable amount of time - TCP timeouts can be up to minutes though. So in any case writing the data stream to influxDB always first is a safe way to me. This is only possible with 32GB high endurance card. The 8GB card will get destroyed too soon for this kind of use case.
One word to Influxdb in general: when creating a database in influxdb you can specify enter a rentention time for the data stored. Influxdb will delete then data stored that is older than a certain amount of time e.g. 24 hours or 1 week (minimum is 1 hour) .... This prevents the database from getting too big after years. You should consider this in your pogramming
My opinion about a possible Node-RED flow:
1. The "fieldbus input" node has to inject the fieldbus data to the "influxdb out" node always which in turn stores all the data in the related influxdb database.
2. With an inject node create a cyclic node. This node connects to "influxdb in" node which reads stored data from the influxdb cyclically. For each read you can specify the time you want to read data from the database somewhere in the past. So you get back exactly the data from a specific time written in the past. The output of the influxdb out node is then connected to the azure out node and hence sent to azure cloud then.
3. If now the azure output node reports a successful write to azure cloud then you are sure that data is in the cloud and you can increase the time for the next influxdb read cycle. How to detect a message could be sent to the azure cloud successfully the documentation says https://flows.nodered.org/node/node-red-...re-iot-hub that the azure debug output message is "Message sent.". So you can check this value and you know about the status.
4. If now the azure node reports a fail when sending the data to azure you know the data is not in the cloud, then do not increase the influxdb time and in the next cycle it will read the same data from influxdb and hence tries to send the data again that failed before.
Thx
Armin
„You never fail until you stop trying.“, Albert Einstein (1879 - 1955)
