IOT and ML Analytics Demo part2

Hi all,

In my previous blog, I wrote about setting up ML analytics tool on an AWS EC2 compute platform. Upon further investigation on AWS, I found out that AWS provides services like IOT core, IOT Analytics and IOT SageMaker which helps to collect data from sensors, store them and process using Analytics tools.

The following picture shows an overview of how AWS Analytics can be used.

Collect data from sensors and send it to Node-red using MQTT. We know how to do this from the previous assignments.

To setup data flow from Node-Red to ML environment on AWS cloud platform, initially I wired up node-red flow as follows.

Http(get request) can be used to get data from the internet. Any device that is connected to the internet can send data to AWS IOT Analytics. MQTT is configured as follows and enabled TLS configuration for security purpose.

TLS Configuration:

Upload the certificates and the private key that AWS provides when you specify the security policies you required. This video helped me to setup AWS security cerificates( https://www.youtube.com/watch?v=sdgXt7Sq2dM)

YOUTUBEMar 28, 2019

AWS IOT core provides with a MQTT client to subscribe to a topic and receive the messages. Collect and store only the data you want. You can use the AWS IoT Analytics console to configure AWS IoT Analytics to receive messages from devices through MQTT topic filters in various formats and frequencies. AWS console after receiving messages is as follows:

Now, We have data collected from MQTT. Next, quickly create IoT Analytics resources like channel, pipeline, data store and data set. A channel collects data from an MQTT topic and stores the unprocessed messages in an Amazon simple storage service(Amazon s3) bucket. A pipeline consumes messages from a channel and enables you to process the messages before storing them in a data store. You retrieve data from a data store by creating a data set. Once, these resources are created, go to dataset and run it to generate a materialized view as follows.

Now, the dataset is ready for further predictions. For this, AWS provides a service called IOT SageMaker. It allows you to create a notebook instance and configure the instance as follows:

Once, the instance is created make sure the service is enabled which lets you open jupyter. The screen is as follows:

Open jupyter and select the framework you want to work on(I used conda_tensorflow_p36).

Then, you write some python code to train data, create model and make predictions accordingly. The predicted data can be visualized using AWS IoT Analytics console. You can view the visualization on the console’s dataset content page.