My first IOT Raspberry PI Experiment

I am going to describe my amazing introduction of Raspberry Pi. I was introduced to this amazing device a couple of weeks back as part of my training and I immediately decided to procure one and get my hands dirty with it. Well no fun comes with pain. So I decided to write a blog to make it simpler for other trying their hands for the first time.

Goal:

Setup a PC with following:

Message Broker supporting MQTT protocol to receive messages over WiFi from Raspberry Pi.
Subscriber code to receive messages from the broker by subscribing to a channel and push the data to Mongo DB.
Mongo Db to collect the data.

Wifi Router: Use the home WiFi to establish connectivity between PC and Raspberry Pi
Raspberry Pi - Publisher module to publish the data to message broker listening in the PC (data will not be from sensor but dummy data)

Section 1

Before we start with the integration, I will start from the beginning as Raspberry Pi is absolutely new to me.

Setup 1: Raspberry Pi Setup -

I purchased "Virlos" Raspberry Pi board from Amazon, but any board should do. I am using a 8 GB board but anything lower would do as well.

All the parts received in the package

Nicely packed in the metal case (optional)

Setup 2: Install Respbian OS on Respbery Pi

Download the Raspberry Pi Imager from https://www.raspberrypi.org/software/
Insert a SD card in the card reader and attach to your Windows PC.
Install the Raspberry Pi Imager.
Once installed, execute Raspberry Pi Imager.
It will ask you to select the OS (choose Raspberry PI OS 32 bit) and Storage (choose the SD card inserted).
It may prompt you to format the card (FAT 32 format), please go ahead and format the card.
The installation may take time but once complete the SD card is ready.
Take out the SD card and insert into Raspberry Pi board.
Add the power supply to the board, add the display HDMI from board to display device.
Add a Bluetooth monitor and keyboard.( you can choose any alternate way to get the display device, keyboard and mouse to the board).

Step 3: Log into Raspberry Pi OS

Switch on the board. You should be able to see the following (not in :

Welcome Message
Update message for software upgrade
Setup the region setting
Network setup

Step 4: Establish connectivity between PC and Raspberry Pi

Get the IP of the PC by running the command prompt. Run the ipconfig command
Get the IP of the of the Raspberry Pi. (Run the 'ifconfig' command from terminal)
Try pinging PC to Raspberry and Raspberry to PC. In both the cases we should receive message like "Reply from 192.168.XX.XXX: bytes=32 time=6ms TTL=64". This is a very important step for the overall experiment.

Troubleshooting Network Error: In case you are receiving "Destination Not Reachable." there can be many reasons for it, But I am trying to list few which I encountered to overcome the network error.

Make sure both PC and Raspberry Pi are on the same WiFi network.
If still you are not able to ping, chances are that "Internet Connection Sharing" is enabled for the WiFi Network Adapter. Following are the steps to do that.

Step 5: Establish SSH from PC to Raspberry (Optional)

This is an optional step but would help in case you would like to transfer the file from PC to Raspberry.

You can use Putty to test the SSH. Likely, it will not work. A sort work around is to connect the SD card to the PC and create an empty SSH folder at the root. Now replace the SD card back to the Raspberry device and switch on the device. Now try to SSH from PC, it should work.

Section 2

Now our basic setup is ready. In the next stage, lets get the PC ready to receive the messages from Raspberry Pi.

Step 1: Install Mongodb

Step 2: Install Mosquitto

It is the broker to establish pub-sub over MQTT protocol. Can be downloaded and installed from https://mosquitto.org/download/ . I am not getting into the detailed steps as it would be a separate topic.

Step 3: Install python (I used version 3.9.3).

Step 4: Install pymongo (to connect with mongo) and paho module (to connect with mosquitto broker)

I used PIP for python package management (https://pip.pypa.io/en/stable/installing/).

$ python -m pip install pymongo

$ python -m pip install paho-mqtt

Step 5: Validate mosquitto connectivity

Mosquitto default port is 1883. So to make sure there are no issues in connecting from Raspberry Pi, try to telnet.

$ telnet 192.168.XX.XXX 1883

Troubleshooting Tip: It telnet do not succeed, open the mosquitto.conf file and in the listeners section make an explicit entry for "listener 1883" which may be missing.

Also in the security section, mention "allow_anonymous true". Now the telnet should work.

Section 3

In the final section we will write the python code for subscriber and publisher to see it in action.

Step 1: Publisher code

The publisher is going to publish simulate the data from sensors and push it to mosquitto broker on the PC over MQTT protocol.

Lets first create the configuration file for the sensor simulation data and other configurations.

publisher-config.json

{

        "broker_host": "192.168.XX.XXX",
        "broker_port": 1883,

        "devices": [
            {
            "type": "temperature",
            "publish_frequency": 5,
            "publish_topic": "devices/temp",
            "std_val": 25,
            "device_count": 2
            },
            {
            "type": "humidity",
            "publish_frequency": 10,
            "publish_topic": "devices/hum",
            "std_val": 40,
            "device_count": 2
            },
            {
            "type": "co2",
            "publish_frequency": 5,
            "publish_topic": "devices/co2",
            "std_val": 20,
            "device_count": 1
            }
        ]
}

publisher.py

import paho.mqtt.client as mqtt
import time
import json
import numpy as np
import datetime

# Callback function - executed when the program successfully connects to the broker


def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe("test")

# Callback function - executed when the program gracefully disconnects from the broker


def on_disconnect(client, userdata, rc):
    print("Disconnected with result code "+str(rc))

# Callback function - executed whenever a message is published to the topics that
# this program is subscribed to


def on_message(client, userdata, msg):
    print(msg.topic, str(msg.payload), "retain",
          msg.retain, "qos", msg.qos, str(userdata))


# Defining an MQTT client object
client = mqtt.Client()

# Setting callback functions for various client operations
client.on_connect = on_connect
client.on_message = on_message
client.on_disconnect = on_disconnect

# Reading the configuration file
f = open("publisher-config.json")
config = json.loads(f.read())
f.close()

# Initialising devices from the config.json file and assigning device_ids to each device
device_config = []
for devices in config['devices']:
    for n in range(devices['device_count']):
        dev = {}
        dev['device_id'] = devices['type']+"_"+str(n)
        dev['device_type'] = devices['type']
        dev['publish_frequency'] = devices['publish_frequency']
        dev['std_val'] = devices['std_val']
        dev['publish_topic'] = devices['publish_topic']
        device_config.append(dev)


# Connecting to broker
client.connect(host=config["broker_host"],
               port=config["broker_port"], keepalive=60)

'''
Start the MQTT client non-blocking loop to listen the broker for messages 
in subscribed topics and other operations for which the callback functions 
are defined
'''
client.loop_start()

clock = 0
while True:
    try:
        # Iterating through the items in device configuration dictionary, every second
        time.sleep(1)
        clock = clock+1
        for devices in device_config:
            if clock % devices['publish_frequency'] == 0:
                print("Published to devices/"+devices["device_type"])
                # Initialize a dictionary to be sent as publish message
                message = {}
                # Generate timestamp in YYYY-MM-DD HH:MM:SS format
                message["timestamp"] = datetime.datetime.now().strftime(
                    "%Y-%m-%d %H:%M:%S")
                message["device_id"] = devices["device_id"]
                message["device_type"] = devices["device_type"]
                # Generate a random value using normal distribution function of the
                # configured standard value for the given device type
                message["value"] = round(
                    np.random.normal(devices["std_val"], 2), 2)
                # Publish the message
                client.publish(devices["publish_topic"], json.dumps(message))
    # Disconnect the client from MQTT broker and stop the loop gracefully at
    # Keyboard interrupt (Ctrl+C)
    except KeyboardInterrupt:
        client.disconnect()
        client.loop_stop()
        break

Step 2: Push config file and publisher.py to Raspberry Pi

I used Filezilla to push the files to Raspberry Pi device from PC. You can directly write in Raspberry Pi as it comes with the Python editor and interpreter.

Step 3: Subscriber code

subscriber-config.json

{

        "broker_host": "localhost",
        "broker_port": 1883,
    
        "db_host": "localhost",
        "db_port": 27017,
        "db_name": "iot-db",
        "db_collection":"iot-sensors-data",
        
}

subscriber.py

import paho.mqtt.client as mqtt
import json
import pymongo

# Callback function - executed when the program successfully connects to the broker


def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe("devices/#")

# Callback function - executed when the program gracefully disconnects from the broker


def on_disconnect(client, userdata, rc):
    print("Disconnected with result code "+str(rc))

# Callback function - executed whenever a message is published to the topics that
# this program is subscribed to


def on_message(client, userdata, msg):
    item = {"topic": msg.topic, "payload": msg.payload}
    dbt.insert_one(item)
    print("Received a messsage on " + msg.topic + " and inserted it to the DB")


# Defining an MQTT client object
client = mqtt.Client()

# Setting callback functions for various client operations
client.on_connect = on_connect
client.on_message = on_message
client.on_disconnect = on_disconnect


# Reading the configuration file
f = open("subscriber-config.json")
config = json.loads(f.read())
f.close()

# Initializing connection to the database
dbclient = pymongo.MongoClient(config["db_host"], config["db_port"])
db = dbclient[config["db_name"]]
dbt = db[config["db_collection"]]

# Connecting to broker
client.connect(host=config["broker_host"],
               port=config["broker_port"], keepalive=60)

'''
Start the MQTT client non-blocking loop to listen the broker for messages 
in subscribed topics and other operations for which the callback functions 
are defined
'''
client.loop_start()

while True:
    try:
        pass
    # Disconnect the client from MQTT broker and stop the loop gracefully at
    # Keyboard interrupt (Ctrl+C)
    except KeyboardInterrupt:
        client.disconnect()
        client.loop_stop()
        break

Step 3: Start the publisher on Raspberry Pi

You should see output like below

abhisar@raspberrypi:~/Documents $ python3 publisher.py
Connected with result code 0
Published to devices/temperature
Published to devices/temperature
Published to devices/co2
Published to devices/temperature
Published to devices/temperature
Published to devices/humidity
Published to devices/humidity
Published to devices/co2
Published to devices/temperature
Published to devices/temperature
Published to devices/co2
Published to devices/temperature
Published to devices/temperature
Published to devices/humidity
Published to devices/humidity
Published to devices/co2

Step 4: Start the subscriber on PC

You should see below output

Received a message on devices/temp and inserted it to the DB
Received a message on devices/temp and inserted it to the DB
Received a message on devices/hum and inserted it to the DB
Received a message on devices/hum and inserted it to the DB
Received a message on devices/co2 and inserted it to the DB
Received a message on devices/temp and inserted it to the DB
Received a message on devices/temp and inserted it to the DB
Received a message on devices/co2 and inserted it to the DB
Received a message on devices/temp and inserted it to the DB
Received a message on devices/temp and inserted it to the DB
Received a message on devices/hum and inserted it to the DB
Received a message on devices/hum and inserted it to the DB
Received a message on devices/co2 and inserted it to the DB

Step 5: Check the data in Mongo

This concludes my first experiment with Raspberry Pi. I hope other trying their hands with device would be able to get through the challenges faster. Hope it helps others

Search This Blog

Abhisar's Blog