What about a mini MQTT broker on the ESP32?

Place your projects here
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Sun Oct 13, 2024 8:54 pm The small number of TCP connections is not a huge problem to me (see my [Local Link Removed for Guests]) as if you want to use MQTT exclusively you would use , say ,use a RPI Zero instead, where you have the capability of 1000(s) of connections.

The advantage ,to me, is to give simply the ability of a MQTT broker within the ESP32 to allow the android display capabilities for an ESP32 project on an Android phone, so no LCD hardware and software development required.
Annex has lots of capability for other types of connections to sensors etc that can be used and then aggregate the data into groups and present to the broker via a single TCP connection.
This approach then leaves what TCP connections are available to use where individual device connections must be used.
The limit imposed by the SDK seems to be at 16 (at least for the ESP32-S3) but some TCP are required by annex to work so, realistically, I think that 10 devices should be a valid max number of devices; probably the real limit should be at around 14.
I'm actually testing it and the broker seems to coexist well inside Annex; it can listen for the messages and also send as if it was a client to the other clients. Practically can work alone acting like a client/server or gather messages coming from other sources (MQTT, ESP-NOW, TCP, URL, ..) and exchange with other MQTT client.
I did not yet tested, but it should also be able to connect with another MQTT server (using the existing client functionality)
Monki
Posts: 79
Joined: Mon Feb 27, 2023 12:56 pm
Location: Berlin
Has thanked: 29 times
Been thanked: 18 times

Re: What about a mini MQTT broker on the ESP32?

Post by Monki »

Good morning Francesco,

can you tell me when we can test the system?
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Wed Oct 16, 2024 4:26 am Good morning Francesco,

can you tell me when we can test the system?
Hi Monki, thanks for your interest.
I'm still testing and evaluating the functionalities so I'm not ready to share something.
Since I’m doing this in my free time, I can’t really say when it’ll be done.
Anyway, so far all seems to work well (I tested with max 3 devices).
I’m still waiting for more MQTT modules to arrive so I can test several devices at once.
After that, I want to create a proper library of functions to include and document in Annex.
Don’t worry, I’ll give you a shout as soon as I’m ready!
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

Okay, I have completed the experimental version that includes the new features of mqttbroker.
You will find the preliminary documentation below.
I'm sending the download link to everyone who has expressed interest in testing.
If others are interested, just indicate your request in this post (with the desired module (esp32, esp32-c3, esp232S3, ..) and I will send you the link for download.

Happy testing!

Monki
Posts: 79
Joined: Mon Feb 27, 2023 12:56 pm
Location: Berlin
Has thanked: 29 times
Been thanked: 18 times

Re: What about a mini MQTT broker on the ESP32?

Post by Monki »

hello francesco,
I tested the Mqtt broker today with 11 shelly devices, so far everything has been running fine for about 4 hours. I was not able to successfully flash the ESP32-S3 QIO_QSPI variant on my atom s3 lite with the toolkit, the standard esp worked without any problems.
One more question: does the broker support the last will function? Would it also be possible to return the client list (MqttBroker.ListClients) as an array?
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Sun Oct 20, 2024 10:12 am hello francesco,
I tested the Mqtt broker today with 11 shelly devices, so far everything has been running fine for about 4 hours. I was not able to successfully flash the ESP32-S3 QIO_QSPI variant on my atom s3 lite with the toolkit, the standard esp worked without any problems.
One more question: does the broker support the last will function? Would it also be possible to return the client list (MqttBroker.ListClients) as an array?
Good news. About the QIO_QSPI version, I don't know if the toolkit support it, I don't use it anymore :(

I think that the broker do not support lastwill, I used this library : https://github.com/terrorsl/sMQTTBroker

edit : Yes, all the lists will be available as string (or string array) in a next version
User avatar
Electroguard
Posts: 1094
Joined: Mon Feb 08, 2021 6:22 pm
Has thanked: 373 times
Been thanked: 392 times

Re: What about a mini MQTT broker on the ESP32?

Post by Electroguard »

Been reading through the Docs to evaluate if would be suitable for "offline operation" on my isolated subnet, but noticed that MqttBroker.ListTopics prints to serial... so can offline broker serial output be achieved without a physical connection ?
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Sun Oct 20, 2024 10:57 am Been reading through the Docs to evaluate if would be suitable for "offline operation" on my isolated subnet, but noticed that MqttBroker.ListTopics prints to serial... so can offline broker serial output be achieved without a physical connection ?
Hi Robin,
for the moment the broker is in test phase.
This list is only for debug, so far, and represent the list of the topics who the broker is subscribed, not all the subscriptions done by all the clients.
I already planned to put this list into an array, eventually also the list of all the topics of all the clients.
Helmut_number_one
Posts: 163
Joined: Fri Dec 09, 2022 10:03 am
Location: Flensburg Deutschland
Has thanked: 114 times
Been thanked: 25 times

Re: What about a mini MQTT broker on the ESP32?

Post by Helmut_number_one »

I tested with shelly Uni and Open_DTU Solarinverter and it's running
MQTT->User, password and inverter serial number must be adjusted

Code: [Local Link Removed for Guests]

' Setup broker on port 1883 with login/password and debug modeenabled
print MqttBroker.Setup 1883, "Geraet_USER", "Passwort", 1
' Start the MQTT broker with a stack size of 20000 bytes
print MqttBroker.Start 20000
' Clear any existing topic subscriptions
print Mqttbroker.ClearTopics
  
print Mqttbroker.Subscribe("DTU_DUOsolar/1111111111/0/power")
 
print Mqttbroker.Subscribe("DTU_DUOsolar/1111111111/0/voltage")
  
print Mqttbroker.Subscribe("shellies/WasserstanShellyuni-C8C9A31B3893/adc/0")

print Mqttbroker.Subscribe("shellies/WasserstanShellyuni-C8C9A31B3893/relay/0")

print Mqttbroker.Subscribe("shellies/WasserstanShellyuni-C8C9A31B3893/relay/1")

print Mqttbroker.Subscribe("shellies/WasserstanShellyuni-C8C9A31B3893/input/0")

print Mqttbroker.Subscribe("shellies/WasserstanShellyuni-C8C9A31B3893/input/1")

' List all currently subscribed topics
'print mqttbroker.ListTopics
' Set up the MQTT event handler to jump to 'doMQTT' when an event occurs
onMQTT doMQTT
' Keep the program running indefinitely

wait
'result = MqttBroker.Publish("relay/1", "1", 1, 1)
' MQTT event handler for processing incoming messages
doMQTT:

If MQTT.Topic$ = "DTU_DUOsolar/1111111111/0/power" 
AnaWert$ = MQTT.Message$
Wlog "Wer: ",MQTT.Topic$,"  ", AnaWert$
end if

If MQTT.Topic$ = "shellies/WasserstanShellyuni-C8C9A31B3893/relay/0" 
Relais0$ = MQTT.Message$
Wlog "Wer: ", MQTT.Topic$,"  ", Relais0$
end if



If MQTT.Topic$ = "shellies/WasserstanShellyuni-C8C9A31B3893/relay/1" 
Relais1$ = MQTT.Message$
Wlog "Wer: ", MQTT.Topic$,"  ", Relais1$
end if


If MQTT.Topic$ = "shellies/WasserstanShellyuni-C8C9A31B3893/input/0" 
Input1$ = MQTT.Message$
Wlog "Wer: ", MQTT.Topic$,"  ", Input1$
end if


If MQTT.Topic$ = "shellies/WasserstanShellyuni-C8C9A31B3893/input/1" 
Input2$ = MQTT.Message$
Wlog "Wer: ", MQTT.Topic$,"  ", Input2$
end if


If MQTT.Topic$ = "DTU_DUOsolar/1111111111/0/voltage" 
Volt$ = MQTT.Message$
Wlog "Wer: ", MQTT.Topic$,"  ", Volt$
end if


return
MQTT.png
You do not have the required permissions to view the files attached to this post.
Last edited by Helmut_number_one on Sun Oct 20, 2024 8:01 pm, edited 1 time in total.
User avatar
cicciocb
Site Admin
Posts: 2784
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: What about a mini MQTT broker on the ESP32?

Post by cicciocb »

Thanks for the feedback.
In the while I'm implementing the Last Will and also improved the connection phase
Post Reply