Your Pages‎ > ‎Users Pages‎ > ‎Robin‎ > ‎

Communications

Your comment did not "demotivate" me Francesco - I am not worried about IF the protocol works... the problem is merely HOW with my limited programming skills to parse and manipulate message contents without adding too much processing burden.
Unfortunately I am at the limit my programming capabilites, so although I can see where I want to go, I don't know how to get there.
In particular, how to decrement the RETRIES=values of all the retried messages in the SentQ, and delete specific ID=identifier messages.

If you remember our original UDP conversations last century, and the important functionality that resulted from them, I consider this at least as important for Baby to blossom.
That is why I have taken much time and effort to create this page solely for the purpose of explaining the concept to you.


The concept is for a simple but flexible network protocol that can be easily extended to provide comprehensive functionality if wished.
Based on UDP whereby messages are broadcast to all nodes, but only the intended Target node takes any interest in the message.
At its simplest, the sender just sends an Instruction to a Target node:
TargetName  Instruction.

The Sender might also include optional data parameters for the Target to retrieve, just by including them in the message after the Instruction.
Only the Sender and Target nodes will have any interest in the message contents.
TargetName  Instruction  Parameter1 ... Parameter9

If the Sender requires confirmation that the message reached Target it could include a unique identifier (eg:UnixTime) to the message, and add the sent message into a Sent_Messages_Queue awaiting return of a confirmation ACKNOWLEDGEMENT with the same identifier.
Any designated Target receiving a message with the optional ID=identifier attached would echo that message back to the sender.
Any Sender receiving back an ACKNOWLEDGEMENT echo would then delete that acknowledged message from the SentQ.
TargetName  Instruction  ID=unique_identifierr

The SentQ would thus hold a record of all un-ACKnowledged messages.
This gives opportunity for including an optional DataLoggerNode, offering the equivalent of MQTT 'Last Will & Testament' facility, whereby the SentQ could be periodically purged of un-ACKnowledged messages and those details be recorded on the designated DataLoggerNode.

The SentQ also offers opportunity for un-acknowledged messages to be re-transmitted.
While un-acknowledged messages remain in the SentQ, an interval timer could keep stepping through them to re-transmit each one.
To avoid keep flogging a dead horse, each message might optionally be assigned a TIME2LIVE=duration and/or a RETRIES=value
Details of all messages deleted from the SentQ for any reason, could be recorded on the optional designated DataLoggerNode if available.
The TIME2LIVE option would allow a message to remain in the SentQ for periodic re-transmission until it was deleted when its duration expired.
TargetName
  Instruction  ID=unique_identifierr  TIME2LIVE=duration

The RETRIES option lets messages be periodically re-transmitted and have their RETRIES decremented until being deleted when RETRIES=0.
TargetName  Instruction  ID=unique_identifierr  RETRIES=value


It should be noted that all enhancements are only extra options to the basic Target Instruction message format.

Other enhancements could be added to individual nodes without affecting any of the other nodes or overall system functionality.
For instance, a node might be chosen to also act as a Bridge node, so all non-local messages not targeted at it would simply be echoed out using a different communications stream, without caring about contents. An incoming Serial message could be appended with STREAM=1, and an incoming Serial2 message could be appended with STREAM=2, therefore the absence of "STREAM=" could indicate a default of UDP.

The bridged device could be on a different subnet, or even a different platform (eg: Rasberry Pi), it is merely a device with serial connection.
The following Serial2 bridged device would be requesting message ACKnowledgement from the Target, which the Bridge would relay via Serial2.
TargetName  Instruction  ID=123r  RETRIES=5  STREAM=2


Future enhancements could easily be added to individual nodes, or to the overall system, without affecting the existing core operation.
Eg: nodes might be able to designate other nodes to keep a 'watchdog' eye on them, and report failures to an optional DataLoggerNode

The following 'theoretical example program incorporates some of the above optional facilities.
It can be addressed by other nodes using its NodeName, or GroupName, or IPaddress... the Groupname can can comprise multiple groups.
The node is assumed to have a Relay, a SAM speaker module connected by Serial Port, and connect to another subnet node via Serial2.
It is configured as BRIDGE, so will automatically echo all non-local messages between UDP and Serial2.
It recognises the 2 instructions contained in its instructionslist$, Speak and RelayToggle, so might be sent messages such as these...

  SamSpeaker  SPEAK  "phrase to speak"
  SAMSPEAKER  RelayToggle 
  SAMSpeaker  RelayToggle  ID=456r  RETRIES=3


Here is a non-working example script which attempts to demonstrate many of the concepts.
I have managed to implement much of the protocol - and for the purpose of illustration I have 'invented' some 'virtual' commands thatallow me to more easily demonstrate what I am having trouble with, which I've colour-highlighted to separate fact from fiction. But considering that it even already includes transparent BRIDGING capability, hopefully you will be able to help it towards a practical working solution.

nodename$  = "SAMspeaker"
groupname$ = "Announcers\Speakers\Relays"
localIP$ = WORD$(IP$,1)
netIP$   = WORD$(localIP$,1,".") + "." + WORD$(localIP$,2,".") + "." + WORD$(localIP$,3,".") + "."
nodeIP$ = WORD$(localIP$,4,".")
udpport = 5001                      
if nodename$ = "" then nodename$ = "Node" + nodeIP$            '(ensures all nodes must have unique nodenames)
bridge = 1  '0 for just local node, 1 for udp bridge to serial, 2 for udp bridge to serial2, 3 for udp bridge to serial and serial2
instructionslist$ = ucase$("Speak RelayToggle ")  'List of Subdir branches available to action as remote triggers
instruction$ = "" 
RXmsg$ = ""  'variable to hold incoming message
sentq$ = ""  'variable to hold all sent messages that require acknowledgement (ie: "ID=nn") while waiting to be acknowledged
qdelimiter$ = "*"  'separates messages in the sentq
DataLoggerNode = ""     'Target name of logging node for reporting error events, if available (will be useful if SD storage becomes available)
relaypin = 12: pin.mode relaypin, output: pin(relaypin) = 0     'initialise relay to OFF
buttonpin = 0: pinmode buttonpin, input, pullup 
interrupt buttonpin, pressed
timer1 5000, ReTransmit                          
udp.begin(udpport)
onudp udpRX
'onserial  serial1RX
'onserial2 serial2RX
wait

serial1RX:
RXmsg$ = serial.input + " STREAM=SER1"
gosub RXmsg
return

serial2RX:
RXmsg$ = serial2.input + " STREAM=SER2"
gosub RXmsg
return

udpRX:
RXmsg$ = udp.read$
gosub RXmsg
return

RXmsg:
target$ = ucase$(word$(RXmsg$,1))   'Target is always first word of message - may be NodeName or GroupName or "ALL" or localIP addr
if (target$=localIP$) OR (target$=ucase$(nodename$)) OR (target$=ucase$(groupname$)) OR (instr(ucase$(groupname$),target$)>0) OR (target$="ALL") then      '"Target Name or IP matched"
 if instr(ucase$(word$(RXmsg$,params)), "RETRIES=ACK") > 0 then
  gosub DeleteFromSentQ  'is a returned acknowledgement, so just delete original msg from sentq$
 else     'is a new incoming instruction
  instruction$ = trim$(ucase$(word$(RXmsg$,2)))   'Instruction is always second word of message
  if word.find(ucase$(instructionslist$),instruction$) > 0 then  'check if a recognised as valid on local instructionslist$
   'is a recognised local instruction, so check if ACK reply is required (requested by adding ID=time$ to the msg)
   if instr(ucase$(RXmsg$), "ID=") > 0 then    'requires sending back ACKnowledgement that message is received
    if word.parse$(RXmsg$, "STREAM=", " ") = "SER2 then print2 RXmsg$ + " RETRIES=ACK"
    elseif word.parse$(RXmsg$, "STREAM=", " ") = "SER1 then print RXmsg$ + " RETRIES=ACK"
    else udp.reply RXmsg$ + " RETRIES=ACK"
    endif   'word.parse$( "STREAM="
   endif   'instr( "ID="
   gosub instruction$       'branch to action the corresponding instruction subroutine
  else
   udp.reply RXmsg$ + " INSTRUCTION NOT RECOGNISED"
  endif 'word.find instruction$
 endif 'instr("RETRIES=ACK"
else    'not target
 if bridge = 1 then print RXmsg$
 elseif bridge = 2 then print2 RXmsg$
 elseif bridge = 3 then
  print RXmsg$
  print2 RXmsg$
 endif   'bridge=1 
endif   'target=
return

DeleteFromSentQ:
'The original version of current RXmsg$ ID=time$ message needs deleting from sentq, but they are no longer identical.
RXmsgID$ = word.parse(RXmsg$, "ID=", " ")   'the identifier of the message to look for
for count = 1 to word.count(sentq$,"*")     'step through each message in the sentq
 countmsg$ = word$(sentq, count, "*")
 if instr(countmsg$, RXmsgID$) > 0 then sentq$ = word.delete$(sentq$, count,"*")   'if sentq msg identifer matches, delete msg from q
next count
return

ReTransmit:
'This is my stumbling block... it's a word manipluation problem - I'm unable to find a practical way to parse each un-ACKed message
' still in the sentq to decrement their "RETRIES=x" value, ReTransmit the message, else delete it from sentq when RETRIES = 0 

If RETRIES = 0 then
 'Delete failed message from SentQ
 if DataLoggerNode <> "" then
  udp.write netip$ + "255", udpport, "DataLoggerNode + " Failure to Send msg " + (failed message)
 endif 'LOGnode
endif  'RETRIES = 0
return

pressed:
if pin(buttonpin) = 0 then
 'send message to Node2, add ID=time$ to signify this message ID requires an ACKnowledgement, add RETRIES=5 to send up to 5 times
 sendmsg$ "Node2 RelayToggle " + " ID=" + time$ + RETRIES=5"
 sentq$ = sentq$ + " " + sendmsg$
 udp.write netip$ + "255", udpport, sendmsg$
 if bridge = 1 then print sendmsg$
 elseif bridge = 2 then print2 sendmsg$
 elseif bridge = 3 then
  print sendmsg$
  print2 sendmsg$
 endif   'bridge = 1
endif   'pin(buttonpin)
return

relaytoggle:
if pin(relaypin) = 1 then pin(relaypin) = 0 else pin(relaypin) = 1
return

SAMspeak:   
print word.parse$(udpmsg, |data="|, |"|)  'Extracts data words in quotes that follow the SAMspeak instruction, and sends to SAM
return
'-------------------- End ---------------------