HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Here we can discuss about the problem found
michaelvrba72
Posts: 15
Joined: Sat Mar 02, 2024 1:32 am
Location: Ořech, Czechia
Has thanked: 33 times
Been thanked: 2 times

HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by michaelvrba72 »

Hi, I have a strange problem with the speed of html operations (CLS, HTML, JSCALL...).

Code: [Local Link Removed for Guests]

OPTION.WLOG 0 '0-vypnuto, 1-normal, 2-vse jako html atd.
wlog date$ + " -- " + time$
print chr$ (13) + date$ + " -- " + time$ + " -- " + "Start programu."

cas = 0
cas_operace$ = ""

onHtmlReload vytvorit_stranku
onHtmlChange vytvorit_stranku
gosub vytvorit_stranku

timer0 2500, read_update
gosub read_update

wait

vytvorit_stranku:
CLS
B$ = ""
B$ = B$ + |<center>|
B$ = B$ + |<p><font color = 'black', style='FONT-FAMILY:Arial;font-weight:bold;font-size: 200%'>TEST</font></p>|
B$ = B$ + |</center>|
B$ = B$ + |<table width="312" border="0" align="center" style='FONT-FAMILY: Arial;font-weight:bold;font-size: larger'>|
B$ = B$ + |  <tbody>|
B$ = B$ + |    <tr>|
B$ = B$ + |      <td width="50%">OPERATION<br>TIME</td>|
B$ = B$ + |      <td id="tab1" width="50%" style='font-size: 200%'></td>|
B$ = B$ + |    </tr>|
B$ = B$ + |    <tr>|
B$ = B$ + |      <td>TIMER TIME</td>|
B$ = B$ + |      <td id="tab4"></td>|
B$ = B$ + |    </tr>|
B$ = B$ + |  </tbody>|
B$ = B$ + |</table><br>|
B$ = B$ + |</center>|
HTML b$
return

read_update:
wlog "START c –– " + str$ (millis - cas) + " ms"
cas_operace_0 = millis - cas
cas_operace_0$ = str$ (cas_operace_0) + " ms"
cas = millis
for c = 2000 to 2007
next c
js$ =       |_$('tab4').innerHTML = "| + cas_operace_0$ + |";|
js$ = js$ + |_$('tab1').innerHTML = "| + cas_operace$ + |";|
jscall js$
cas_operace = millis - cas
cas_operace$ = str$ (cas_operace) + " ms"
wlog "END c –– " + cas_operace$ + " freemem: " + str$ (ramfree)
return
Here is a test code that uses timer0 and every 2,5 seconds it runs a code where it calculates the duration and prints this to HTML via javascript. The normal duration time is 16 ms. With each additional client connected, the time increases a bit (16 ms => 28 ms... etc.). But when the connection to the client is lost (typically the client is a mobile phone and I turn off the mobile internet), after a time of tens of seconds the loop time increases significantly to (16 ms => 546 ms).

annex_01.png

After another minute, the serial log will start to output an "ERROR: Too many messages queued" error on every HTML (CLS, HTML b$, JSCALL ...) operation.

annex_02.png

After a longer period of time (10 minutes?) the Annex side closes the communication channel and returns to normal loop time. This condition cannot be resolved by anything other than waiting/restarting. Stopping the program or running other code does not help.

Annex32 CAN 1.53.7 LFS is running on EPS32.

Thank you for your help and advice!
You do not have the required permissions to view the files attached to this post.
Last edited by michaelvrba72 on Mon Jul 29, 2024 12:23 am, edited 1 time in total.
User avatar
PeterN
Posts: 641
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 296 times
Been thanked: 357 times
Contact:

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by PeterN »

Just a thought ... I'm not sure ...
Error messages could come from WLOG
Try disabling the WLOG statement lines with a comment character; don't just use option.wlog 0
michaelvrba72
Posts: 15
Joined: Sat Mar 02, 2024 1:32 am
Location: Ořech, Czechia
Has thanked: 33 times
Been thanked: 2 times

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by michaelvrba72 »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Sun Jul 28, 2024 8:45 pm Just a thought ... I'm not sure ...
Error messages could come from WLOG
Try disabling the WLOG statement lines with a comment character; don't just use option.wlog 0
I tried everything I could think of. This is a broken connection problem with the browser. Normally when you close the browser, the connection is terminated. Annex prints "ws[/ws][27] disconnect" and it's OK. But if you disconnect along the way, this doesn't happen and you get this problem and the recurring error message "ERROR: Too many messages queued". This condition only stops when the dead connection terminates Annex. Unfortunately, even restarting the connection does not cause the condition to terminate because a new connection number is assigned. In purely local traffic from the PC this does not occur, but in traffic over the mobile network + VPN to the local network it does.
User avatar
PeterN
Posts: 641
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 296 times
Been thanked: 357 times
Contact:

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by PeterN »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 5:22 am
…. Annex prints "ws[/ws][27] disconnect" and it's OK. But if you disconnect along the way, this doesn't happen and you get this problem and the recurring error message "ERROR: Too many messages queued". This condition only stops when the dead connection terminates Annex. Unfortunately, even restarting the connection does not cause the condition to terminate because a new connection number is assigned. In purely local traffic from the PC this does not occur, but in traffic over the mobile network + VPN to the local network it does.
That was my thought. What happens to the Winsock messages that are still constantly generated by WLOG but can no longer find their destination? They are put in a queue, and the waiting messages are kept in a limited memory of the ESP32.
This will not occur for pure http traffic.
Your logging seems to be for clarity only. So try it out and disable the wlog lines - maybe just to disprove my theory ;-)
michaelvrba72
Posts: 15
Joined: Sat Mar 02, 2024 1:32 am
Location: Ořech, Czechia
Has thanked: 33 times
Been thanked: 2 times

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by michaelvrba72 »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 7:33 am
[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 5:22 am
…. Annex prints "ws[/ws][27] disconnect" and it's OK. But if you disconnect along the way, this doesn't happen and you get this problem and the recurring error message "ERROR: Too many messages queued". This condition only stops when the dead connection terminates Annex. Unfortunately, even restarting the connection does not cause the condition to terminate because a new connection number is assigned. In purely local traffic from the PC this does not occur, but in traffic over the mobile network + VPN to the local network it does.
That was my thought. What happens to the Winsock messages that are still constantly generated by WLOG but can no longer find their destination? They are put in a queue, and the waiting messages are kept in a limited memory of the ESP32.
This will not occur for pure http traffic.
Your logging seems to be for clarity only. So try it out and disable the wlog lines - maybe just to disprove my theory ;-)
That's an interesting thought, but the wlog doesn't cause it. I've tried disabling it before and now I've made the attempt again. I removed all wlog and still disabled it with the "OPTION.WLOG 0" command. But those Winsock messages might be those automatic time updates via javascript or other interactions via the web interface.
I've made two attempts.
The first one - completely without updating the web page - the problem exists, but does not manifest itself.
Second - instead of javascript, I redraw the page every 2,5 seconds - the problem is even bigger - the delay is 1095 ms.
So you are right where the problem will be, just not so easy to solve. Every html operation is trying to send to a dead Winsock and causing huge delays in running the program and generating error messages.
User avatar
cicciocb
Site Admin
Posts: 2781
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by cicciocb »

Here are some explanations:

All the commands (WLOG, HTML, CSS, etc.) use the same WebSocket communication channel and are essentially the same command with slight variations that are interpreted and handled differently by the JavaScript side of the browser.

The WebSocket server hosted in the ESP is based on an asynchronous implementation so that, each time a WLOG, HTML, etc., command is executed, the command is stored in a queue and handled in the background.

Obviously, because of the limited memory size of these devices, the queue has a limited size. As soon as it saturates, the message "ERROR: Too many messages queued" is shown, and a 500ms delay is triggered.

This delay is required to ensure that all the messages waiting in the queue have enough time to be flushed, preventing the queue from saturating and losing the message to be sent.

If I understand correctly, the problem is that the server does not immediately recognize when a connection is lost (it seems this happens after about 10 minutes).

When the browser closes the connection, it sends a "disconnect" message so the WebSocket server knows that the connection is lost. However, in your case, this information is not received, so the server waits for a very long timeout.

I can see if this can be changed, but I fear that this risks having a negative impact on the overall behavior of the module.
michaelvrba72
Posts: 15
Joined: Sat Mar 02, 2024 1:32 am
Location: Ořech, Czechia
Has thanked: 33 times
Been thanked: 2 times

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by michaelvrba72 »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 2:18 pm Here are some explanations:

All the commands (WLOG, HTML, CSS, etc.) use the same WebSocket communication channel and are essentially the same command with slight variations that are interpreted and handled differently by the JavaScript side of the browser.

The WebSocket server hosted in the ESP is based on an asynchronous implementation so that, each time a WLOG, HTML, etc., command is executed, the command is stored in a queue and handled in the background.

Obviously, because of the limited memory size of these devices, the queue has a limited size. As soon as it saturates, the message "ERROR: Too many messages queued" is shown, and a 500ms delay is triggered.

This delay is required to ensure that all the messages waiting in the queue have enough time to be flushed, preventing the queue from saturating and losing the message to be sent.

If I understand correctly, the problem is that the server does not immediately recognize when a connection is lost (it seems this happens after about 10 minutes).

When the browser closes the connection, it sends a "disconnect" message so the WebSocket server knows that the connection is lost. However, in your case, this information is not received, so the server waits for a very long timeout.

I can see if this can be changed, but I fear that this risks having a negative impact on the overall behavior of the module.
Thank you for the explanation, Master Yoda. :D

I could see a possible solution in the possibility to set a timeout for automatic termination of the connection by Annex. This would allow to leave the behavior as it is now or use a new timeout adjustment.

Maybe at the moment of transition to an error message to terminate dead connections (which are terminated in long minutes anyway)?
That way delays and error messages would not be generated.
User avatar
cicciocb
Site Admin
Posts: 2781
Joined: Mon Feb 03, 2020 1:15 pm
Location: Toulouse
Has thanked: 592 times
Been thanked: 1990 times
Contact:

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by cicciocb »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 2:48 pm Thank you for the explanation, Master Yoda. :D

I could see a possible solution in the possibility to set a timeout for automatic termination of the connection by Annex. This would allow to leave the behavior as it is now or use a new timeout adjustment.

Maybe at the moment of transition to an error message to terminate dead connections (which are terminated in long minutes anyway)?
That way delays and error messages would not be generated.
Well, I think I found a configuration that seems to work well.
Testing in progress ....
michaelvrba72
Posts: 15
Joined: Sat Mar 02, 2024 1:32 am
Location: Ořech, Czechia
Has thanked: 33 times
Been thanked: 2 times

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by michaelvrba72 »

[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 5:47 pm
[Local Link Removed for Guests] wrote: [Local Link Removed for Guests]Mon Jul 29, 2024 2:48 pm Thank you for the explanation, Master Yoda. :D

I could see a possible solution in the possibility to set a timeout for automatic termination of the connection by Annex. This would allow to leave the behavior as it is now or use a new timeout adjustment.

Maybe at the moment of transition to an error message to terminate dead connections (which are terminated in long minutes anyway)?
That way delays and error messages would not be generated.
Well, I think I found a configuration that seems to work well.
Testing in progress ....
If you need to join the testing, I'm available. :)
User avatar
PeterN
Posts: 641
Joined: Mon Feb 08, 2021 7:56 pm
Location: Krefeld, Germany
Has thanked: 296 times
Been thanked: 357 times
Contact:

Re: HTML SPEED PROBLEM WHEN ONE OF MULTIPLE CONNECTED CLIENTS LOSES CONNECTION

Post by PeterN »

Francesco was again faster than lightning and added the update here ...
[Local Link Removed for Guests]
Post Reply