[wifi] DEBUG: closing | freezes on alpha.63+

Toit

#[wifi] DEBUG: closing | freezes on alpha.63+

Thread channel in help

Rikke 03/13/2023 01:48 PM

Hello,

I've noticed how the esp32 can freeze and never recover, after getting the [wifi] DEBUG: closing. I've seen this issue when the esp32 goes completely out of memory, and now after not being able to read from sd-card.

******************************************************************************
Decoding by `jag`, device has version <2.0.0-alpha.64>
******************************************************************************
EXCEPTION error. 
FILE_NOT_FOUND
  0: stat                      <pkg:pkg-host>/file.toit:197:3
  1: is_directory              <pkg:pkg-host>/file.toit:178:11
  2: get_file_names            ../src/sd_card/sd_card_driver.toit:68:6
  3: delete_old_files          ../src/persistance.toit:182:22
  4: handle_data_message.<lambda> ../src/persistance.toit:39:21
******************************************************************************

[wifi] DEBUG: closing

Nothing happens after this, when it is supposed to print out something 1-5 times every second.
I'm unsure how to provide more useful information right now, but I didn't notice this on alpha-47 and I saw this problem when out of memory on alpha-60

bitphlipphar 03/13/2023 01:53 PM

One thing that might have changed fairly recently is that your apps may need to be run with the --critical flag.

bitphlipphar 03/13/2023 01:53 PM

Do you use tools/firmware to build up the bits for your devices?

Rikke 03/13/2023 01:55 PM

Yes, compile with sdk/bin/toit.compile, and install with sdk/tools/firmware

bitphlipphar 03/13/2023 01:55 PM

Without giving it too much consideration, I think we've ended up flipping the default for the --critical flag which makes the system restart if the apps terminates with an error.

bitphlipphar 03/13/2023 01:56 PM

When you use tools/firmware container install you can pass --critical to it.

bitphlipphar 03/13/2023 01:56 PM

With that flag, the system process will try to recover through a restart in case an app fails due to unforeseen issues.

bitphlipphar 03/13/2023 01:56 PM

That used to be the default.

Rikke 03/13/2023 01:56 PM

Ooh I can try that quickly, the esp would normally restart in these situations

Rikke 03/13/2023 01:57 PM

Oh and thats actually nice for our tests on hardware, I had a problem with esp restarting when a test failed, and kept rerunning the tests

bitphlipphar 03/13/2023 08:33 PM

Let me know if adding back --critical improves things for you, @Rikke.

Rikke 03/14/2023 07:35 AM

It looks like it is working now again

ty. I added the critial flag to our building server

12 messages in total