helppp illegal base64 data at input byte 440 error

Toit

#helppp illegal base64 data at input byte 440 error

Thread channel in help

~~boss~~ 05/26/2025 11:41 AM

i was runing gpio example when this error show up

floitsch 05/26/2025 11:44 AM

Do you have any idea why this error happened?

floitsch 05/26/2025 11:44 AM

Is it reproducible?

floitsch 05/26/2025 11:45 AM

My immediate guess is that there was an exception on the device, and Jaguar tried to decode the stack trace but failed because it contained bad characters

floitsch 05/26/2025 11:46 AM

That could happen if we have concurrent writes on the UART (although the stack trace messages are normally small enough that they are written in one go) or maybe also a bad cable?

floitsch

Do you have any idea why this error happened?

~~boss~~ 05/26/2025 11:49 AM

floitsch

Do you have any idea why this error happened?

~~boss~~ 05/26/2025 11:50 AM

i think it was because i flash my device too many time and some file corrupts

floitsch 05/26/2025 11:52 AM

can you paste the actual base64 string?

OP~~boss~~

i think it was because i flash my device too many time and some file corrupts

floitsch 05/26/2025 11:53 AM

it's probably not the reason for the base64 error.

floitsch 05/26/2025 11:53 AM

But your screenshot confirms my suspicion. The "Received a Toit system message" is almost always a stacktrace for an exception.

~~boss~~ 05/26/2025 11:53 AM

floitsch

can you paste the actual base64 string?

~~boss~~ 05/26/2025 11:54 AM

this one?

floitsch 05/26/2025 11:54 AM

Wait: you are running with -d host. That means you are not running with a device.

OP~~boss~~

Click to see attachment.

floitsch 05/26/2025 11:54 AM

I mean as text.

floitsch 05/26/2025 11:54 AM

I can't copy the string and check it.

~~boss~~ 05/26/2025 11:54 AM

import gpio

// The ESP DevKit V1 has an LED connected to pin 2.
LED ::= 2

main:
pin := gpio.Pin.out LED
pin.set 1
sleep --ms=2000
pin.set 0

~~boss~~ 05/26/2025 11:55 AM

jag decode WyNVBVVYU1UQdjIuMC4wLWFscGhhLjE4MFNVAFskVSNVECSEkGZrEVWZtpebppNXCtFbI1UEVUVTVRdQUklNSVRJVkVfTE9PS1VQX0ZBSUxFRFNVHUZhaWxlZCB0byBmaW5kIHByaW1pdGl2ZSAyNTowWyNVAlVTWyNVD1sjbAAAAANVRlUASQEBWyNsAAAAA1VGVQFVB1sjbAAAAANVRlUCSQ32WyNsAAAAA1VGVQNJDlZbI2wAAAADVUZVBEkCZFsjbAAAAANVRlUFSQHQWyNsAAAAA1VGVQZV6VsjbAAAAANVRlUHSQglWyNsAAAAA1VGVQhJCeZbI2wAAAADVUZVCUkJ/VsjbAAAAANVRlUKSSRUWyNsAAAAA1VGVQtJJGxbI2wAAAADVUZVDEkjkFsjbAAAAANVRlUNSSQYWyNsAAAAA1VGVQ5JCD4

floitsch 05/26/2025 11:55 AM

Thanks.

floitsch 05/26/2025 11:56 AM

I'm getting the same base64 error. So that makes sense.

floitsch 05/26/2025 11:56 AM

Wait. No.

floitsch 05/26/2025 11:56 AM

Once I add the trailing '=' I actually get an error message:

ᐅ jag decode WyNVBVVYU1UQdjIuMC4wLWFscGhhLjE4MFNVAFskVSNVECSEkGZrEVWZtpebppNXCtFbI1UEVUVTVRdQUklNSVRJVkVfTE9PS1VQX0ZBSUxFRFNVHUZhaWxlZCB0byBmaW5kIHByaW1pdGl2ZSAyNTowWyNVAlVTWyNVD1sjbAAAAANVRlUASQEBWyNsAAAAA1VGVQFVB1sjbAAAAANVRlUCSQ32WyNsAAAAA1VGVQNJDlZbI2wAAAADVUZVBEkCZFsjbAAAAANVRlUFSQHQWyNsAAAAA1VGVQZV6VsjbAAAAANVRlUHSQglWyNsAAAAA1VGVQhJCeZbI2wAAAADVUZVCUkJ/VsjbAAAAANVRlUKSSRUWyNsAAAAA1VGVQtJJGxbI2wAAAADVUZVDEkjkFsjbAAAAANVRlUNSSQYWyNsAAAAA1VGVQ5JCD4=
PRIMITIVE_LOOKUP_FAILED error. 
Failed to find primitive 25:0
No such file: /home/flo/.local/state/toit/snapshots/24849066-6b11-5599-b697-9ba693570ad1.snapshot
Error: cannot decode stacktrace without snapshot for program: 24849066-6b11-5599-b697-9ba693570ad1

~~boss~~ 05/26/2025 11:57 AM

me too but when i delete = it say base64 error

floitsch 05/26/2025 11:57 AM

Could you also try to add the trailing '=' and see if that fixes the base64 error?

floitsch 05/26/2025 11:58 AM

The '=' is part of the base64 string. So you shouldn't delete it.

floitsch 05/26/2025 11:58 AM

So the primitive 25:0 error is because you are trying to use GPIO pins, but you are running on your host (desktop/laptop) machine.

floitsch 05/26/2025 11:58 AM

While we have support for GPIO pins on the raspberry pi, you would need to use a different API.

floitsch 05/26/2025 11:59 AM

Since you mentioned that you have flashed your device, I'm assuming you want to run the code on your ESP32.

floitsch 05/26/2025 11:59 AM

What you want to do:

floitsch 05/26/2025 11:59 AM

run jag scan

floitsch 05/26/2025 11:59 AM

If your ESP32 is powered on and managed to connect to the WiFi, you should see one device.

floitsch 05/26/2025 12:00 PM

Independently, and maybe in another terminal/tab, I would run jag monitor. That gives you the serial output of the device.

floitsch 05/26/2025 12:01 PM

That is the best way to see what the device is doing, and whether it is having any kind of error (like not being able to connect to the WiFi, ...)

floitsch 05/26/2025 12:01 PM

In my setup, I usually split the terminal into two (left/right), where the right is always just running jag monitor).

floitsch 05/26/2025 12:01 PM

If your program does a print, then you would also see that with jag monitor.

~~boss~~ 05/26/2025 12:02 PM

thanks i will try

~~boss~~ 05/27/2025 11:03 AM

i tried runing it on esp32 device name but it shows up another error

~~boss~~ 05/27/2025 11:03 AM

EXCEPTION error.
ILLEGAL_UTF8
0: ByteArrayBase.to-string <sdk>\core\collections.toit:1520:5
1: StringSegment.read-element D:\a\toit\toit\tools\snapshot.toit:1550:24
2: ListSegment.read-contents.<block> D:\a\toit\toit\tools\snapshot.toit:1385:25
3: Array.<block> <sdk>\core\collections.toit:876:31
4: SmallInteger.repeat <sdk>\core\numbers.toit:1288:3
5: Array <sdk>\core\collections.toit:876:10
6: List <sdk>\core\collections.toit:187:31
7: ListSegment.read-contents D:\a\toit\toit\tools\snapshot.toit:1385:12
8: SourceSegment.content D:\a\toit\toit\tools\snapshot.toit:1363:33
9: ListSegment.content D:\a\toit\toit\tools\snapshot.toit:1387:27
10: SourceSegment.read-string D:\a\toit\toit\tools\snapshot.toit:1370:21
11: MethodSegment.read-element D:\a\toit\toit\tools\snapshot.toit:1435:14
12: MapSegment.read-contents.<block> D:\a\toit\toit\tools\snapshot.toit:1405:18
13: SmallInteger.repeat <sdk>\core\numbers.toit:1288:3
14: MapSegment.read-contents_ D:\a\toit\toit\tools\snapshot.toit:1404:12
15: SourceSegment.content D:\a\toit\toit\tools\snapshot.toit:1363:33
16: MapSegment.content D:\a\toit\toit\tools\snapshot.toit:1409:26
17: Program D:\a\toit\toit\tools\snapshot.toit:90:68
18: SnapshotBundle.decode D:\a\toit\toit\tools\snapshot.toit:213:12
19: snapshot-to-image D:\a\toit\toit\tools\snapshot_to_image.toit:154:30
20: build-command.<lambda> D:\a\toit\toit\tools\snapshot_toimage.toit:127:16
21: Command.run.<block> <pkg:pkg-cli>\cli.toit:308:40
22: Parser.parse <pkg:pkg-cli>\parser_.toit:212:5
23: Command.run <pkg:pkg-cli>\cli.toit:306:12
24: main D:\a\toit\toit\tools\toit.toit:517:16

floitsch 05/27/2025 11:05 AM

Can you show the command you used? (And when posting code/output here try to use triple backticks like in markdown)(edited)

~~boss~~ 05/27/2025 11:10 AM

PS C:\Users\bossc\OneDrive\เดสก์ท็อป\test website> jajag scan
Scanning ...
v fierce-egg (address: http://192.168.1.126:9000, 32-bit)
PS C:\Users\bossc\OneDrive\เดสก์ท็อป\test website>   jag run test.toit
Running 'test.toit' on 'fierce-egg' ...
EXCEPTION error.

floitsch 05/27/2025 11:10 AM

Thanks. I'm investigating

floitsch 05/27/2025 11:13 AM

I'm currently not at my computer, but my guess is that we are running into a problem with code pages.

floitsch 05/27/2025 11:15 AM

That is, the compiler might not work in UTF-8 but a local code page.

floitsch 05/27/2025 11:15 AM

Could you try the following command:

Get-WinSystemLocale | Select-Object Name, DisplayName, 
                        @{ n='OEMCP'; e={ $_.TextInfo.OemCodePage } }, 
                        @{ n='ACP';

floitsch 05/27/2025 11:15 AM

That should give us the code page you are using.

floitsch 05/27/2025 11:16 AM

Ideally, I could get the snapshot that Jaguar is producing in the background. Let me see if there is an easy way to get to it

floitsch 05/27/2025 11:18 AM

Also: the likely reason for this problem is some non-ascii text in a string. If you have one in your file you could try to remove it for now (as a temporary workaround)(edited)

floitsch 05/27/2025 11:19 AM

That said. It could also be the file path

~~boss~~ 05/27/2025 11:22 AM

PS C:\Users\bossc\OneDrive\เดสก์ท็อป\test website> GeGet-WinSystemLocale                                    
                                                                          
LCID             Name             DisplayName
----             ----             -----------
1054             th-TH            Thai (Thailand)

~~boss~~ 05/27/2025 11:23 AM

PS C:\Users\bossc\OneDrive\เดสก์ท็อป\test website> @{Get-WinSystemLocale | Select-Object Name, DisplayName,                                           
>>                         @{ n='OEMCP'; e={ $_.TextInfo.OemCodePage } }, 
>>                         @{ n='ACP';   e={ $_.TextInfo.AnsiCodePage } }
   
Name  DisplayName     OEMCP ACP
----  -----------     ----- ---
th-TH Thai (Thailand)   874 874

floitsch 05/27/2025 11:24 AM

Right. So it's not UTF-8.

While it's not confirmed, you are likely hitting a bug in our tools when:
- running on windows
- not using a utf-8 code page
- having a string (maybe in the path) that isn't ASCII.

floitsch 05/27/2025 11:27 AM

You could try to change the code page:

$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding

(Maybe use chatgpt to confirm that this command is benign)

floitsch 05/27/2025 11:28 AM

With a bit of luck the jag command then succeeds.

floitsch 05/27/2025 11:28 AM

When I'm back at my computer I will see if we can force the correct code page from within our program.

floitsch 05/27/2025 02:29 PM

@~~boss~~

Can you show me (or send it to me: [email protected]) your test.toit?

floitsch 05/27/2025 02:30 PM

Going through the stack-trace it looks like toit is failing while trying to parse the name of an element. However that should always be ascii, so I'm not sure how we can get the error there.

floitsch 05/27/2025 02:32 PM

hmm. Actually, it might happen when we read the first name. At that point it reads in all strings; I think.

floitsch 05/27/2025 02:32 PM

So yes: if you could show me your test.toit that could help. If that contains non-ascii characters that probably explains it.

floitsch 05/27/2025 02:33 PM

Are you using vscode to edit the file?

floitsch 05/27/2025 02:44 PM

@~~boss~~

: If you are using vscode, please try to convert your input file to UTF-8: https://stackoverflow.com/a/40365121

~~boss~~ 05/27/2025 02:53 PM

~~boss~~ 05/27/2025 02:56 PM

i think it is already utf-8

~~boss~~ 05/27/2025 02:56 PM

on vscode

floitsch 05/27/2025 02:56 PM

Ok. Thanks.

floitsch 05/27/2025 02:56 PM

So it's not the input file.

floitsch 05/27/2025 02:57 PM

I'm guessing it must be the filepath.

~~boss~~ 05/27/2025 02:58 PM

i think so my file path contain my language so it cant convert to ascii?

floitsch 05/27/2025 02:59 PM

I'm still looking into it, but it looks to me like we get the file path in a non-utf8 encoding.

floitsch 05/27/2025 02:59 PM

And later we try to use the path in some way and fail.

floitsch 05/27/2025 02:59 PM

It could also be that there is a string in the core libraries that needs UTF-8 and was saved the wrong way.

floitsch 05/27/2025 03:01 PM

I'm adding more checks to the toit compiler now. Also trying to see if we can potentially get the paths in the wrong encoding.

floitsch 05/27/2025 03:12 PM

If you know how to do it, could you try to zip the lib files in your toit folder?
I think that would be in c:\Users\bossc\AppData\Local\jaguar\sdk\lib (maybe?)

I'm still leaning towards the filenames being the issue, but if the lib files were written in the wrong encoding, then that could maybe also be bad (and definitely something that would need to be fixed).

~~boss~~ 05/27/2025 03:18 PM

Ok I will try it out tomorrow (in my local time its night already)

floitsch 05/27/2025 03:18 PM

Thanks.

floitsch 05/27/2025 07:44 PM

I think I have found the problem. Working on a fix.
I hope to have something tomorrow.

~~boss~~ 05/28/2025 11:47 AM

i tried it and the error still shows up today

~~boss~~ 05/28/2025 11:47 AM

do i need to run some command to update toit ?

floitsch 05/28/2025 11:48 AM

We haven't released a new Toit yet.

floitsch 05/28/2025 11:48 AM

Until the next release maybe try to run outside the เดสก์ท็อป folder

floitsch 05/28/2025 11:48 AM

I think that's what's causing the issues.

floitsch 05/28/2025 11:49 AM

This PR https://github.com/toitlang/toit/pull/2855 (together with 2854 and 2856) should, hopefully, fix the issue.

Add UTF-8 manifest to all executables. by floitsch · Pull Request ...

This instructs Windows (10+) to interact with the program in UTF-8.

floitsch 05/28/2025 11:50 AM

They are now committed and should thus be in the next release. We will try to do one relatively soon.

~~boss~~ 05/28/2025 11:59 AM

i try using path thats contain english only then it worked!!

floitsch 05/28/2025 11:59 AM

Great. I really hope the next release will also work with non-utf8 paths.

~~boss~~ 05/28/2025 11:59 AM

thanks alot

floitsch 05/28/2025 12:00 PM

Fwiw, the issue isn't really that the path contains non-ascii, but that your system uses a non-utf8 code page.

floitsch 05/28/2025 12:00 PM

Our program assumed that everything it receives would be utf-8 and than crashed when it didn't.

floitsch 05/28/2025 12:01 PM

In other words: the compiler received the paths in the wrong encoding which it didn't like.(edited)

floitsch 05/28/2025 12:02 PM

From the user's point of view the distinction doesn't really make a difference. Just wanted to clarify.

floitsch 06/03/2025 08:00 AM

@~~boss~~

The new releases of Toit and Jaguar should handle your setup now. If not, please let us know.

~~boss~~ 06/06/2025 02:53 PM

floitsch 06/06/2025 02:53 PM

floitsch 06/06/2025 02:54 PM

Can you show the full stacktrace?

floitsch 06/06/2025 02:54 PM

And which Windows are you running?

floitsch

And which Windows are you running?

~~boss~~ 06/06/2025 03:04 PM

windows 11

floitsch 06/06/2025 03:05 PM

ok. In theory the fix should have worked for everything windows 10+.
So that's not it...

~~boss~~ 06/06/2025 03:05 PM

PS C:\Users\bossc\OneDrive\เดสก์ท็อป\test website> jajag run test.toit
Running 'test.toit' on 'fierce-egg' ...
EXCEPTION error. 
ILLEGAL_UTF_8
  0: ByteArrayBase_.to-string  <sdk>\core\collections.toit:1520:5
  1: StringSegment.read-element_ D:\a\toit\toit\tools\snapshot.toit:1550:24
  2: ListSegment.read-contents_.<block> D:\a\toit\toit\tools\snapshot.toit:1385:25
  3: Array_.<block>            <sdk>\core\collections.toit:876:31
  4: SmallInteger_.repeat      <sdk>\core\numbers.toit:1288:3
  5: Array_                    <sdk>\core\collections.toit:876:10
  6: List                      <sdk>\core\collections.toit:187:31
  7: ListSegment.read-contents_ D:\a\toit\toit\tools\snapshot.toit:1385:12
  8: SourceSegment.content     D:\a\toit\toit\tools\snapshot.toit:1363:33
  9: ListSegment.content       D:\a\toit\toit\tools\snapshot.toit:1387:27
 10: SourceSegment.read-string_ D:\a\toit\toit\tools\snapshot.toit:1370:21
 11: MethodSegment.read-element_ D:\a\toit\toit\tools\snapshot.toit:1435:14
 12: MapSegment.read-contents_.<block> D:\a\toit\toit\tools\snapshot.toit:1405:18
 13: SmallInteger_.repeat      <sdk>\core\numbers.toit:1288:3
 14: MapSegment.read-contents_ D:\a\toit\toit\tools\snapshot.toit:1404:12
 15: SourceSegment.content     D:\a\toit\toit\tools\snapshot.toit:1363:33
 16: MapSegment.content        D:\a\toit\toit\tools\snapshot.toit:1409:26
 17: Program                   D:\a\toit\toit\tools\snapshot.toit:90:68
 18: SnapshotBundle.decode     D:\a\toit\toit\tools\snapshot.toit:213:12
 19: snapshot-to-image         D:\a\toit\toit\tools\snapshot_to_image.toit:154:30
 20: build-command.<lambda>    D:\a\toit\toit\tools\snapshot_to_image.toit:127:16
 21: Command.run.<block>       <pkg:pkg-cli>\cli.toit:308:40
 22: Parser_.parse             <pkg:pkg-cli>\parser_.toit:212:5
 23: Command.run               <pkg:pkg-cli>\cli.toit:306:12
 24: main                      D:\a\toit\toit\tools\toit.toit:523:16

floitsch 06/06/2025 03:06 PM

Ok. That's promising.
Could you:

jag toit compile --snapshot -o test.snapshot test.toit

floitsch 06/06/2025 03:06 PM

That should yield a test.snapshot file.

floitsch 06/06/2025 03:07 PM

Could you then send this file to me? ([email protected])
I hope that it contains the broken Unicode, and, hopefully, an indication of what went wrong.

~~boss~~ 06/06/2025 03:09 PM

email sent!

floitsch 06/06/2025 03:09 PM

thanks.

floitsch 06/06/2025 03:09 PM

I will try to have a look a bit later today.

floitsch 06/06/2025 05:02 PM

It's definitely still the file-path: C:\Users\bossc\OneDrive\�ʡ�ͻ\test website\test.toit.
Now trying to be able to reproduce on our buildbot.

floitsch 06/07/2025 09:01 AM

I did reproduce, and I think I have a fix. I can already cross compile but still need to fight the build bot (which compiles natively). So looking good.
Again: thanks for the testing and helping us fix this.

floitsch 06/08/2025 12:19 PM

I think I have fixed the issue (assuming the build cycles green):
https://github.com/toitlang/toit/pull/2884
Edit: didn't go green... More debugging needed.
Edit 2: finally cycling green.(edited)

Fix windows unicode issues. by floitsch · Pull Request #2884 · to...

~~boss~~ 06/09/2025 08:43 AM

108 messages in total