toit build fails on ARM64

Toit

#toit build fails on ARM64

Thread channel in help

meercat 04/24/2023 03:29 AM

I'm building Jaguar on ARM64, and my toit build fails. generating sdk/bin/toit.lsp fails, saying undefined: atomic.Bool (with note: module requires Go 1.19

bitphlipphar 04/24/2023 03:32 AM

Which version of Go are you using?

bitphlipphar

Which version of Go are you using?

meercat 04/24/2023 03:33 AM

1.18.1 linux/arm64

meercat 04/24/2023 03:44 AM

now getting no rule to make target '.../toit.pkg', needed by 'install-dependencies'. Stop.

meercat 04/24/2023 03:56 AM

okay, updated to 1.20.3

bitphlipphar 04/24/2023 03:56 AM

I am in front of my computer now, so maybe I can help from here.

bitphlipphar 04/24/2023 03:57 AM

Maybe it's worth deleting the toit/build and jaguar/build directories.

bitphlipphar 04/24/2023 03:58 AM

Then if the submodules are correctly initialized start by building from toit.

bitphlipphar 04/24/2023 03:58 AM

Can you verify that toit/third_party/esp-idf isn't empty?

meercat 04/24/2023 04:03 AM

seems to be working now? I just did make all in my clone of Jaguar

bitphlipphar 04/24/2023 04:03 AM

That is good news!

bitphlipphar 04/24/2023 04:04 AM

You should have build/jag in your jaguar directory -- and toit/build/esp32/firmware.envelope should have the ESP32 bits.

bitphlipphar

That is good news!

meercat 04/24/2023 04:27 AM

hmm now it's File not found: '.../toit/tools/toit.run.toit'

bitphlipphar 04/24/2023 04:29 AM

Does it start with .../toit? (three dots)

bitphlipphar 04/24/2023 04:29 AM

What is your JAG_TOIT_REPO_PATH?

meercat 04/24/2023 04:29 AM

no sorry, I mean it has a directory -- /home/hydrologic/JagToit/toit/build/toit.run.toit

bitphlipphar 04/24/2023 04:30 AM

Ah, good

meercat 04/24/2023 04:30 AM

toit being the folder at which the clone is located(edited)

bitphlipphar

What is your JAG_TOIT_REPO_PATH?

meercat 04/24/2023 04:30 AM

//home/hydrologic/JagToit/toit

bitphlipphar 04/24/2023 04:30 AM

Can you build from toit/? Try something like make sdk.

meercat 04/24/2023 04:31 AM

tells me the same thing, can't find that file

OPmeercat

no sorry, I mean it has a directory -- /home/hydrologic/JagToit/toit/build/toit.run.toit

meercat 04/24/2023 04:31 AM

this one

bitphlipphar 04/24/2023 04:31 AM

kasper@mole toit % build/host/sdk/bin/toit.run examples/hello.toit 
Hello, World!

bitphlipphar 04/24/2023 04:32 AM

Do you have the file toit/tools/toit.run.toit?

meercat 04/24/2023 04:32 AM

no, I don't

OPmeercat

//home/hydrologic/JagToit/toit

bitphlipphar 04/24/2023 04:32 AM

Does it start with //?

bitphlipphar

Does it start with //?

meercat 04/24/2023 04:32 AM

it does

meercat 04/24/2023 04:33 AM

wait, I do have the file

meercat 04/24/2023 04:33 AM

toit.run.toit

meercat 04/24/2023 04:33 AM

and it is in that directory

bitphlipphar 04/24/2023 04:33 AM

Hmm.

bitphlipphar

Does it start with //?

meercat 04/24/2023 04:36 AM

same error after changing this to one /

meercat 04/24/2023 04:37 AM

bmentink 04/24/2023 07:25 AM

@bitphlipphar

Hi, I tried this as well. A make all seems to have built build/jag but I do not have a toit directory in my jaguar top level. Also, I got this error on the compile:

bmentink 04/24/2023 07:25 AM

make: *** No rule to make target '/home/bmentink/src/jaguar/build/sdk/bin/toit.pkg', needed by 'install-dependencies'.  Stop.

meercat 04/24/2023 07:27 AM

yup I got that too

bitphlipphar 04/24/2023 07:32 AM

Looks like JAG_TOIT_REPO_PATH isn't set correctly or isn't handled correctly by the Makefile.

bitphlipphar 04/24/2023 07:33 AM

It should look for $(JAG_TOIT_REPO_PATH)/build/host/sdk and find toit.pkg in there, not in jaguar/build.

bmentink 04/24/2023 07:34 AM

ok, how do I fix?

bitphlipphar 04/24/2023 07:34 AM

Maybe the way make starts a new shell is messing with us here.

bmentink 04/24/2023 07:35 AM

There is also another bug:

bmentink 04/24/2023 07:35 AM

./build/jag version
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xa2d3fc]

goroutine 1 [running]:
github.com/toitlang/jaguar/cmd/jag/commands.VersionCmd.func1(0x4000226000?, {0x12299e0?, 0x0?, 0x0?})
        /home/bmentink/src/jaguar/cmd/jag/commands/version.go:30 +0x5c
github.com/spf13/cobra.(*Command).execute(0x4000226000, {0x12299e0, 0x0, 0x0})
        /home/bmentink/go/pkg/mod/github.com/spf13/[email protected]/command.go:920 +0x5ac
github.com/spf13/cobra.(*Command).ExecuteC(0x4000212300)
        /home/bmentink/go/pkg/mod/github.com/spf13/[email protected]/command.go:1044 +0x340
github.com/spf13/cobra.(*Command).Execute(...)
        /home/bmentink/go/pkg/mod/github.com/spf13/[email protected]/command.go:968
github.com/spf13/cobra.(*Command).ExecuteContext(0xba4929?, {0xcc6530?, 0x40001b1b30?})
        /home/bmentink/go/pkg/mod/github.com/spf13/[email protected]/command.go:961 +0x4c
main.main()
        /home/bmentink/src/jaguar/cmd/jag/main.go:34 +0x17c

bmentink 04/24/2023 07:37 AM

Also, when I did git submodule update --init --recursive nothing happened ..

floitsch 04/24/2023 08:34 AM

Let's take a step back.
Here are some instructions that should work. Let me know at which point things go bad.

export TOIT_PATH=YOUR_ABSOLUTE_PATH_TO_THE_TOIT_CLONE
export JAG_PATH=YOUR_ABSOLUTE_PATH_TO_THE_JAGUAR_CLONE
export JAG_TOIT_REPO_PATH=$TOIT_PATH
# Check that the paths are set up correctly:
ls $TOIT_PATH
ls $JAG_PATH
# Let's do some cleanups. They shouldn't be necessary, but can't hurt.
cd $TOIT_PATH
make clean
cd $JAG_PATH
rm -rf build
# recursively init the submodules of Toit
cd $TOIT_PATH
git submodule update --init --recursive .
# While we are here, make sure we have the esp-idf requirements installed.
third_party/esp-idf/install.sh
# Let's build Toit here.
make
# This should take some time and compile quite a few things.
# If you get a build-error, it's likely the mbedtls issue I mentioned.
#   Instructions for that one are below.
# You should end up with files in:
ls build/host/sdk/bin
# If the build completed succesfully, let's build Jaguar now.
cd $JAG_PATH
make
# This will do another build in Toit (this time the ESP32 firmware), and then
# finish by building 'build/jag'.
build/jag version
# At this point you should be able to use the 'build/jag' executable as if it was
# a downloaded Jaguar binary.

floitsch 04/24/2023 08:36 AM

Now if there was a compilation error because of mbedtls:

cd $TOIT_PATH/third_part/esp-idf/components/mbedtls/mbedtls
wget https://raw.githubusercontent.com/toitlang/toit/438ab4e65213816f9ed60a72cde0cc89e374d6d7/aarch64.diff
patch -p1 < aarch64.diff
# And now try to build Toit again:
cd $TOIT_PATH
make

(edited)

floitsch

Now if there was a compilation error because of mbedtls: ``` cd $TOIT_PATH/third_part/esp-idf/components/mbedtls/mbedtls wget https://raw.githubusercontent.com/toitlang/toit/438ab4...(edited)

meercat 04/24/2023 09:56 AM

error relating to toit.run.toit, I think 404 when I run this wget command

meercat 04/24/2023 10:01 AM

nvm didn't have my glasses on LOL

floitsch

Now if there was a compilation error because of mbedtls: ``` cd $TOIT_PATH/third_part/esp-idf/components/mbedtls/mbedtls wget https://raw.githubusercontent.com/toitlang/toit/438ab4...(edited)

meercat 04/24/2023 10:06 AM

same error about toit.run.toit(edited)

meercat 04/24/2023 10:07 AM

also make clean returns rm: missing operand

meercat 04/24/2023 10:08 AM

and the submodule command doesn't return anything

OPmeercat

same error about toit.run.toit(edited)

meercat 04/24/2023 10:11 AM

despite the fact that the file does exist there and has information according to nano

floitsch 04/24/2023 10:48 AM

weird.

floitsch 04/24/2023 10:49 AM

Could you try to run the command by hand?

floitsch 04/24/2023 10:49 AM

On my machine it looks like the following:

/usr/bin/cmake -E env ASAN_OPTIONS=detect_leaks=false /home/flo/code/opentoit/build/host/sdk/bin/toit.compile --dependency-file /home/flo/code/opentoit/build/host/src/boot.dep --dependency-format ninja -w /home/flo/code/opentoit/build/host/generated/toit.run.snapshot /home/flo/code/opentoit/tools/toit.run.toit

(edited)

floitsch 04/24/2023 10:51 AM

But really, the command is:

/home/flo/code/opentoit/build/host/sdk/bin/toit.compile -w /home/flo/code/opentoit/build/host/generated/toit.run.snapshot /home/flo/code/opentoit/tools/toit.run.toit

The rest is just build-stuff.

OPmeercat

and the submodule command doesn't return anything

floitsch 04/24/2023 10:52 AM

That's ok. Once the submodules have been initialized 'git' doesn't print anything anymore.

OPmeercat

also make clean returns rm: missing operand

erikcorry_arbat 04/24/2023 11:41 AM

https://github.com/toitlang/toit/pull/1571 thanks for the report

floitsch

But really, the command is: ``` /home/flo/code/opentoit/build/host/sdk/bin/toit.compile -w /home/flo/code/opentoit/build/host/generated/toit.run.snapshot /home/flo/code/opentoit/to...

meercat 04/24/2023 03:28 PM

what's the prefix here?

meercat 04/24/2023 03:29 PM

as in what command*

floitsch 04/24/2023 03:29 PM

In this case /home/flo/code/opentoit would be TOIT_PATH.

meercat 04/24/2023 03:29 PM

I can't run the toit command

floitsch 04/24/2023 03:29 PM

do you have the file $TOIT_PATH/build/host/sdk/bin/toit.compile ?

meercat 04/24/2023 03:30 PM

yup

floitsch 04/24/2023 03:30 PM

ok. What does $TOIT_PATH/build/host/sdk/bin/toit.compile --version say?

meercat 04/24/2023 03:32 PM

v2.0.0-alpha.79.2+a4d6336e

floitsch 04/24/2023 03:32 PM

ok. So the compiler is compiled and seems to work.

floitsch 04/24/2023 03:33 PM

Does this work?

$TOIT_PATH/build/host/sdk/bin/toit.compile -w $TOIT_PATH/build/host/generated/toit.run.snapshot $TOIT_PATH/tools/toit.run.toit

meercat 04/24/2023 03:33 PM

brings up the toit usage

floitsch 04/24/2023 03:33 PM

can you show me?

meercat 04/24/2023 03:34 PM

floitsch 04/24/2023 03:34 PM

You can use $TOIT_PATH if you have set the variable. No need to expand it yourself.

floitsch 04/24/2023 03:34 PM

Can you show me your command-line you used?

meercat 04/24/2023 03:36 PM

just punched it in as you typed

meercat 04/24/2023 03:37 PM

says it can't find toit.run.toit

meercat 04/24/2023 03:37 PM

though in the directory it does exist and is populated

floitsch 04/24/2023 03:37 PM

interesting...

meercat 04/24/2023 03:37 PM

echo

ing the env vars does return the expected values

floitsch 04/24/2023 03:38 PM

Can you try $TOIT_PATH/build/host/sdk/bin/toit.run $TOIT_PATH/tools/toit.run.toit

floitsch 04/24/2023 03:38 PM

It should try to run the executable and then abort:

ᐅ $TOIT_PATH/build/host/sdk/bin/toit.run $TOIT_PATH/tools/toit.run.toit
EXCEPTION error. 
OUT_OF_BOUNDS
  0: SmallArray_.[]            <sdk>/core/collections.toit:899:5
  1: main                      ABSOLUTE_PATH/tools/toit.run.toit:105:28

(edited)

meercat 04/24/2023 03:43 PM

can't find toit.run

floitsch 04/24/2023 03:44 PM

ok. that could be. It makes sense that toit.run is built after the boot snapshot is generated.

floitsch 04/24/2023 03:45 PM

Looking through the code to see if there is a good way to debug this.

floitsch 04/24/2023 03:59 PM

Could you try this patch?

floitsch 04/24/2023 03:59 PM

patch --dry-run -p1 < arm64.patch

, and if that looks good: patch -p1 < arm64.patch.

floitsch 04/24/2023 04:00 PM

there is a weird work-around we added for running on older machines. And I wonder if that is hurting us now.

meercat 04/24/2023 04:03 PM

do I cd into /mbedtls/?

floitsch 04/24/2023 04:03 PM

No.

floitsch 04/24/2023 04:03 PM

This time from cd $TOIT_PATH.

floitsch 04/24/2023 04:04 PM

I have to run now.
Will check messages in ~25 minutes, and then again in a few hours.

meercat 04/24/2023 04:10 PM

now getting

Not a regular file: '/home/hydrologic/JagToit/toit/tools/package.lock'
Not a regular file: '/home/hydrologic/JagToit/toit/tools/toit.run.toit'

meercat 04/24/2023 04:14 PM

floitsch 04/24/2023 04:29 PM

@erikcorry_arbat

Do you have any ideas? This should just do a normal stat.

meercat 04/24/2023 04:29 PM

same stuff after running make clean btw

floitsch 04/24/2023 04:30 PM

@meercat

could you try to run stat /home/hydrologic/JagToit/toit/tools/toit.run.toit(edited)

floitsch 04/24/2023 04:31 PM

Will be offline again for a while. Will look at the messages in ~3 hours.

meercat 04/24/2023 04:33 PM

File: /home/hydrologic/JagToit/toit/tools/toit.run.toit
  Size: 4287          Blocks: 16         IO Block: 4096   regular file
Device: 1ah/26d    Inode: 388031      Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/hydrologic)   Gid: ( 1000/hydrologic)
Access: 2023-04-23 21:46:23.673297708 -0500
Modify: 2023-04-23 21:46:23.673297708 -0500
Change: 2023-04-23 21:46:23.673297708 -0500
 Birth: 2023-04-23 21:46:23.673297708 -0500

floitsch 04/24/2023 07:33 PM

That's really strange. Normally we use stat to see if a file exists. I wonder if we are using the wrong filesystem class. Will prepare another patch to confirm we are actually calling stat.

floitsch

That's really strange. Normally we use stat to see if a file exists. I wonder if we are using the wrong filesystem class. Will prepare another patch to confirm we are actually call...

meercat 04/24/2023 09:36 PM

gotcha. is toit meant to run on arm64 Ubuntu?

floitsch 04/24/2023 09:40 PM

We have Toit running on Linux Arm 32bit, and on Macos Arm 64 bit.

floitsch 04/24/2023 09:41 PM

My expectation would have been that it also works on Linux Arm 64 bit.

floitsch 04/24/2023 09:41 PM

But I don't think anyone every really tried.

floitsch 04/24/2023 09:41 PM

I will see if I can get a qemu image to work on my machine now.

floitsch

I will see if I can get a qemu image to work on my machine now.

meercat 04/24/2023 10:16 PM

sure, keep me posted

floitsch 04/24/2023 10:17 PM

running inside a qemu-emulated docker container right now.

floitsch

running inside a qemu-emulated docker container right now.

meercat 04/24/2023 10:44 PM

gotcha, is it working?

floitsch 04/24/2023 10:45 PM

still struggling to install the packages (python).

floitsch 04/24/2023 10:45 PM

due to being in a container things aren't working as normal.

meercat 04/24/2023 10:45 PM

ah yeah I fumbled a little with that initially idk if it's just me

bmentink 04/24/2023 10:48 PM

I have jag compiled/running on my M1 mac running Asahi (Arch) Linux, but it does not pull in the Toit source, do I have to do that manualy? If so, where does it need to be

bmentink 04/24/2023 10:48 PM

in relation to the jag source?

floitsch 04/24/2023 10:49 PM

If you have JAG_TOIT_REPO_PATH set up and built Jaguar, then it should automatically use the Toit that is in JAG_TOIT_REPO_PATH.

bmentink 04/24/2023 10:49 PM

I have not set up the variable, how do I do that?

floitsch 04/24/2023 10:49 PM

how did you compile jag?

bmentink 04/24/2023 10:50 PM

just a simple make(edited)

floitsch 04/24/2023 10:50 PM

Ok.

floitsch 04/24/2023 10:50 PM

If you clone https://github.com/toitlang/toit.git into a directory (usually next to Jaguar), then set the JAG_TOIT_REPO_PATH to it.

GitHub - toitlang/toit: Program your microcontrollers in a fast and...

Program your microcontrollers in a fast and robust high-level language. - GitHub - toitlang/toit: Program your microcontrollers in a fast and robust high-level language.

floitsch 04/24/2023 10:51 PM

Let's say you have a playground directory, and jaguar already in it.

floitsch 04/24/2023 10:53 PM

Then

cd playground
git clone https://github.com/toitlang/toit.git
export JAG_TOIT_REPO_PATH=$PWD/toit
cd toit
git submodule update --init --recursive
third_party/esp-idf/install.sh
make
cd ../jaguar
make

bmentink 04/24/2023 10:53 PM

Great, will switch over to that machine and give it a try .. thanks

floitsch 04/24/2023 10:54 PM

As mentioned earlier, you might need to apply a patch to get the compilation going.

floitsch 04/24/2023 10:54 PM

https://discord.com/channels/918498540232253480/1099899917574610944/1099976599660662845

bmentink 04/24/2023 10:54 PM

ok, remind me on the patch syntax again?

floitsch 04/24/2023 10:55 PM

cd $TOIT_PATH/third_party/esp-idf/components/mbedtls/mbedtls
wget https://raw.githubusercontent.com/toitlang/toit/438ab4e65213816f9ed60a72cde0cc89e374d6d7/aarch64.diff
patch -p1 < aarch64.diff

(edited)

bmentink 04/24/2023 10:55 PM

thanks!

bmentink 04/24/2023 11:04 PM

Is the above step ./third_party/esp-idf/install.sh or do I have to cd to that directory(edited)

floitsch 04/24/2023 11:05 PM

it's from within the toit directory.

bmentink 04/24/2023 11:05 PM

right

bmentink 04/24/2023 11:06 PM

Just waiting for my submodules to finish ..

floitsch 04/24/2023 11:06 PM

yeah... that takes a long time.

bmentink 04/24/2023 11:09 PM

just doing the install ..

floitsch 04/24/2023 11:09 PM

I'm finally making small progress. I was unable to install pip in the virtualized environment with apt. Now switched to just installing it in my home dir, and that seems to have worked. Just finished install the esp-idf dependencies.
I guess the next step is running a compile (which will take horribly long as the CPU is emulated).

floitsch 04/24/2023 11:09 PM

pretty much the same here.

bmentink 04/24/2023 11:12 PM

Tried a make, is the following error because of the required patch?

bmentink 04/24/2023 11:12 PM

[441/475] Generating ../generated/toit.run.snapshot
FAILED: generated/toit.run.snapshot /home/bmentink/src/toit/build/host/generated/toit.run.snapshot 
cd /home/bmentink/src/toit/build/host/src && /usr/bin/cmake -E env ASAN_OPTIONS=detect_leaks=false /home/bmentink/src/toit/build/host/sdk/bin/toit.compile --dependency-file /home/bmentink/src/toit/build/host/src/boot.dep --dependency-format ninja -w /home/bmentink/src/toit/build/host/generated/toit.run.snapshot /home/bmentink/src/toit/tools/toit.run.toit && /usr/bin/cmake -E cmake_transform_depfile Ninja gccdepfile /home/bmentink/src/toit /home/bmentink/src/toit/src /home/bmentink/src/toit/build/host /home/bmentink/src/toit/build/host/src /home/bmentink/src/toit/build/host/src/boot.dep /home/bmentink/src/toit/build/host/CMakeFiles/d/03dea3e29e5c1a8315551c5d33d65a59747c8b24fcd66724fb3956f58c69957c.d
File not found: '/home/bmentink/src/toit/tools/toit.run.toit'
ninja: build stopped: subcommand failed.
make: *** [Makefile:113: tools] Error 1
make  2:15.43 user 8.611 system 537% cpu (26.796 wasted time).

floitsch 04/24/2023 11:13 PM

Interesting. you are running into the same issue as cc-4477

floitsch 04/24/2023 11:13 PM

My machine is only at 34/475...

bmentink 04/24/2023 11:14 PM

Right, you have a way to go, my build to that point took seconds

floitsch 04/24/2023 11:14 PM

pfff..

floitsch 04/24/2023 11:14 PM

qemu is nice, but it's sometimes sooo slow.

bmentink 04/24/2023 11:15 PM

Yep, you need to grab someones M1 ..

floitsch 04/24/2023 11:15 PM

I'm still hoping that just running in the virtual machine will be faster than installing a Linux on Kasper's M1. But not so sure anymore

floitsch 04/24/2023 11:17 PM

and now I'm running into the issue that my image doesn't have go19.

bmentink 04/24/2023 11:17 PM

I am on 1.20.3

floitsch 04/24/2023 11:18 PM

seems like go18 is the last golang in this ubuntu version.

floitsch 04/24/2023 11:18 PM

I guess I accidentally picked ubuntu 18.04 or so.

bmentink 04/24/2023 11:19 PM

Which is why I love Arch based Distros ..

floitsch 04/24/2023 11:19 PM

but it's fine. Only interested in toit.compile right now.

floitsch 04/24/2023 11:19 PM

Me too.

floitsch 04/24/2023 11:19 PM

I'm Arch all the way, but found an ubuntu based docker container faster.

bmentink 04/24/2023 11:20 PM

Interesting .. what about an Alpine one?

floitsch 04/24/2023 11:20 PM

lol. It's now at 40/243.

floitsch 04/24/2023 11:20 PM

not going to go through all the installation again.

floitsch 04/24/2023 11:20 PM

I'm really just trying to figure out why toit.compile doesn't seem to find files.

floitsch 04/24/2023 11:20 PM

I can do that on ubuntu too

bmentink 04/24/2023 11:20 PM

I will go have a cuppa, see you soon ..

floitsch 04/24/2023 11:21 PM

"soon" ...

bmentink 04/24/2023 11:21 PM

... ish

floitsch 04/24/2023 11:34 PM

ok. Found the issue.

floitsch 04/24/2023 11:37 PM

I was so close with the initial patch that I cave to cc-4477. Just missed a few more places...

meercat 04/24/2023 11:37 PM

oh sweet

floitsch 04/24/2023 11:38 PM

You should be able to apply this patch from the Toit repository:

patch -p1 < stat.patch

floitsch 04/24/2023 11:40 PM

@meercat

you already have another patch applied on the same file. I think the easiest is to just undo that change:

# Go into your Toit repository. Then:
git checkout src/compiler/filesystem_local.cc

That should revert any changes to that file.
Then apply the new patch:

patch -p1 < stat.patch

You have another debug change (with Not a regular file but that shouldn't hurt).

floitsch 04/24/2023 11:41 PM

We will properly fix this tomorrow. I'm pretty sure we don't need the work-around for older Linux versions anymore.
This way we will do a clean stat call.

floitsch 04/24/2023 11:42 PM

aah. And I think I understand now, why you don't need the other patch I was talking about.
The cross-compiler uses clang and the native build system uses gcc. And it was clang that was hitting the warnings in mbedtls.

floitsch 04/24/2023 11:43 PM

so after fixing the stat thing, Toit on Linux arm64 should just work.

meercat 04/24/2023 11:46 PM

okay. toit make all worked

meercat 04/24/2023 11:47 PM

now do I need to build jaguar too?

floitsch 04/24/2023 11:47 PM

yes.

floitsch 04/24/2023 11:47 PM

simply go into the jaguar directory and type make there.

floitsch 04/24/2023 11:47 PM

as long as JAG_TOIT_REPO_PATH is set, it should start using it.

floitsch 04/24/2023 11:47 PM

It will actually do another compilation of Toit. This time for the esp32.

floitsch 04/24/2023 11:55 PM

And since I already had an ARM64 environment I also tested the cross-compiled executable there.

floitsch 04/24/2023 11:55 PM

It needed the same patch (stat issue), but after that it worked.

floitsch 04/24/2023 11:56 PM

Will see if we can add the ARM64 builds to the releases soon.

meercat 04/25/2023 12:06 AM

failed to open 'toit.bin' for reading (INVALID_ARGUMENT)

floitsch 04/25/2023 12:07 AM

Where does this come from?

meercat 04/25/2023 12:07 AM

make

on jaguar

bmentink 04/25/2023 12:08 AM

@floitsch

where do I get stat.patch from?

floitsch 04/25/2023 12:08 AM

https://discord.com/channels/918498540232253480/1099899917574610944/1100204163209629736

floitsch 04/25/2023 12:08 AM

I uploaded it here

OPmeercat

make on jaguar

floitsch 04/25/2023 12:09 AM

Doesn't ring a bell. Give me two minutes and I will see if I can find where it is coming from.

floitsch 04/25/2023 12:11 AM

Ok. I think it should be the toit.bin in the build/esp32 folder of the toit checkout.

floitsch 04/25/2023 12:11 AM

If JAG_TOIT_REPO_PATH is set, it should automatically build that, though.

bmentink 04/25/2023 12:13 AM

Toit compiled fine with the patch. Jaguar fails here .. almost completes:

bmentink 04/25/2023 12:13 AM

FAILED: CMakeFiles/firmware.util 
cd /home/bmentink/src/toit/build/esp32 && /home/bmentink/src/toit/toolchains/idf/components/toit/../../../../build/host/sdk/bin/toit.compile -w /home/bmentink/src/toit/build/esp32/system.snapshot --project-root /home/bmentink/src/toit/toolchains/idf/components/toit/../../../../system /home/bmentink/src/toit/toolchains/idf/components/toit/../../../../system/extensions/esp32/boot.toit && cd /home/bmentink/src/toit/build/esp32 && rm -f firmware.envelope && /home/bmentink/src/toit/toolchains/idf/components/toit/../../../../build/host/sdk/tools/firmware --envelope=firmware.envelope create --bootloader.bin=bootloader/bootloader.bin --firmware.bin=toit.bin --firmware.elf=toit.elf --partitions.bin=partition_table/partition-table.bin --partitions.csv=/home/bmentink/src/toit/toolchains/esp32/partitions.csv --otadata.bin=ota_data_initial.bin --flashing.json=flasher_args.json --system.snapshot=/home/bmentink/src/toit/build/esp32/system.snapshot && echo envelope: created && cd /home/bmentink/src/toit/build/esp32 && /home/bmentink/src/toit/toolchains/idf/components/toit/../../../../build/host/sdk/tools/firmware --envelope=firmware.envelope extract --format=binary -o toit-firmware.bin && echo envelope: firmware extracted
Failed to open 'toit.bin' for reading (INVALID_ARGUMENT).
ninja: build stopped: subcommand failed.
ninja failed with exit code 1
make[2]: *** [Makefile:239: esp32-no-env] Error 2
make[2]: Leaving directory '/home/bmentink/src/toit'
make[1]: *** [Makefile:234: esp32] Error 2
make[1]: Leaving directory '/home/bmentink/src/toit'
make: *** [Makefile:92: /home/bmentink/src/jaguar/build/sdk.build] Error 2
make  2:19.14 user 19.612 system 449% cpu (35.323 wasted time).

floitsch 04/25/2023 12:13 AM

Ok. So you too...

bmentink 04/25/2023 12:13 AM

yep [1257/1258]

floitsch 04/25/2023 12:14 AM

can you check if there is a toit.bin in your build/esp32 folder? (toit repository)

bmentink 04/25/2023 12:15 AM

yes

floitsch 04/25/2023 12:18 AM

I think it's the same issue.

floitsch 04/25/2023 12:20 AM

I think this should fix it. < EDIT: removed patch here>(edited)

floitsch 04/25/2023 12:20 AM

In the toit repository: patch -p1 < primitive.patch

floitsch 04/25/2023 12:21 AM

And thanks for your patience (both @meercat and @bmentink )

floitsch 04/25/2023 12:22 AM

nope.

floitsch 04/25/2023 12:22 AM

The patch doesn't work. Give me a sec to fix it.

floitsch 04/25/2023 12:24 AM

Updated version of primitive.patch:(edited)

bmentink 04/25/2023 12:25 AM

Do we reverse the previous patch when asked?

floitsch 04/25/2023 12:26 AM

shouldn't need to.

floitsch 04/25/2023 12:26 AM

can you show me the full message?

bmentink 04/25/2023 12:26 AM

patching file src/primitive_file_posix.cc
Reversed (or previously applied) patch detected! Assume -R? [n]

floitsch 04/25/2023 12:26 AM

Did you already apply the previous primitive.patch?

bmentink 04/25/2023 12:26 AM

yes

floitsch 04/25/2023 12:26 AM

If yes, just git checkout src/primitive_file_posix.cc first.

floitsch 04/25/2023 12:27 AM

That reverts any changes we made.

bmentink 04/25/2023 12:27 AM

ok thanks

floitsch 04/25/2023 12:27 AM

Then the new one should apply cleanly.

bmentink 04/25/2023 12:28 AM

yep, yay it all works .. jaguar completes

floitsch 04/25/2023 12:28 AM

nice.

bmentink 04/25/2023 12:29 AM

is there a make install?

floitsch 04/25/2023 12:29 AM

No.

floitsch 04/25/2023 12:29 AM

for now just add it to your path: export PATH="$PWD/build:$PATH"(edited)

bmentink 04/25/2023 12:30 AM

will do, thanks

floitsch 04/25/2023 12:30 AM

and don't forget to set JAG_TOIT_REPO_PATH.

floitsch 04/25/2023 12:30 AM

You still need that one to be set.

floitsch 04/25/2023 12:31 AM

We will probably fix the stat thing tomorrow.
And I will then try to improve our AUR scripts.

bmentink 04/25/2023 12:31 AM

sounds good

floitsch 04/25/2023 12:31 AM

But ideally we should have releases of arm64 (and since the cross-compilation seems to work, that's hopefully not too far in the future).

floitsch 04/25/2023 12:31 AM

Again; big thanks for all the testing.

bmentink 04/25/2023 12:32 AM

No problem at all ..

bmentink 04/25/2023 12:43 AM

For future updates, will it be just git checkout; make in each of the toit and jaguar dirs?(edited)

meercat 04/25/2023 01:03 AM

uh getting needed by 'install-dependencies'

bmentink 04/25/2023 01:12 AM

@floitsch

Confirmed: I can flash Jaguar onto a dev board and see it with jag scan cool .... containers install and run normal ..(edited)

bmentink 04/25/2023 02:46 AM

So I compiled yaourt for the M1 and it shows:(edited)

bmentink 04/25/2023 02:46 AM

yaourt -Ss jaguar
aur/jaguar-bin 1.9.20-1 (0) (0.00)
    Tool to develop Toit programs for the ESP32

bmentink 04/25/2023 02:46 AM

Is that a binary for x86 then?

meercat 04/25/2023 03:19 AM

`Project build complete. To flash, run this command:
/home/hydrologic/.espressif/python_env/idf3.2_py3.10_env/bin/python third_party/esp-idf/components/esptool_py/esptool/esptool.py -p (PORT) -b 460800 --before default_reset --after hard_reset --chip esp32  write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x1000 build/esp32/bootloader/bootloader.bin 0x8000 build/esp32/partition_table/partition-table.bin 0xd000 build/esp32/ota_data_initial.bin 0x10000 build/esp32/toit-firmware.bin
or run 'idf.py -p (PORT) flash'
make[2]: Leaving directory '/home/hydrologic/JagToit/toit'
make[1]: Nothing to be done for 'esptool'.
make[1]: Leaving directory '/home/hydrologic/JagToit/toit'
mkdir -p /home/hydrologic/JagToit/jaguar/build
echo "2023-04-25T03:10:32Z" > /home/hydrologic/JagToit/jaguar/build/sdk.build
/bin/sh: 1: cannot create /home/hydrologic/JagToit/jaguar/build/sdk.build: Permission denied
make: *** [Makefile:94: /home/hydrologic/JagToit/jaguar/build/sdk.build] Error 2

meercat 04/25/2023 04:00 AM

now getting this

meercat 04/25/2023 04:01 AM

bmentink 04/25/2023 04:12 AM

If that was a build of jaguar, I would trash that directory and get a fresh one .. and do another make all .. I presume you have built the toit stuff ?(edited)

bmentink

If that was a build of jaguar, I would trash that directory and get a fresh one .. and do another make all .. I presume you have built the toit stuff ?(edited)

meercat 04/25/2023 04:49 AM

yup toit builds fine

meercat 04/25/2023 04:50 AM

you mean rm -rf the directory?

meercat 04/25/2023 04:50 AM

and then reclone jaguar?

bmentink 04/25/2023 04:58 AM

Yep

meercat 04/25/2023 05:04 AM

still the same thing

meercat 04/25/2023 05:25 AM

any ideas @bitphlipphar ?

floitsch

Updated version of primitive.patch:(edited)

meercat 04/25/2023 07:21 AM

redoing this from scratch, do I need anymore patches besides this one?

OPmeercat

redoing this from scratch, do I need anymore patches besides this one?

floitsch 04/25/2023 07:48 AM

You also need this one: https://discord.com/channels/918498540232253480/1099899917574610944/1100204163209629736 (stat)

bmentink

Is that a binary for x86 then?

floitsch 04/25/2023 07:49 AM

It is. I guess we need to mark it as such.

bmentink 04/25/2023 07:51 AM

Ok, looking forward to arm64 version, even a -git one

OPmeercat

Click to see attachment.

floitsch 04/25/2023 08:09 AM

These look like the debug lines I added earlier. They don't look fatal.

floitsch 04/25/2023 08:10 AM

You could get rid of them by doing git checkout src/compiler/ and then applying the stat patch again.

meercat 04/25/2023 01:54 PM

hmm running jag after that returned nothing

meercat 04/25/2023 01:55 PM

as in, command not found

floitsch 04/25/2023 01:55 PM

What were your last steps?

floitsch 04/25/2023 01:56 PM

After a make in the jaguar repository?

meercat 04/25/2023 01:56 PM

that was before I reset everything. I'm gonna reapply the patches as mentioned and then make toit and make jaguar (am I required to make both separately?)

floitsch

After a make in the jaguar repository?

meercat 04/25/2023 01:56 PM

yup

floitsch 04/25/2023 01:57 PM

You don't need to do the make in Toit.

floitsch 04/25/2023 01:57 PM

The one in jaguar will do it for you.

floitsch 04/25/2023 01:57 PM

We just did it this way since the Toit compilation was failing.

floitsch 04/25/2023 01:57 PM

Just make sure you have the JAG_TOIT_REPO_PATH set.

floitsch 04/25/2023 01:57 PM

Once everything is compiled you should have a jag file in the build directory of jaguar.

floitsch 04/25/2023 01:58 PM

If it's there, just run it with build/jag.

floitsch 04/25/2023 01:58 PM

The make will not install the executable. So you need to point to it.

floitsch 04/25/2023 01:58 PM

(In more correct terms: jag is not in your PATH variable, and must thus be located via a relative or absolute path)

floitsch 04/25/2023 01:59 PM

(actually the build folder is not in PATH but I hope it's clear what I meant)

meercat 04/25/2023 02:02 PM

gotcha gotcha

floitsch

for now just add it to your path: export PATH="$PWD/build:$PATH"(edited)

meercat 04/25/2023 02:03 PM

so that's why this is necessary?

floitsch 04/25/2023 02:03 PM

exactly.

meercat 04/25/2023 02:03 PM

(are the quotes needed?)

floitsch 04/25/2023 02:03 PM

The quotes are only needed if one of your paths contains spaces or so.

floitsch 04/25/2023 02:03 PM

In general it's not.

floitsch 04/25/2023 02:03 PM

(and I rarely do it)

meercat 04/25/2023 02:03 PM

sweet. I'll take a look once I'm off work in an hour or so

meercat 04/25/2023 04:28 PM

okay, worked

meercat 04/25/2023 04:28 PM

Enter the URL of the ARM SDK (version v2.0.0-alpha.79)

floitsch 04/25/2023 04:28 PM

That should only happen if JAG_TOIT_REPO_PATH is not set.

meercat 04/25/2023 04:29 PM

it is set

floitsch 04/25/2023 04:30 PM

Ah. Are you running jag setup ?

floitsch 04/25/2023 04:30 PM

You don't need to do that with JAG_TOIT_REPO_PATH.

floitsch

Ah. Are you running jag setup ?

meercat 04/25/2023 04:30 PM

ah yea I was

floitsch

You don't need to do that with JAG_TOIT_REPO_PATH.

meercat 04/25/2023 04:30 PM

do I add this to the ~./bashrc?(edited)

meercat 04/25/2023 04:30 PM

so I don't have to do it every time I reload shell

floitsch 04/25/2023 04:31 PM

you can add PATH and JAG_TOIT_REPO_PATH with the correct values into your .bashrc

floitsch 04/25/2023 04:32 PM

Another common approach is to write the export JAG_TOIT_REPO_PATH=... and export PATH=... into a jag.env file and then write source jag.env.
That's equivalent to writing the lines that are in the file.

floitsch 04/25/2023 04:32 PM

Both are good options and mostly depending on preferences.

meercat 04/25/2023 04:33 PM

yeah I did an init.sh file that I'm just gonna run when I have this open

floitsch 04/25/2023 04:33 PM

just make sure to use source and not just calling it.

floitsch 04/25/2023 04:34 PM

When calling init.sh it won't change your current environment variables.

floitsch 04/25/2023 04:34 PM

Only source init.sh (or equivalently . init.sh) will do that.

meercat 04/25/2023 04:34 PM

right yeah

floitsch 04/25/2023 04:35 PM

btw. We committed the fixes for ARM64 to master.
So a future checkout of the Toit repository should compile on ARM64 without needing to apply patches.

meercat 04/25/2023 04:35 PM

awesome!

meercat 04/25/2023 04:35 PM

also can I connect to WPA2-Enterprise via jag flash?

floitsch 04/25/2023 04:36 PM

pff. Need to check what esp-idf supports. My initial guess would have been yes, but not sure.

floitsch 04/25/2023 04:37 PM

Looks like esp-idf supports it, but we might not expose it.

meercat 04/25/2023 04:40 PM

can you read the MAC address via some jag command?

floitsch 04/25/2023 04:40 PM

of the device?

meercat 04/25/2023 04:40 PM

yeah of the ESP board

floitsch 04/25/2023 04:41 PM

I'm not sure. I don't think so.
But you can use the esptool in the Toit build: build/host/sdk/tools/esptool read_mac

meercat 04/25/2023 04:46 PM

yup that worked, thanks

meercat 04/25/2023 04:47 PM

I think we can leave this thread

OPmeercat

also can I connect to WPA2-Enterprise via jag flash?

floitsch 04/25/2023 04:48 PM

I filed https://github.com/toitlang/toit/issues/1576

support wpa2-enterprise · Issue #1576 · toitlang/toit

It looks like Toit doesn't support WPA2-enterprise

floitsch 04/25/2023 04:48 PM

Note that it's currently not high priority, but a bit of nagging can change that

295 messages in total