local packages registry error

Toit

#local packages registry error

Thread channel in help

kaxori 09/11/2024 04:13 PM

I created a local packages folder "T:\packages" and tried to add it to the registry using:

jag pkg registry add local t:\packages --local

it returned with:

Unhandled error: open C:\Users\Axel\.config\toit\config.yaml: Das System kann den angegebenen Pfad nicht finden.

=> then I copied C:\Users\Axel\.config\jaguar\config.yaml to

C:\Users\Axel\.config\toit

and the command above succeeded.

jag pkg registry list
toit: github.com/toitware/registry (git)
local: t:\packages (local)

A path specification may need to be adjusted ?

kaxori 09/12/2024 08:32 AM

Is it possible to use a 'global package' inside the example folder of a 'local package' ?

import pixel_strip show *   // github.com/toitware/toit-pixel-strip
import rgb-strip show *     // local package

reports: Package for prefix 'pixel-strip' not found

I struggle with package.yaml/.lock ?
Does anyone have a tip?(edited)

floitsch 09/12/2024 03:39 PM

Sorry. Didn't see your messages. Reading through them now.

OPkaxori

I created a local packages folder "T:\packages" and tried to add it to the registry using: jag pkg registry add local t:\packages --local it returned with: `Unhandled error: open...

floitsch 09/12/2024 03:41 PM

Hmm. I will have a look at this. Looks like we are missing a check.
Thanks for reporting.

OPkaxori

Is it possible to use a 'global package' inside the example folder of a 'local package' ? ``` import pixel_strip show * // github.com/toitware/toit-pixel-strip import rgb-strip s...(edited)

floitsch 09/12/2024 03:43 PM

Each package.yaml/lock has its own entries.
If your example needs a package it needs to install it too.

floitsch 09/12/2024 03:44 PM

So basically you just have to jag pkg install pixel_strip inside the examples folder.

kaxori 09/12/2024 03:59 PM

ok, I thought the pixel_strip is a global package, installed in <Project-Base>/.packages, and importable from all modules.

floitsch 09/12/2024 04:00 PM

The moment you have a package.yaml/lock you are a separate project.

floitsch 09/12/2024 04:01 PM

Tests and examples often have different requirements than the actual package one is working with.

floitsch 09/12/2024 04:01 PM

For exampel, the pixel_strip might be needed by the example, but not by every user of the package.

floitsch 09/12/2024 04:01 PM

-> it's only in the examples.

floitsch 09/12/2024 04:02 PM

Some other languages have separate sections (like "dev_dependencies" or so).

kaxori 09/12/2024 04:02 PM

So but if I use this pixel_strip functionality inside own 'rgb-strip' package ?

floitsch 09/12/2024 04:02 PM

Then you need to duplicate the work.

floitsch 09/12/2024 04:02 PM

Alternatively, your package can export some of the classes if it really makes sense (quite rare).

OPkaxori

So but if I use this pixel_strip functionality inside own 'rgb-strip' package ?

floitsch 09/12/2024 04:03 PM

Note that your users wouldn't benefit from the nesting either, so they would need to install the pixel_strip package themselves too.

floitsch 09/12/2024 04:03 PM

What we could do, though, is have something like "suggested packages".

floitsch 09/12/2024 04:04 PM

So if you install rgb-strip it could suggest to install pixel_strip too.(edited)

kaxori 09/12/2024 04:12 PM

My intention with creating rgb-strip was to encapsulate PixelStrip, ByteArray, Colors, ... and to reuse that package in different applications. I thought if pixel_strip is globally installed in `.packages' and 'rgb-strip' is installed in 'local-packages'.
And the jag apps have only to import a single rgb-strip definition.(edited)

floitsch 09/12/2024 04:37 PM

Toit doesn't support that. It would make it hard to control which package gets which prefix...
You shouldn't really think of "local" and external packages as different. They are pretty much the same, except that one is referring to code on your disk, while the other is downloaded.

floitsch 09/12/2024 04:40 PM

What you can do is create a package/library that reexports code from multiple packages.
Like:

import pixel-strip show *
import something-else show *
export *

If you then import that library (in a package or not) you get all the types from it.
However, it also means that you lose the prefix. So all of the identifiers would be mixed together.

You could avoid that by having different files:

// pixel-strip.toit
import pixel-strip show *
export *

// something-else.toit
import something-else show *
export *

You could then import merged.pixel-strip or import merged.something-else.

floitsch 09/12/2024 04:40 PM

That's probably the closest to what you want.

kaxori 09/12/2024 04:42 PM

import .color show *
import .bytearray show *

import pixel_strip show *
import gpio

export *

/**
Implements a universal (application independent) Neopixel led strip interface.
*/
class Rgb-Strip:
...

thats what I tried, but i always get an error after >>> import pixel_strip show *

floitsch 09/12/2024 04:43 PM

Where is this file located?

kaxori 09/12/2024 04:43 PM

in 'src'-folder

floitsch 09/12/2024 04:43 PM

What's the package.yaml for that file?

floitsch 09/12/2024 04:44 PM

What's the error message?

kaxori 09/12/2024 04:45 PM

t:\local-packages\rgb-strip\package.yaml

kaxori 09/12/2024 04:45 PM

PS T:\local-packages\rgb-strip\examples> jag run .\app.toit --device Radar
Scanning for  device with name: 'Radar'
Running '.\app.toit' on 'Radar' ...
<pkg:..>\rgb-strip.toit:4:8: error: Package for prefix 'pixel-strip' not found
import pixel_strip show *
       ^~~~~~~~~~~
Compilation failed
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x15b1a3b]

goroutine 1 [running]:
main.main()
        D:/a/jaguar/jaguar/cmd/jag/main.go:38 +0x1db
PS T:\local-packages\rgb-strip\examples>

(edited)

floitsch 09/12/2024 04:46 PM

Can you show me the package.lock file of the example folder?(edited)

kaxori 09/12/2024 04:47 PM

floitsch 09/12/2024 04:47 PM

Try running jag pkg install in your example folder

floitsch 09/12/2024 04:48 PM

It looks like you changed the dependencies of the package but didn't rerun install. So it doesn't have all dependencies in the example folder yet.

kaxori 09/12/2024 04:48 PM

PS T:\local-packages\rgb-strip\examples> jag pkg install
Error: Failed to parse package description 'T:\local-packages\rgb-strip\package.yaml': %!w(*yaml.TypeError=&{[line 5: cannot unmarshal !!map into []tpkg.descPackage]})
PS T:\local-packages\rgb-strip\examples>

floitsch 09/12/2024 04:49 PM

Ok. What if you run pkg install in the rgb-strip folder?

floitsch 09/12/2024 04:50 PM

The error indicates that the package file is corrupted.
I don't really see what's wrong, though

floitsch 09/12/2024 04:50 PM

(not on my computer to test it)

kaxori 09/12/2024 04:51 PM

PS T:\local-packages\rgb-strip> jag pkg install
Error: Failed to parse package description 'T:\local-packages\rgb-strip\package.yaml': %!w(*yaml.TypeError=&{[line 5: cannot unmarshal !!map into []tpkg.descPackage]})

floitsch 09/12/2024 04:51 PM

Ok. So it really isn't happy with that file

floitsch 09/12/2024 04:51 PM

Did you edit it by hand?

kaxori 09/12/2024 04:51 PM

yes

floitsch 09/12/2024 04:52 PM

It looks like one of these should be a list and is a map instead.

kaxori 09/12/2024 04:53 PM

how can I generate a fresh one

floitsch 09/12/2024 04:53 PM

Hmm. Isn't that the syntax of the lock file?

floitsch 09/12/2024 04:54 PM

Remove package.* And then jag pkg init

floitsch 09/12/2024 04:54 PM

Then jag pkg install toit-pixel-strip

kaxori 09/12/2024 04:56 PM

PS T:\local-packages\rgb-strip> jag pkg init
PS T:\local-packages\rgb-strip> jag pkg install toit-pixel-strip
Error: Description at 'T:\local-packages\rgb-strip\package.yaml' is missing a name

floitsch 09/12/2024 04:59 PM

Right. Just add a name and description to your package file

floitsch 09/12/2024 05:01 PM

That said: I'm surprised you need that just for pkg install.

floitsch 09/12/2024 05:08 PM

I need to leave now, but I will read from time to time and try to answer.
If you don't manage to get it to work, feel free to send me whatever you have and I will fix it.(edited)

kaxori 09/12/2024 09:35 PM

Danke, sehr gerne:

kaxori 09/12/2024 09:36 PM

File from path T:/local-packages

floitsch 09/12/2024 09:36 PM

I will have a look tomorrow. Please remind me if I forget.

kaxori 09/12/2024 09:37 PM

floitsch 09/13/2024 08:32 AM

It was almost working.
I just did, in the example folder:

jag pkg init
jag pkg install --local ..

And then removed the pixel_strip line:

// import pixel_strip show * // github.com/toitware/toit-pixel-strip

With that everything worked.

floitsch 09/13/2024 08:33 AM

I can send back the zip on request.

kaxori 09/13/2024 08:44 AM

jag pkg init
==> creates empty package.yaml/.lock files
jag pkg install --local ..
==> Error: Description at 'T:\local-packages\rgb-strip\examples\package.yaml' is missing a name

PS T:\local-packages\rgb-strip\examples> jag pkg init
PS T:\local-packages\rgb-strip\examples> jag pkg install --local ..
Error: Description at 'T:\local-packages\rgb-strip\examples\package.yaml' is missing a name

floitsch 09/13/2024 08:57 AM

which version of jaguar do you use?

kaxori 09/13/2024 08:58 AM

PS T:\local-packages\rgb-strip\examples> jag version
Version: v1.39.0
SDK version: v2.0.0-alpha.159
Build date: 2024-06-24T08:44:15Z

floitsch 09/13/2024 08:58 AM

Could you try updating to the latest version?

kaxori 09/13/2024 09:03 AM

nothing changed

PS T:\local-packages\rgb-strip\examples> jag pkg init
PS T:\local-packages\rgb-strip\examples> jag pkg install --local ..
Error: Description at 'T:\local-packages\rgb-strip\examples\package.yaml' is missing a name
PS T:\local-packages\rgb-strip\examples> jag version
Version:         v1.41.0
SDK version:     v2.0.0-alpha.160
Build date:      2024-09-01T11:40:26Z

floitsch 09/13/2024 09:04 AM

I don't see this. Maybe we have a bug with windows

floitsch 09/13/2024 09:05 AM

Let me send you the package files so you are unstuck.
I'm in the middle of another debugging session, but I will investigate this.

floitsch 09/13/2024 09:05 AM

floitsch 09/13/2024 09:07 AM

I looked up the error.
My current understanding is that it should only happen when the package-manager tries to parse a registry.

floitsch 09/13/2024 09:08 AM

Could you do a jag pkg registry list ?

floitsch 09/13/2024 09:08 AM

If you added the example folder as registry folder, it might explain the error.

kaxori 09/13/2024 09:08 AM

PS T:\local-packages\rgb-strip\examples> jag pkg registry list
toit: github.com/toitware/registry (git)
local: T:\local-packages (local)

floitsch 09/13/2024 09:08 AM

right. That explains it.

floitsch 09/13/2024 09:09 AM

I think I understand more what you are trying to do.

floitsch 09/13/2024 09:10 AM

A local registry allows you to have packages that aren't visible on pkg.toit.io.

floitsch 09/13/2024 09:10 AM

This makes it possible to write jag pkg install xxx where xxx is not on pkg.toit.io but only in your local registry.

floitsch 09/13/2024 09:11 AM

I have started a process of rewriting our package manager and would like to remove local registries (and only support git repositories as registries), but for now it's supported and works.

floitsch 09/13/2024 09:12 AM

A registry (local or not) is fundamentally just a bunch of description of available packages.

floitsch 09/13/2024 09:12 AM

That is, in doesn't contain any code.

floitsch 09/13/2024 09:12 AM

So if you want a local registry, you should point it to a "fresh" directory.

floitsch 09/13/2024 09:12 AM

And then add descriptions to it.

floitsch 09/13/2024 09:13 AM

Descriptions are generated by jag pkg describe(edited)

floitsch 09/13/2024 09:15 AM

By default they are geared for "global" access. That is, they want a url, version and hash.
Let me check, how to make "local" package descriptions. (I'm pretty sure we use them for tests).

floitsch 09/13/2024 09:16 AM

Nope. Looks like we only have git descriptions.(edited)

kaxori 09/13/2024 09:17 AM

AHA: my idea ...
- In the first step, I created the application containing the rgp-strip as sub folder. (it works)
- then I moved the rgp-strip subfolder to a local-package and the app code into examples (to test the package)
- possibly later, the local package could be pulished into the toit-package environment(edited)

floitsch 09/13/2024 09:17 AM

For now you have to just jag pkg install --local ...

floitsch 09/13/2024 09:18 AM

I would like to add functionality so that you can tell jag pkg that it should add a certain local folder as URL@version for resolution.

floitsch 09/13/2024 09:18 AM

This would make it much easier to test local packages (even if they are used transitively by other packages).

floitsch 09/13/2024 09:19 AM

Unfortunately, that's not implemented yet.

floitsch 09/13/2024 09:19 AM

As I said: I started rewriting the package manager (in Toit), and new features will probably have to wait until that is done.

floitsch 09/13/2024 09:20 AM

In any case: you should remove the local registry again. Any yaml file in the local-packages folder will trip up the package manager.

kaxori 09/13/2024 09:20 AM

Thank you for the support,
I wil try ...

kaxori 09/13/2024 11:07 AM

- app in examples folder runs
- now trying to install the local package

PS T:\> jag pkg install --local t:\rgb-strip
Info: Package 't:\rgb-strip' installed with name 'rgb-strip'
PS T:\> jag pkg list rgb-strip
Error: Failed to parse package description 'rgb-strip\examples\package.yaml': %!w(*yaml.TypeError=&{[line 2: cannot unmarshal !!map into []tpkg.descPackage]})

kaxori 09/13/2024 11:08 AM

I don't understand why the yaml in examples is taken ?

kaxori 09/13/2024 11:14 AM

if I rename yaml in examples to 'package.yaml_'
and complete package.yaml in T:/pkg-strip to

name: rgb-strip
description: Implements a universal (application independent) Neopixel led strip interface.
license: proprietary
version: v1.0.0
url: T:/rgb-strip

PS T:> jag pkg list rgb-strip
rgb-strip:
rgb-strip - 1.0.0

floitsch 09/13/2024 11:15 AM

Did you remove the local registry?

kaxori 09/13/2024 11:15 AM

PS T:> jag pkg registry list
toit: github.com/toitware/registry (git)
yes

floitsch 09/13/2024 11:16 AM

hmm. Where does jag pkg list find the rgb-strip package from?

kaxori 09/13/2024 11:16 AM

jag pkg install --local t:\rgb-strip

floitsch 09/13/2024 11:17 AM

jag pkg list

lists all packages of a registry.

floitsch 09/13/2024 11:17 AM

The argument must either be the name of a registry or a folder.

floitsch 09/13/2024 11:17 AM

Wait.

floitsch 09/13/2024 11:18 AM

hmm. No. Had a hunch that doesn't apply..(edited)

kaxori 09/13/2024 11:18 AM

floitsch 09/13/2024 11:19 AM

Right. That would be all packages that are in the default pkg registry.

floitsch 09/13/2024 11:20 AM

When you did jag pkg list rgb-strip in which directory were you?(edited)

floitsch 09/13/2024 11:20 AM

was there a folder rgb-strip there?

kaxori 09/13/2024 11:20 AM

I generated the list in project root 'T:/'

floitsch 09/13/2024 11:20 AM

I just reread the failed command.

floitsch 09/13/2024 11:20 AM

Makes complete sense.

floitsch 09/13/2024 11:21 AM

jag pkg list rgb-strip
Error: Failed to parse package description 'rgb-strip\examples\package.yaml': %!w(*yaml.TypeError=&{[line 2: cannot unmarshal !!map into []tpkg.descPackage]})

floitsch 09/13/2024 11:21 AM

You are asking jag pkg list to list all packages that are in the registry folder 'rgb-strip'.

floitsch 09/13/2024 11:21 AM

However that folder isn't a registry.

floitsch 09/13/2024 11:21 AM

It's your package.

floitsch 09/13/2024 11:22 AM

(for a registry all yaml files are treated like package descriptions).

floitsch 09/13/2024 11:22 AM

So the question now: what did you intend to do with jag pkg list rgb-strip?

floitsch 09/13/2024 11:23 AM

And maybe the same question for: jag pkg install --local t:\rgb-strip

kaxori 09/13/2024 11:25 AM

I wanted to check if the installation of the local package 'rgb-strip' worked.
In the next step i want to use that package in a app import rgb-strip show *

floitsch 09/13/2024 11:25 AM

The easiest way to check is to have a look at your package.lock file.

floitsch 09/13/2024 11:25 AM

But since there wasn't any error you should be good.

kaxori 09/13/2024 11:26 AM

floitsch 09/13/2024 11:26 AM

exactly.
Your current project now has a package rgb-strip at the given path.

kaxori 09/13/2024 11:29 AM

So for a new app referencing to pkg rgb-strip , I have to create a yaml file with dependencies ?

floitsch 09/13/2024 11:29 AM

You have to jag pkg init and jag pkg install --local t:\rgb-strip
That should do it.

floitsch 09/13/2024 11:30 AM

That will download all of rgb-strip's dependencies and then give you access to them through rgb-strip (since you do an export from it).

kaxori 09/13/2024 11:31 AM

Installing container 'APP' from 'app.toit' on 'Radar' ...
T:\Test-rgb-strip\Package-App\package.lock:5:11: error: Package '/T:/rgb-strip' not found at 'T:\Test-rgb-strip\Package-App\T:\rgb-strip'
path: /T:/rgb-strip

floitsch 09/13/2024 11:37 AM

argh.

floitsch 09/13/2024 11:37 AM

That looks like a Windows bug.

floitsch 09/13/2024 11:38 AM

I will look into this.
In the meantime you can work around it by installing the package with a relative path.

kaxori 09/13/2024 11:39 AM

ok solved!
packages:
rgb-strip:
path: T:/rgb-strip

kaxori 09/13/2024 11:40 AM

The preceeding / was too much for the path

floitsch 09/13/2024 11:41 AM

So you changed the package.lock file by hand?

kaxori 09/13/2024 11:42 AM

yes
For me it took a long way with a lot fumbling, and i have to repeat this again ...

floitsch 09/13/2024 11:42 AM

It's definitely a bug in the package manager.

floitsch 09/13/2024 11:42 AM

I have it already on my TODO list.

kaxori 09/13/2024 11:43 AM

Nevertheless Thank you for the support

floitsch 09/13/2024 11:44 AM

Very welcome.
Sorry you are encountering so many issues...

kaxori 09/13/2024 11:44 AM

the life of a scout

kaxori 09/13/2024 04:04 PM

I did it again,
it works fine using relative paths

135 messages in total