Follow

more progress towards packaging, because i'm tired of unsafe uninstall rules in mkfiles
git.sr.ht/~raingloom/package

· brutaldon · 3 · 1 · 0

doesn't work as intended just yet, but you can see how it's put together
look at the package.tar rule in the mkfile for a usage example

it really is dirt simple:
auth/none: build as a user who has no access rights
ramfs: create a /tmp we can access
divergefs /: analogous to Linux's overlayfs, catches all modifications in /tmp/files
create an archive of /tmp/files and print it to stdout
bam, packaged.

caveat emptor: /env and all temporary build files are packaged too. working on a fix.

gonna switch it to use nar once i have a way of opening nar files, or i could try to create the tar file deterministically with some hacks? but ye, once it's working i'll try to write some build scripts for /sys/src/cmd.

this is _not_ going to handle dependencies, that's going into the Nix port.

oh, and i need a way to dynamically enable and disable packages.
bind won't be enough.
divergefs might work but a custom file system might perform better. it might also be a good idea to support compressing package archives, since they won't be modified anyways.

@grainloom

Uhm... maybe I'm missing the overall vision.

Which Nix are you referring to?
This one? quanstro.net/plan9/nixterm/ter
Or maybe this one? nixos.org/

Also, why you say that `bind` is not enough?

@Shamar Nix the package manager.
bind doesn't do proper overlays by itself, it's the whole reason divergefs exists.
multiple binds would become ugly very quickly.

@grainloom

Actually I didn't know about divergefs... did you write it yourself?

@Shamar nah, it's quite old. I stumbled upon it while looking for an overlayfs implementation. which it is.

@grainloom

This one? www.cs.bell-labs.com/wiki/plan9/divergefs/

I can't find the sources... where did you get it?

@Shamar
original author: cs.cmu.edu/~412/history/2004S/

found it here: 9p.io/wiki/plan9/Contrib_index

divergefs-fix.tgz is the one that works

the way i actually found the sources was by grepping through /n/9pio/contrib

@grainloom

Thanks a lot!

I was considering to write something like that for #Jehanne's package manager.

But honestly I also like the obvious alternative: add to each package a importIntoNS.rc script that take care of

- importIntoNS dependencies
- recursively bind every clashing folder

The package manager would invoke the importIntoNS.rc of the packages you want to use (potentially using different versions in different rio windows/namespaces).

In the example at cs.cmu.edu/~412/history/2004S/ the `b` packages would not bind b/ over the mount rock that already contains aa/ from the a/ package, but bind b/aa/ over mountrock/aa/.

What's your opinion on such design?

@Shamar I'm not sure what you mean by that example, or by "mount rock"?

@grainloom

"mount rock" is simply the name used in the kernel for the Chan* that correspond to the "old" in bind(2) man.cat-v.org/9front/2/bind

I suppose it's called "rock" because it's the fixed point on top of which you "build" your namespace.

Suppose you have two packages A and B each containing a d directory, that contains different files.

You might have the packages extracted in directories like this:

A/
A/d/
A/d/file1.rc

B/
B/d/
B/d/file2.rc

if you overlay B over A, you'll have into the mount point M

M/
M/d/
M/d/file2.rc
M/d/file1.rc

But you could get the same effect by binding A before M and then B/d/ before M/d/

This can results into ugly tree but looks like the simplest solution (to some extent).

@grainloom

uhm... in the general case, it probably is pretty ugly.

But ideally, each package would contain few folders that require merge to be run.

In the perfect world, only $pkg/bin would need to be merged (#Plan9 and #Jehanne only support static linking).

$pkg/lib would only be required to be merged if you need to _build_ software based on such library.

If we consider scripting languages things becomes more convoluted (as @ekaitz_zarraga noticed with #Chibi) but still, the shape of a package and it's impact on the namespace should be based on few conventional directories.

My approach becomes ugly only if each package violate such conventions.

Or maybe I'm missing something?

@Shamar @ekaitz_zarraga not sure how it's implemented in the kernel, but i'm pretty sure file lookup is O(n) in the number of binds/mounts. the kernel doesn't know that the packages are immutable, i wanna create a file system that builds a lookup table of the packages.

@grainloom

Yes, for each bind/mount on a certain "mount rock", the kernel sends at least one Twalk to the bound server/device.
So, on average, it should be roughly O(n) in the number of bind/mounts.

But since the path is hierarchical and you could have to traverse several mount point before reaching the rock, I'd say the actual time complexity can be much bigger as the file's path grows.

This shouldn't be an issue for conventional mount points such as /bin (/cmd in #Jehanne) or /lib, thus O(n) should be an appropriate estimation (as long as each bound server/device doesn't do funny things).

Anyway, your project sounds cool but I can't get why immutable packages is relevant in this context.
How are you going to exploit this condition?

@ekaitz_zarraga

@Shamar @ekaitz_zarraga
Lookup tables. That's it really. It shouldn't be too hard to implement and it would make the namespace cleaner.

@Shamar @ekaitz_zarraga Yup. Kinda like what Guix does with profiles.
This also has the side effect of catching conflicting packages.

@Shamar @ekaitz_zarraga Like, all it does it store which file is in which package directory. Put all that in a hash map or something and lookup becomes O(log(n)).

@grainloom

Sounds very interesting, but consider you are trading read() time for lookup time.

With your fs, file lookup would be faster than in kernel as the kernel would have to try every bound package until it finds the desired program.

But then, each read will need to pass through your file server.

I don't think it would be a large overhead, but it would be linear to the number of reads.

It should also be noted that the #Plan9 kernel caches (for a while) .text pages from binaries, so this read-time overhead would probably be almost invariant on successive executions of a binary.

BUT scripts are not cached in kernel (they are text, after all), so such read-time overhead would be larger.

Also Plan9 memory page is 4096 bytes, so on a large binary the number of reads might be high.

@ekaitz_zarraga

Show more

@grainloom I saw your name of sr.ht, and thought to myself “Did I misread their name all this time?”. I look up, and no. I didn't.

That begs the question, which is your official name?

@grainloom interesting! this is intended to replace the ports framework stuff?

@qbit yup! ports (at least the one i use from 9front) is not very reliable, so i'm making my own

Sign in to participate in the conversation
Cybrespace

Cybrespace is an instance of Mastodon, a social network based on open web protocols and free, open-source software. It is decentralized like e-mail.