Page MenuHomePhabricator

Integrating Guix/Nix Package Manager
Open, NormalPublic

Description

Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. Also makes packaging easier than Debian. It also supports compiler hardening flags.

https://nixos.org/nix/

GNU Guix is based on it with some modifications.

https://nixos.org/nix/


Pros:

  • Has the potential to solve our problem of including external binaries in a safe reproducible way without the Debian freeze restrictions.
  • Does not interfere with apt or the system config in any way.

Cons:

  • Usability impact of running two package managers to keep the system updated. Wrapper scripts or some other way to notify users may be needed.

Found the dev thread about GNU Guix's Debian packaging attempts mentioned in:

https://unix.stackexchange.com/questions/290423/can-guix-packages-be-delivered-to-other-distros

There was a GSoC 2016 project to get it into Debian but it didn't happen for many reasons.

There are good reasons why Guix and Nix do not follow the FHS but that means there will never be a package for them upstream. Attempts to conform to FHS caused Guix functionality to break and would mess up the nice clean install system it has:

https://lists.gnu.org/archive/html/guix-devel/2016-02/msg01126.html

Instead the guix project distributes a signed tarball with instructions to compile and install it which is far from optimal for us:

https://www.gnu.org/software/guix/manual/guix.html#Binary-Installation


Instead of trying to get into Debian, the devs talked about the possibility of providing a .deb via their own repository that does not follow Debian standards but is very convenient for people who want to bootstrap install Guix:

https://lists.gnu.org/archive/html/guix-devel/2016-02/msg01144.html

"Hosting a .deb file on our own server that users could download and
install with dpkg would be perfect for us."

The blocker for this is Ubuntu's build process requirements are different from Debian's and so they shelved the whole thing:

https://lists.gnu.org/archive/html/guix-devel/2016-02/msg01205.html

"Without a fully automated process to build .deb for several distros, I
don’t think we can offer to distribute .deb ourselves. :-/"


Here a Guix enthusiast discusses how to run Guix without compiling it. interesting but doesn't make it any easier in our case to distribute:

https://dustycloud.org/blog/guix-package-manager-without-make-install/


With this path blocked I decided to look at the Nix manager instead which is what Guix is based on anyway. And fortunately its a better situation. Signed Debian packages for stable are provided! and a script that fetches and validates the .deb. That means with some leg work it can be included in Whonix via our repos:

https://nixos.org/wiki/Installing_Nix_on_Debian

I looked at their package selection and it is impressive. Grsecurity, GNUnet and much more:

https://nixos.org/nixos/packages.html


Other interesting reads:

https://nixos.org/wiki/Hardened_NixOS#Features_in_development

https://www.reddit.com/r/NixOS/comments/4btjnf/fully_setting_up_a_custom_private_nix_repository/


Details

Impact
Normal

Event Timeline

HulaHoop created this task.Jan 12 2017, 3:44 AM
HulaHoop closed this task as Invalid.Jan 18 2017, 5:15 PM
HulaHoop claimed this task.

The main and only purpose of getting Nix into Whonix was GNUnet. Now that Nix stopped making new debs, there is no point of considering this task further.

Closing.

HulaHoop reopened this task as Open.Jan 20 2017, 5:25 PM
HulaHoop renamed this task from Integrating Nix Package Manager to Integrating Guix/Nix Package Manager.
HulaHoop updated the task description. (Show Details)

GNUnet guys are looking into having the Guix devs provide debs.

ng0 added a subscriber: ng0.EditedJun 25 2017, 9:12 PM

I want to clarify something:
the GNUnet "guys" in this case is just myself at the moment.
Torbrowser is not considered my priority, but it will definitely happen at some point within the next years.
That's a revised priority list, as I'm currently in the process of writing infotropique OS which is currently a one person job on the development end.
I'm not getting full time paid for this, so this is worked on whenever I have the time, hopefully at least part time.
More details can be found later this year in an published handbook.
I try to keep the forks with Guix as little as possible but they are not avoidable. One of the many individual ideas is use of a different libc (musl).
Torproject software is not the priority of infotropique OS, but I'm not throwing it out, so torbrowser could still happen in either Guix or infotropique.

I'd happily give this a try some day (started working on upstream firefox aswell), but:
The bottomline here is, I left enough "bread crumbs" of my plan for torbrowser, so if you want torbrowser through Guix and not Nix you can get this done in Guix. Don't rely on me getting this done between writing an operating system as long as I am alone on the developer side.

*e: Just to be sure and to refresh my memory from the email exchanges we had a couple of hundreds of emails ago: this is just about torbrowser, or was there more to it?

Understood and thanks for chiming in :)

*e: Just to be sure and to refresh my memory from the email exchanges we had a couple of hundreds of emails ago: this is just about torbrowser, or was there more to it?

That was the main thing I believe. Its true I was exploring a grsec patched libre-linux kernel via Guix too but with recent developments its an obsolete idea (grsec gone private and KSPP mainlining everything).

ng0 added a comment.Jul 3 2017, 10:31 AM

"We" and "ours" in the text below refers to infotropique, not Guix.

I am currently preparing firefox-nightly and thunderbird to use them as a base for a free / non-free choice. I believe that this can serve as a better base for writing the torbrowser package than icecat.
This will go into one of the repositories which are used to build our systems, so you should be able to use it for your purposes.
At some point there'll be binary substitutes for this, probably musl based in the future (or dual choice, musl or glibc) but for now glibc. We will run our own build infrastructure but it is up for discussion *if* torbrowser will be in these source or if it will be an inofficial distributed build and/or should end up in Guix.

But as pointed out, this can take some time and I will soon need to adjust my work schedule to university.

So, in other words: if you want it now or very soon, start digging into Guix packaging yourselves and please inform me if you do take on this task because I hate duplicate efforts and like to cooperate.

We are preparing for a major point release so nothing on this front is planned from our side in the short to medium term. We are on the lookout for anything coming from your side whenever you feel like it.

ng0 added a comment.Oct 30 2017, 8:52 PM

A short update on torbrowser in Guix:

Torproject trademarks team as well as Torproject torbrowser team made it very clear that my proposed configuration and changes will be considered acceptable for them.
Furthermore they will adjust some last parts that read "Firefox" instead of "Torbrowser" which makes it usable for us in Guix.
Inclusion is welcome from Guix side aswell.

However I'm working on multiple, very complex projects at the same time, and web browsers are no walk in the park (even the not so complex Pale Moon was annoying enough and I ended up forking its New Moon variant for license reasons). It is autumn of 2017 already. What I have done so far for TB on Guix is looking at the build process and how it can be replicated. I've spent some time with Thunderbird aswell (comparable build system), because that's always a requirement for people and institutions I know to judge a system by (I'll never have real sympathy with this, but all email clients suck anyway).

Generic news on Guix, it seems like we might come up with a way to generate .deb etc files for Guix itself on our own, via code, but this was just mentioned in discussions in IRC on multiple occasions, I'm not sure if anyone is actively working on this.

iry added a subscriber: iry.Feb 20 2018, 9:17 PM
ng0 added a comment.EditedMar 8 2018, 1:00 PM

FYI, I started working on the tor browser package for Guix, building from source, unlike the Nix package. This is in a similar style to poncho's torbrowser-overlay for Gentoo.

If all goes well, I should be able to finish debugging it next month. Worst case I assume I'll have a working torbrowser sometime this summer. You never know with webbrowsers...

We still got no news on the .deb generation for Guix itself.

ng0 added a comment.Mar 16 2018, 11:57 AM

Back on topic. [Disclaimer: I'm just a contributor to Guix, I am not one of the maintainers.]
A recent chat with lynX made me think about an alternative approach you maybe haven't considered yet:

Right now both Guix and Nix are in this position where they break with the FHS and other standards for very good reasons. Diverging from the FHS is one of the core functionalities of our systems. If you try to re-integrate Guix into FHS structure (for example moving /gnu/store to /opt/gnu/store) you won't get binary substitutes from the official build farm, one of the benefits (otherwise you could just run your own with the same path). It's our intention to diverge from FHS and not just an error. Debian should support this as an exception from the default expectations they have about packages.
Why don't you call for a Debian community vote to make an exception for Guix?
I don't know about Nix and I can't speak for them, technically it should be the same, but there might be licensing issues for Debian in the software that can be accessed through Nix.

There are other systems which already recognize this exception and put them into their correct places (iirc at least Archlinux and Gentoo).
We have these standards like FHS so that people don't start reinventing the wheel, but Guix and Nix are new and legitimate reasons for new /dirs. In Guix we care about cross distro support. It's not just GuixSD first, rest later. So if a system distribution includes guix as a package, they should include it in a way which provides its users with the full functionality of it. Reintegration breaks Guix.

I understand that you are a Downstream Distro of Debian, so https://www.debian.org/vote/ should be the place to address if you decide to go this way.

Awesome progress thanks for the updates :) Right, the advantage of Guix is precisely that it doesn't follow FHS. Its worth trying to ask for an exception from upstream no matter how slim the chances - at least there's more of a chance than never asking at all.

Do you know if Gentoo or Arch document their technical implementation of Guix? I would also like your help in writing a proposal outlining the exact details of an exception on Debian that would work for you and Nix potentially. I will focus on Guix because they are Free Software focused but its still good to keep the door open for those who make different choices.

ng0 added a comment.EditedMar 19 2018, 10:56 PM

@HulaHoop I think at this point we should start a discussion on guix-devel and/or whonix-devel mailinglist for this. If you read https://gnunet.org/bot/log/guix/ of 2018-03-19 around ~21:40 - 22:00 UTC (note: our log bot is not logging this at the moment it is written, so it'll appear later online) you will find a braindump one of us did + a discussion on the subject of Debian+Guix. I can not speak for the project in the position of a maintainer, this is not my position to take.

Log starts here: https://gnunet.org/bot/log/guix/2018-03-19#T1655382

A switch to /opt/ seems like a great compromise that can shut up the FHS zealots. Is this something the Guix guys can do or is it for the adopting distro to handle?

ng0 added a comment.Apr 29 2018, 2:35 PM

Basically you can configure guix with your own arguments. prepending /opt/ by default would make the path in some cases too long. You can have a custome store path, but this implies that no substitutes from official, central, buildfarm will be usable (however you can run your own substitutes server). You could mount /gnu/store to /opt/guix or some other location. If you would have read the chat content (which I assume you didn't), you would see some insight into the problems and what possible solutions there are.

HulaHoop added a comment.EditedMay 18 2018, 3:17 PM

If you would have read the chat content (which I assume you didn't), you would see some insight into the problems and what possible solutions there are.

I actually did. Some of it may be over my head by I'm getting there:) I think if Debian wants to customize Guix to conform to their "standard" they would probably be OK with running their own substitute mirror. Would you see this as a good thing or something that introduces fragmentation?

Sorry if this is taking longer than it should. I want to nail down the fine details and find the best solution before writing a proposal. This way I won't have to edit it and lose mind-share in the process.

ng0 added a comment.Jul 19 2018, 5:12 PM

Acknowledged and I'm lagging behind. You can expect an answer somewhere between mid August and beginning of October.

Some parts in guix pull changed (substitutes for guix itself offered) and bootstrappable is making progress.

ng0 added a comment.EditedJul 19 2018, 5:19 PM

Just this bit already:

substitute mirrors are okay. Debian would probably even have all the resources to do this.

I myself am working on extending the ideas of guix with a friendly (= colaborative) project based on guix with a different default "store" path (instead of /gnu/store). The moment Debian changes the store prefix they will have to offer substitutes through servers.

there's a difference between (caching) mirrors and builders/buildfarm. it depends on what you have to do / how you setup guix.

HulaHoop added a comment.EditedJul 22 2018, 6:23 PM

@ng0 I wrote a proposal draft. Feel free to improve it before I post:

Hello Dear Debian Coders and Maintainers,

I will propose a process for integrating GNU Guix into Debian and working around potential roadblocks that have made the process difficult in the past.

For those unfamiliar, Guix is a package manager that supports reproducible builds, transactional upgrades and roll-backs, unprivileged package management, per-user profiles and more (please see: https://www.gnu.org/software/guix/manual/html_node/Features.html).

It uses Guile scheme to define package structure and system configurations. Its unique self-contained package installation allows it to be used seamlessly on any Linux distro while making sure no conflicts between it and the distro's default package manager arises. As an official GNU project it supports nothing but libre software so it should be compatible with DFSG. Also having Guix in Debian will allow access to more recent versions on software that is developed too rapidly to be otherwise practically usable from the official repos (for example GNUnet).


Potential problems and the way forward:

The fact that the default repo path is /gnu/store conflicts with Debian's strict FHS guidelines. A solution is to mount or patch /gnu/store to use /opt/guix. However this would break use of the official binary buildfarms and would require Debian to host its own independent instance that works with our customized paths.

There are some missing build/runtime dependencies such as guile-git and guile-ssh that would need packaging first but its not possible without more widespread interest:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644

A good starting point for a Guix deb package is this latest effort:
https://github.com/detrout/debian-guix

If you have any further suggestions or requests please post them and the Guix people would be interested to collaborate on whatever it takes to realize this.

I hope to kick-start a dialogue/interest in this project so please share your thoughts. Thanks.

HulaHoop removed HulaHoop as the assignee of this task.Aug 16 2018, 5:16 PM