Page MenuHomePhabricator

qubes-whonix update issue because of bug in whonix-setup-wizard / whonix_repository
Closed, ResolvedPublic

Description

T232 affects qubes-whonix.

Due due qubes-whonix 9.6 using packages whonix-repository and whonix-setup-wizard that are affected by T232, users will run into issues when attempting to upgrade from 9.6 to higher versions.

qubes-whonix users are using the the literal stable code name in /etc/apt/sources.list.d/whonix.list, which this repository is no longer in use. It was last updated for Whonix 8,

To let users qubes-whonix 9.6 users do binary upgrade, they also have to be told to use at least once sudo whonix_repository --codename wheezy beforehand or we need to get the repository with the code name stable up again. The latter would break systems of remaining users of Whonix 8 that are still having the stable repository enabled (because 8 -> 9 upgrades were kinda "bumpy"). But since Whonix 8 is no longer supported for a long time now anyhow, that would be acceptable.

I somehow knew it wasn't ideal to derivate from Whonix default package selection and to sneak in features into the stable that are purposed for the development version.

Details

Impact
High

Event Timeline

Patrick created this task.Mar 19 2015, 12:24 PM
Patrick raised the priority of this task from to Normal.
Patrick updated the task description. (Show Details)
Patrick added subscribers: troubadour, HulaHoop, nrgaway and 2 others.

I can confirm that this is happening. Where whonix-repository is using wheezy repo and whonix-setup-wizard is using literal stable repo.

I do suggest running sudo whonix_repository on the TemplateVMs, which will then use the wheezy repo, as part of the Binary Update Guide: https://www.whonix.org/wiki/Qubes/Binary_Update

Looks like this is where the whonix-setup-wizard package is being installed from independently of Whonix in the QubesBuilder build process: https://github.com/marmarek/qubes-builder/blob/master/example-configs/templates.conf

Marek mentioned to me that he will soon be building a new Whonix template, due to another problem. Maybe we could suggest fixing this too at the same time by using a different version of whonix-setup-wizard than 0.5-1 which would consistently use the wheezy repo instead?

I will test latest whonix-setup-wizard version tonight and if it works we can just get a signed tag and I will switch the qubes-builder scripts to use new version.

Sounds good. Thanks @nrgaway. Does this sound okay for making updated whonix-setup-wizard tag @Patrick?

You also need updated whonix-repository tool (cli version, which is the backend).

Okay, thanks. Its building now. Think I just got that component built just in time for it to be installed ;)

Ah, please tell me, if you need, I can bump changelog version and create a signed git tag for either one or both.

In T233#3200, @Patrick wrote:

Ah, please tell me, if you need, I can bump changelog version and create a signed git tag for either one or both.

The builds started about 15 minutes ago and will most likely take about another hour or so if there are no issues (I have one building and one in queue). I will let you know my test results as soon as possible.

I just finished testing with whonix-setup-wizard and only found a few issues.

The first issue is that argparse is is not allowing any of QT's command line arguments and terminates the program. I start whonix-setup-wizard passing the QT style argument like follows: XDG_CURRENT_DESKTOP=gnome sudo /usr/bin/whonix-setup-wizard setup -style gtk+. This allows a nice pretty dialog to display instead of an ugly non styled root dialog. I could also forgo calling with the style if I provide a default trolltech.conf file in /root/.config/trolltech.conf.

Anyway, the fix is easy. Change line number 29 from:
args = parser.parse_args()
to
args, unknown_args = parser.parse_known_args()

The other issue I have to track down which is not related to whonix-setup-wizard. I set an environmental variable QT_X11_NO_MITSHM=1 which is not being set it seems. That option prevents QT from using the MIT-SHM X11 shared memory extension.

If you could see to the change and then tag and sign whonix-setup-wizard and whonix-repository I will finalize the changes on my side and re-test.

In T233#3209, @nrgaway wrote:

I could also forgo calling with the style if I provide a default trolltech.conf file in /root/.config/trolltech.conf.

I think shipping a /root/.config/trolltech.conf should be avoided if possible. (More hacks, more room for new issues.)

I am getting confused. This is is a separate issue, unrelated here?

Anyway, the fix is easy. Change line number 29 from:
args = parser.parse_args()
to
args, unknown_args = parser.parse_known_args()

Depends on what @troubadour thinks and how @troubadour wants to proceed. I don't know if @troubadour reads this Qubes specific ticket. Since it's off-topic here.

For small, simple changes, we got used to just posting the commit message followed by a colon with a link to the change. Here is an example:
https://www.whonix.org/forum/index.php/topic,705.msg7421.html#msg7421

For more complicated issues, we got used to create separate tickets.

Then the other reviews, tests, and merged it.

The other issue I have to track down which is not related to whonix-setup-wizard. I set an environmental variable QT_X11_NO_MITSHM=1 which is not being set it seems. That option prevents QT from using the MIT-SHM X11 shared memory extension.

I am wondering why this is. Looks like this turns into fixing further, unrelated issues?

We are using these startup wrappers:

(These are required for root. kdesudo is unappropriated, because KDE specific, and the generic su-to-root supports no command line parameters [submitted a patch: T85].)

Looks like those should work differently if some gtk marker file or detection exists and also work differently if some qubes marker file or detection exists? Anyhow, seems like those should go into separate tickets.

If you could see to the change

See above.

and then tag and sign whonix-setup-wizard and whonix-repository

Done with whonix-repository 1.1-1.

In T233#3219, @Patrick wrote:
In T233#3209, @nrgaway wrote:

I could also forgo calling with the style if I provide a default trolltech.conf file in /root/.config/trolltech.conf.

I think shipping a /root/.config/trolltech.conf should be avoided if possible. (More hacks, more room for new issues.)
I am getting confused. This is is a separate issue, unrelated here?

Now I am confused :) The issue I have with whonix-setup-wizard at the moment is that it is preventing command line arguments from reaching the QTGui since the current argparse call throws an error when unknown command line arguments are provided. A solution to this is in the example I provided below.

Anyway, the fix is easy. Change line number 29 from:
args = parser.parse_args()
to
args, unknown_args = parser.parse_known_args()

Depends on what @troubadour thinks and how @troubadour wants to proceed. I don't know if @troubadour reads this Qubes specific ticket. Since it's off-topic here.
For small, simple changes, we got used to just posting the commit message followed by a colon with a link to the change. Here is an example:
https://www.whonix.org/forum/index.php/topic,705.msg7421.html#msg7421
For more complicated issues, we got used to create separate tickets.
Then the other reviews, tests, and merged it.

The other issue I have to track down which is not related to whonix-setup-wizard. I set an environmental variable QT_X11_NO_MITSHM=1 which is not being set it seems. That option prevents QT from using the MIT-SHM X11 shared memory extension.

I am wondering why this is. Looks like this turns into fixing further, unrelated issues?

Yes, this issue not directly related to whonix-setup-wizard, although it would prevent whonix-setup-wizard from displaying any text in dialog boxes it creates. The Qubes template does not allow access to shared memory and setting QT_X11_NO_MITSHM instructs QT not to attempt to use it.

It should only effect Qubes release 3 at the moment and there have been no whonix-* releases published by ITL for release 3. I am investigating the issue on my end.

We are using these startup wrappers:

(These are required for root. kdesudo is unappropriated, because KDE specific, and the generic su-to-root supports no command line parameters [submitted a patch: T85].)
Looks like those should work differently if some gtk marker file or detection exists and also work differently if some qubes marker file or detection exists? Anyhow, seems like those should go into separate tickets.

If you could see to the change

See above.

and then tag and sign whonix-setup-wizard and whonix-repository

Done with whonix-repository 1.1-1.

Thanks, I have implement the change to use whonix-repository 1.1-1. I will wait on @troubadour to see if he can implement my suggested change otherwise I will need to create some type of work-around.

In T233#3226, @nrgaway wrote:

Now I am confused :) The issue I have with whonix-setup-wizard at the moment is that it is preventing command line arguments from reaching the QTGui since the current argparse call throws an error when unknown command line arguments are provided. A solution to this is in the example I provided below.

Yes, this issue not directly related to whonix-setup-wizard, although it would prevent whonix-setup-wizard from displaying any text in dialog boxes it creates. The Qubes template does not allow access to shared memory and setting QT_X11_NO_MITSHM instructs QT not to attempt to use it.

Looks like two separate issues to me. None of them is related to this one. Let me explain where I am coming from. There seem to be too many unoutspoken, implicit assumptions. I would like to keep this ticket focused on the topic of this one:
qubes-whonix update issue because of bug in whonix-setup-wizard / whonix_repository
See also original ticket description. About binary updates, apt repository line creation, --codename vs --repository and stable vs wheezy and /etc/apt/sources.list.d/whonix.list. Very limited scope.
What's the good for? Well, for example

  • T232 was mostly my area. About the bug in whonix-repository and whonix-setup-wizard.
  • T233, this one, is mostly @nrgaway's and @WhonixQubes's area. About how T232 effects qubes-whonix. I assumed, this one would be about how to tell existing qubes-whonix users that are affected by this bug how to update, about signed git tags and a bug fix release.
  • whonix-setup-wizard, argparse is mostly @troubadour's area
  • whonix-setup-wizard, setting env var QT_X11_NO_MITSHM condionally, is mostly @troubadour's area

These are all very different tasks. Now, when we are in the middle of a ticket, this one, that is about /etc/apt/sources.list.d/whonix.list, I find it very non-ideal to discuss for example argparse there, because the ticket where it's written, is not @troubadour's main area. It may be read, but without having a reminder set up, the request is easily forgotten in the overwhelming flood of tasks. To find out what's to do next, I for one, have a bookmark for the Whonix 10 tag open+review search. I am not sure, but I guess others do similar searches for other tags (components, such as whonix-setup-wizard). But when viewing those search results, I only read the subjects. That's why good subjects are very important. From there I decide on what to work on next. Any task that is off-topic in the middle of some ticket gets overlooked at that point. When you are lucky, it gets addressed when working on the ticket, because one might re-reads the whole discussion or when closing the ticket when hopefully checking if everything has been addressed.

Perhaps when there are easy changes (ex: argparse), that are important to you for point releases and need them to be done quickly, because otherwise you'd need to apply workarounds, create a new ticket, that only talks about argparse. Perhaps, use priority "high". This should help to know to get it done next. (Changing priority of T203, T207, etc. to priority "high", does not look so useful to me. It's true that those would be great enhancements, but those tasks do not get any simpler and cannot/will not be implemented any faster, because priority is set to "high".)

I do actually see the argparse being related to this ticket on a broad scope since the issue directly impacted the ability to use whonix-setup-wizard.

On a narrow scope, I understand your method and reasoning for separating out tickets and agree that they have merit to create their own tickets at this point. My goal was to provide the impact this bug had which contains the necessary information for someone reading this ticket. My other comments were in direct response to your questions and comments and again detailing the issues.

Unfortunately I have not had the time to become very familiar with the functionality of this ticketing system and therefore welcome @WhonixQubes or yourself to move the appropriate messages into its own task. I can not see an option to do that. I will try to set aside some time in the coming weeks to familiarize myself better with the system.

I do actually see the argparse being related to this ticket on a broad scope since the issue directly impacted the ability to use whonix-setup-wizard.

Does it impact the ability to use the wizard or is that a cosmetic issue? When run with sudo or kdesudo (or gksu), the style defaults to a root one, not that pretty, I agree.

I have implemented the proposed change in argparse without usability issue (if the second argument is not recognized, it's ignored) but before I push the change, could you check that there is no problem with the layout. The styles are changed and the tor status page is sort of messed up with the gtk style (no dynamic resizing in the wizard). Even if it reports:

QGtkStyle was unable to detect the current GTK+ theme.

the buttons are changed, and so it seems, the font and spacing.

In order that the two versions of Whonix don't start to drift apart, the solution would be the patch to su-to-root proposed by Patrick in T85, Tested, it works and respects the environment style (Oxygen in Whonix). I assume you would not have to specify gtk+ in Gnome.

In order that the two versions of Whonix don't start to drift apart, the solution would be the patch to su-to-root proposed by Patrick in T85, Tested, it works and respects the environment style (Oxygen in Whonix). I assume you would not have to specify gtk+ in Gnome.

The patch won't land in Debian before Debian stretch. So quite some time to go.

The wrappers are here:

We could add some "if qubes" (or "if gtk"), then "set these env vars..." logic to thse wrappers.

I do actually see the argparse being related to this ticket on a broad scope since the issue directly impacted the ability to use whonix-setup-wizard.

Does it impact the ability to use the wizard or is that a cosmetic issue? When run with sudo or kdesudo (or gksu), the style defaults to a root one, not that pretty, I agree.
I have implemented the proposed change in argparse without usability issue (if the second argument is not recognized, it's ignored) but before I push the change, could you check that there is no problem with the layout. The styles are changed and the tor status page is sort of messed up with the gtk style (no dynamic resizing in the wizard). Even if it reports:

QGtkStyle was unable to detect the current GTK+ theme.

the buttons are changed, and so it seems, the font and spacing.

I ran the tests with your changes and everything seems fine with the visual tests with the one exception you mentioned. The top part of the Tor status page scrolls a bit but is acceptable IMO. I am still not finished with my overall tests since I have a problem on initial run of whonix-setup-wizard where Tor is not restarting. Will get back on that once I can find the problem.

In T233#3268, @nrgaway wrote:

I have a problem on initial run of whonix-setup-wizard where Tor is not restarting. Will get back on that once I can find the problem.

I bumped into this issue when creating Debian jessie based Whonix builds. My speculation is, that it's a bug in Tor and/or Tor's systemd unit / sysvinit script. I think it only happens on first boot, so for debugging it would make sense to make a snapshot at the point where Tor gets reload etc. I am quite certain the issue is not directly caused by whonixsetup or whonix-setup-wizard. For debugging purposes, I think it would be easiest if the the Tor enable logic should be tested in a simple, minimal shell script. Scripts that can be manually run for testing:

Fixing the likely systemd related issue requires to make those scripts compatible with systemd.

In T233#3269, @Patrick wrote:
In T233#3268, @nrgaway wrote:

I have a problem on initial run of whonix-setup-wizard where Tor is not restarting. Will get back on that once I can find the problem.

I bumped into this issue when creating Debian jessie based Whonix builds. My speculation is, that it's a bug in Tor and/or Tor's systemd unit / sysvinit script. I think it only happens on first boot, so for debugging it would make sense to make a snapshot at the point where Tor gets reload etc. I am quite certain the issue is not directly caused by whonixsetup or whonix-setup-wizard. For debugging purposes, I think it would be easiest if the the Tor enable logic should be tested in a simple, minimal shell script. Scripts that can be manually run for testing:

Fixing the likely systemd related issue requires to make those scripts compatible with systemd.

Created a task T251

In T233#3268, @nrgaway wrote:

I ran the tests with your changes and everything seems fine with the visual tests with the one exception you mentioned. The top part of the Tor status page scrolls a bit but is acceptable IMO.

Pushed the modified argparser (+ unknown_args) and increased the vertical size to accommodate the style change with GTK (looks better than the top text box with a scroll bar). Would have had to do it with Qt too, and the root "style" is only slightly shifted.

https://github.com/troubadoour/whonix-setup-wizard/commit/33b63f7d6aca34cf47b2d12e22180205050598ef

In T233#3268, @nrgaway wrote:

I ran the tests with your changes and everything seems fine with the visual tests with the one exception you mentioned. The top part of the Tor status page scrolls a bit but is acceptable IMO.

Pushed the modified argparser (+ unknown_args) and increased the vertical size to accommodate the style change with GTK (looks better than the top text box with a scroll bar). Would have had to do it with Qt too, and the root "style" is only slightly shifted.
https://github.com/troubadoour/whonix-setup-wizard/commit/33b63f7d6aca34cf47b2d12e22180205050598ef

Awesome. I will add it to my build config once Patrick gets the chance to tag and sign it. Thanks for going to the effort to get this issue resolved so quickly.

Signed git tag 0.7-1 contains that change.

In T233#3369, @Patrick wrote:

Signed git tag 0.7-1 contains that change.

Thanks, Will try running tests tonight.

Patrick set Impact to High.

If there is anything todo in the Whonix core for Whonix 10 release, please re-add the Whonix 10 tag.

Actually, I am not sure about the tag. Does this contain anything, that would speak against a stable release of Whonix 10? If you don't know yet, it's also fine to re-add the Whonix 10 tag. Just want to make sure, that this won't needlessly block the release of Whonix 10.

Resolved with qubes-whonix unreleased 9.6.9 and 10

nrgaway closed this task as Resolved.May 16 2015, 3:04 PM
nrgaway claimed this task.