Page MenuHomePhabricator
Create Task
Maniphest T417

up to date versions of Tor Browsers in newly created AppVMs inherited from updated TemplateVMs
Closed, ResolvedPublic
Actions

Description

Prerequisite knowledge:
https://www.whonix.org/wiki/Dev/Qubes#tb-updater_vs_TemplateVM

Goal:
After updating the TemplateVM, at least newly created AppVMs based on the updated TemplateVM should come with an up to date version of Tor Browser.

Non-goal:
Updating existing installations of Tor Browser in existing AppVMs. [Economically impossible in the absence of The Tor Project maintaining a proper Debian package while preserving user data (bookmarks, etc.).] Those still have to be updated with Tor Browser's internal updater. If further discussion on this non-goal is required, a separate discussion should be opened.

Alternative technical task title:
ship Tor Browser tarballs in Qubes TemplateVMs in /var/cache/tb-binary and extract in AppVMs at boot time to user's home folder

Implementation:

  • in tb-updater postinst / update-torbrowser

Deprecated:

  • Create a package tb-binary, that ships a folder /var/cache/tb-binary that includes the Tor Browser tarball tor-browser-linux64-x.x_en-US.tar.xz as well as signature tor-browser-linux64-x.x_en-US.tar.xz.asc.
  • During boot of AppVMs, a script should check if Tor Browser is already installed in user's home folder. And if not, verify [reusing tb-updater code] and extract Tor Browser from /var/cache/tb-binary to user's home folder.
    • [The verification makes shipping malicious files in the tb-binary package less attractive.]
  • Configurable through /etc/torbrowser.d folder (can be turned off).

Questions:

  • Is there any more appropriate folder than /var/cache/tb-binary as per FHS?

Details

Impact
Normal

Related Objects

Event Timeline

Patrick created this task.Oct 14 2015, 10:02 AM
Patrick raised the priority of this task from to Normal.
Patrick updated the task description. (Show Details)
Patrick added projects: Qubes, Whonix 13, tb-updater.
Patrick set Impact to Normal.
Patrick added subscribers: Patrick, marmarek, nrgaway and 3 others.
Herald added a project: Whonix. · View Herald TranscriptOct 14 2015, 10:02 AM
Herald added subscribers: • WhonixQubes, troubadour. · View Herald Transcript
Patrick mentioned this in T419: no longer write to home folder directly; use /etc/skel.Oct 17 2015, 4:45 PM
Patrick added a comment.Dec 15 2015, 11:28 PM
allow storing downloaded archives in arbitrary folders

https://phabricator.whonix.org/T417

https://github.com/Whonix/tb-updater/commit/1c85166a20af638ceab25900fb75b9f62a9645d7

Patrick added a comment.Dec 25 2015, 4:36 PM

I can't host Tor Browser binaries.

A new package has been created, but it's still boilerplate without binaries:
https://github.com/adrelanos/tb-binary

I can't host this for trademark issues, see:
https://www.whonix.org/wiki/Tor_Browser#Not_installed_by_Default_Footnote

Also source code license is too difficult to get right, see:
https://github.com/adrelanos/tb-binary/blob/master/debian/copyright

And I don't know if there is such a thing as a compilation license of the binary itself.

This is going to be very ugly. This will bloat Whonix source code size. All the old binaries of Tor Browser will be kept by git as newer versions are added. Removing them, rewriting git history, would require to use ugly things such as git push --force and git merge --force. I really appreciated suggestions on how this could be implemented less ugly.

Patrick closed this task as Invalid.Jan 5 2016, 4:17 PM
Patrick claimed this task.

As discussed at 32c3, a tb-binary package is the wrong way to implement this. Instead, Tor Browser will be downloaded during postinst of tb-updater. Will be done in T461.

Patrick reopened this task as Open.Jan 5 2016, 4:29 PM
Patrick updated the task description. (Show Details)

This isn't done. Just the implementation details changed.

Patrick added a comment.Jan 5 2016, 11:19 PM

work on up to date versions of Tor Browser in newly created AppVMs:
https://github.com/Whonix/tb-updater/commit/cbdb6924e6deeae1381e8e4c3a6812ba71bacd46

Patrick added a comment.Jan 5 2016, 11:21 PM

TLDR:
Would installation of Whonix packages to build Qubes-Whonix happen inside a Qubes TemplateVM* or inside a chroot? @marmarek

*Or StandaloneVM. [Or TemplateBasedVM for testing purposes.]

Long:
Background:
I need to know this for T461 (Installation from Repository / proverbial "sudo apt-get install whonix"). The initial installation of Tor Browser either happens inside a TemplateVM or a chroot.

More background:
The tb-updater postinst mechanism to download Tor Browser archives to /var/cache/tb-binary makes only sense in a TemplateVM. So I will detect this from the tb-updater postinst script. Building Whonix may be a special case. The initial installation of Tor Browser might happen from within a chroot. Depending on how we want this ticket to work out.

Even more background:
The tb-updater postinst mechanism to download Tor Browser archives to /var/cache/tb-binary makes only sense in a TemplateVM.

  • TemplateBased or Standalone AppVMs would just update using Tor Browser's internal updater. Or download a new Tor Browser using tb-updater to the home folder.
  • TempalteVMs would store Tor Browser archives to /var/cache/tb-binary, so newly created TemplateBased [or Standalone] AppVMs would start with an update to date version of Tor Browser, which is the goal of this ticket.
marmarek added a comment.Jan 6 2016, 12:33 AM
Would installation of Whonix packages to build Qubes-Whonix happen inside a Qubes TemplateVM* or inside a chroot? @marmarek

chroot.

Patrick added a comment.Jan 6 2016, 4:33 PM

work on up to date versions of Tor Browser in newly created AppVMs:
https://github.com/Whonix/tb-updater/commit/b2b79121d7149d09ce7d5d20dec766e231db4223

This feature is being documented here:
https://www.whonix.org/wiki/Tor_Browser#tb-updater_postinst

Patrick added a comment.Jan 6 2016, 5:24 PM

Please leave some feedback on the usability impact here. I appreciated a small "okay" message. @bnvk @mfc @marmarek
(To avoid spending time on implementing something we later decide to not enable by default.)

Implementing this feature would make upgrading the Qubes-Whonix-Workstation on every tb-updater package upgrade (on new Tor Browser releases) at least 6 minutes slower. On slower [Tor] connections it could easily take 12 or more minutes.

The worst case here would be users that only use a single Qubes-Whonix-Workstation TemplateBased AppVM and sticking with it for a long time. These would not benefit from this feature.

[Presuppose they keep their TempalteVM up to date.]
Users who would benefit from this feature would be users who often create new AppVMs. These would start with an up to date version of Tor Browser when they create new AppVMs. Therefore would not need to update Tor Browser using Tor Browser's internal updater in newly created AppVMs.

Very likely (depending on how DispVMs will be implemented in future), this feature is required to have Qubes-Whonix-Workstation based DispVMs start with up to date versions of Tor Browser.

Speaking here about the default settings. This could be disabled by advanced users as documented.

Don't worry. It's not rocket science to implement this. And fun work.

mfc added a comment.Jan 6 2016, 5:36 PM

Patrick (Patrick Schleizer):

Very likely (depending on how DispVMs will be implemented in future), this feature is required to have Qubes-Whonix-Workstation based DispVMs start with up to date versions of Tor Browser.

Is this the case with how DispVMs are currently implemented? It would be
ideal to get to the state where the user can create Tor Browser DispVMs
that are up-to-date.

Patrick added a comment.EditedJan 6 2016, 5:41 PM

mfc (mfc):

Is this the case with how DispVMs are currently implemented?

Yes.

It would be ideal to get to the state where the user can create Tor Browser DispVMs that are up-to-date.

If nothing goes wrong, this is close (Whonix 13).

marmarek added a comment.Jan 6 2016, 10:17 PM
In T417#7845, @Patrick wrote:

Implementing this feature would make upgrading the Qubes-Whonix-Workstation on every tb-updater package upgrade (on new Tor Browser releases) at least 6 minutes slower. On slower [Tor] connections it could easily take 12 or more minutes.

IMHO not a big problem. And if it is, user can disable this feature (as you pointed in documentation).

The worst case here would be users that only use a single Qubes-Whonix-Workstation TemplateBased AppVM and sticking with it for a long time. These would not benefit from this feature.

How does Tor Browser internal updater work? Maybe it can be configured to look for a package downloaded by tb-updater package first? Just an idea, ignore if not trivial.

Very likely (depending on how DispVMs will be implemented in future), this feature is required to have Qubes-Whonix-Workstation based DispVMs start with up to date versions of Tor Browser.

Your idea of DispVMs (read-only private image, instead of no private image) makes it not so obvious. If DispVM could be started out of any AppVM, then user could have already installed Tor Browser in that VM (used for DispVM).
But generally IMHO we should think how to make use of already downloaded Tor Browser in that case. To simplify update procedure. It would be terrible UX if user is required to update some applications separately (->manually) from package manager (->done by management stack in the future as one-click button for the whole system).

So, generally I like the approach presented here.

Patrick added a comment.Jan 6 2016, 10:25 PM

Thanks! So I will work on completing this feature.

In T417#7852, @marmarek wrote:

How does Tor Browser internal updater work? Maybe it can be configured to look for a package downloaded by tb-updater package first? Just an idea, ignore if not trivial.

It uses different files for that. .mar files. And Firefox's update magic. Really non-trivial. Unfortunately. Otherwise I also had in mind to initiate updating Tor Browser automatically in AppVMs [in background] by starting/scripting its updater. Too fragile/difficult. (rejected ticket: https://trac.torproject.org/projects/tor/ticket/17136) Would have to manually figure out the correct .mar file. Could not use the internal logic to fetch the right one as this is not provided by the command line interface.

Patrick added a comment.Jan 6 2016, 10:36 PM
avoid requirement for sudoers exception thanks to suggestion by @marmarek ;
work on up to date versions of Tor Browser in newly created AppVMs

https://github.com/Whonix/tb-updater/commit/e3c3b0b816cba1a65992b8810517f9f2c76b181e

Patrick mentioned this in T463: Qubes-Whonix-Workstation DispVM Support.Jan 6 2016, 11:41 PM
Patrick added a comment.Jan 7 2016, 6:22 PM

Do you think it would be crazy to download and extract Tor Browser to /etc/skel rather than /var/cache/tb-binary? It would violate Debian policy. (Non-packaging software writing to /etc.) But it would be a bit simpler. Then /usr/lib/anon-base-files/first-boot-skel would automagically keep care of copying Tor Browser to the user's home. No separate tb-updater specific /var/cache/tb-binary to user home population systemd service required then. @marmarek

marmarek added a comment.Jan 7 2016, 10:54 PM

I'm not sure. Technically it sounds good, but I feel it's something wrong...

Patrick added a comment.Jan 8 2016, 2:34 AM

Alright. tb-updater got its own first-boot-home-population systemd service.

https://github.com/Whonix/tb-updater/commit/b51d4580dc2f056e78ea7d83a65d0dd2f763c492

Patrick added a comment.Jan 8 2016, 2:38 AM

systemd packaging
https://github.com/Whonix/tb-updater/commit/2324576595df433cd6614d591f777ec9e7244bde

Patrick changed the task status from Open to Review.Jan 8 2016, 2:55 AM

It works! We have Tor Browser in Qubes-Whonix-Workstation DispVMs now!

mfc added a comment.Jan 8 2016, 11:45 AM

niiiiiiiiice!!

Patrick updated the task description. (Show Details)Mar 18 2016, 3:58 PM
Patrick mentioned this in T461: Installation from Repository / proverbial "sudo apt-get install whonix".Apr 7 2016, 11:26 PM
Patrick added a comment.Apr 8 2016, 9:06 PM
moved hardcoded version for postinst into its own file for easier maintenance

/usr/share/tb-updater/tbb_hardcoded_version

https://phabricator.whonix.org/T417

https://github.com/Whonix/tb-updater/commit/e9469d65c457dec971a2ce31521f1707a7d69deb

Patrick added a comment.Apr 26 2016, 5:22 AM

https://github.com/Whonix/qubes-whonix/commit/8438d13d75822e9ea800b9eb6024063f476636ff
https://github.com/Whonix/tb-updater/commit/1620a4361ca719089be43bb717dcbd4536198e10

Patrick changed the task status from Review to Open.Apr 29 2016, 4:55 AM

TODO

1)

On manual run of update-torbrowser in TemplateVM stores Tor Browser still in the home folder, so up to date versions of Tor Browsers in newly created AppVMs are not inherited from updated TemplateVMs.

(Currently only works when update-torbrowser is run during tb-updater package upgrade.)

Somewhat counter intuitive. Probably. And should be fixed.

2)

Minor, cli output should explain hardcoded version number.

Patrick added a comment.Apr 30 2016, 12:42 AM

That is done.

https://github.com/Whonix/tb-updater/commit/0f116907a9911161e4ae9837a2f1766c674313c4
https://github.com/Whonix/tb-updater/commit/aec876116f900c6450325f5896b09349a92a5e88
https://github.com/Whonix/tb-updater/commit/5424e6bad9175dc5fc917865792a649f6450e604
https://github.com/Whonix/tb-updater/commit/90bc9f530ec132255248509095b664540acf1527
https://github.com/Whonix/tb-updater/commit/0991750e8255fb80d94b9cb41249fadbd2c2753c
https://github.com/Whonix/tb-updater/commit/38881d0ee10a62182637a632c4dba48ee717ac88
https://github.com/Whonix/tb-updater/commit/a7d5e57884ced88f9dc7a733638312e94d135870

Patrick added a comment.Apr 30 2016, 12:52 AM

Problem:

Upgraded TemplateVMs - i.e. installed prior Whonix 13 - will still have Tor Browser installed in their home folder. Since the home folder of TemplateVMs is still always and by default inherited by TemplateBasedVMs as of Qubes R3.1 (R3.2 probably also), this feature does not work. (Because TemplateVM home contains old version, which gets inherted by TemplateBasedVM and the new tb-updater first boot population systemd service will not touch existing folders.)

TODO:

  • 1) Decide how to resolve the station. Either
    • a) delete folders ~/.tb and ~/cache/tb using a maintainer script during template upgrade
    • b) or better move these folders somewhere out of the way, to avoid the user having data loss (customized Tor Browser in TemplateVM) [and avoid eventual bugs from causing damage such as running the deletion code in a non-TemplateVM]
    • c) not automate it, but inform the user about this inside the tb-updater output
    • d) do nothing about it, wait and see if Qubes R4 will no longer inherit TemplateVMs home folder
  • 2) Implement the solution.
Patrick added a comment.May 4 2016, 9:51 PM

chosen c)

https://github.com/Whonix/tb-updater/commit/dd33b036ac8a5c0d0466983ed1e6c52ae1684eda

Patrick changed the task status from Open to Review.May 4 2016, 9:58 PM
Patrick closed this task as Resolved.May 5 2016, 3:06 AM
Patrick mentioned this in T666: Tor Browser 6.5.2 tb-updater update.Apr 29 2017, 10:19 AM
Patrick mentioned this in T671: old Tor Browser versions in /var/cache/tb-binary/.tb/ accumulate in Qubes-Whonix, users run into full up disk error issues.Jun 5 2017, 1:15 PM
Whonix OLD Issue Tracker · PLEASE DO NOT POST NEW TICKETS HERE · OLD Issue Tracker - Unread Notifications · OLD Issue Tracker - Feed · OLD Issue Tracker - Open Issues · NEW Issue Tracker · Homepage · Blog · Forum · Legal · Imprint · Privacy Policy · Terms of Use · Disclaimer