At the moment we have three choices:

• Run TBB headlessly (xvfb) on GW and configure pluggable transports through torrc. Its a compromise on usability but gains us the advantage of the latest pluggable transports and reproducible binaries. Should upstream ever add standlone support for torlauncher-gui it should be easy to enable it for GW.

• Run TBB with its full GUI but make sure the browser window cannot connect anywhere, to allow using torlauncher-gui too. This might however trigger the redistribution problems we avoid with running headlessly.

• Keep the status quo - the "worst" option because the poor situation of pluggable transport support outside TBB.

Good ideas! I would call this development / TODO research tasks, though.

In T118#6198, @HulaHoop wrote:

At the moment we have three choices:

Four. Don't forget the best solution described in the original ticket description. It just doesn't materialize because no one speaks/contributes javascript. I am wondering how useful it is to go through any of the hacks below and not using that time to learn javascript and doing the real fix.

• Run TBB headlessly (xvfb) on GW and configure pluggable transports through torrc. Its a compromise on usability but gains us the advantage of the latest pluggable transports and reproducible binaries. Should upstream ever add standlone support for torlauncher-gui it should be easy to enable it for GW.

We would still need scripts for starting/stopping TBB then. Or documentation on which commands to run. Would also need to check which config file changes are required to make TBB just connect without needing to click anything in tor-launcher (which would then not be visible). And debugging we would need documentation on how to make it visible? (Imagine it doesn't work for someone.)

• Run TBB with its full GUI but make sure the browser window cannot connect anywhere, to allow using torlauncher-gui too. This might however trigger the redistribution problems we avoid with running headlessly.

Would also cause an usability issue, because of the redundant (from usability perspective, not technical) browser window.* Also a security issue, because blocking the browser from connecting anywhere would be hard. Would either need some patch upstream (environment variable TBB_LOCKDOWN or so) or modifying TBB after extracting it (which can break due to upstream changes).

*Especially a usability issue on Qubes, because there the window would not be hidden in a different VM window but end up on the usual task bar. The user could move the redundant window to another virtual desktop, though.

In any case, please consider implementing T116 first.

Launching Tor and TBB from command line:
https://askubuntu.com/questions/320545/how-to-launch-tor

/etc/rc.local for xvfb auto startup on Debian:
http://technology.manyask.com/viewtopic.php?t=1488398

I'm still trying to figure out how to run TBB under xvfb. With firefox the command is:
xvfb-run firefox
http://elementalselenium.com/tips/38-headless

We would still need scripts for starting/stopping TBB then. Or documentation on which commands to run. Would also need to check which config file changes are required to make TBB just connect without needing to click anything in tor-launcher (which would then not be visible). And debugging we would need documentation on how to make it visible? (Imagine it doesn't work for someone.)

The browser part of TBB will always be necessary for some pluggable transports that directly depend on it like meek. So I'm not sure the upstream solution for stopping TBB from starting will help us here. We have to hide the window.

HulaHoop (HulaHoop):

Launching Tor and TBB from command line:

Easy. No need to worry about this right now. tb-starter must already
know how to start it, no? Just type torbrowser. Or `bash -x
torbrowser` to see what it's doing. Just cd into the TBB folder, run
./start-tor-browser.desktop

https://askubuntu.com/questions/320545/how-to-launch-tor

Outdated and not required.

/etc/rc.local for xvfb auto startup on Debian:
http://technology.manyask.com/viewtopic.php?t=1488398

The autostart is one of the simplest things here. (Can be implemented as
systemd service, and/or /etc/xdg/autostart, there are many ways.) No
need to worry about this right now.

I'm still trying to figure out how to run TBB under xvfb. With
firefox the command is: xvfb-run firefox
http://elementalselenium.com/tips/38-headless

Won't work with torbrowser script. Launches into the background using
&. (Just remove the & in /usr/bin/torbrowser for lines containing
start-tor-browser.

Or cd into the TBB folder. xvfb-run ./start-tor-browser.desktop
won't work. (Using detached start.) `xvfb-run
./Browser/start-tor-browser` works.

The browser part of TBB will always be necessary for some pluggable
transports that directly depend on it like meek. So I'm not sure the
upstream solution for stopping TBB from starting will help us here.
We have to hide the window.

Stopping the whole thing, TBB, required for restarting Tor and clean
shutdowns. The process will likely be stoppable by a usual sigterm. Not
hard. Just needs to be implemented.

The easiest would be, if tor-launcher wouldn't open the Tor Browser
window and just stay as is. Perhaps as a tray icon. (By the time you can
see tor-launcher, Firefox is already running. Just it's main window does
not come up because tor-launcher prevents this. tor-launcher is an
add-on running inside Firefox itself.)

The next step really should be T116. Auto start, stop, xvfb are
periphery. T116 requires more difficult work. (compatibility with system
Tor, stopping system Tor, maybe no longer auto starting system Tor,
having TBB's Tor provide as system wide Tor, have TBB use Whonix's
/usr/share/tor/tor-service-defaults-torrc. All needs instructions.)

I'm assuming we are completely replacing Debian repo's system Tor with TBB Tor.

Where torrc is for Tor Browser Bundle:
https://tor.stackexchange.com/a/1072

(will need to update paths to torrc in wiki documentation and for shortcuts. A benefit of this change is users can replace torrc files before first run because its in a non-root directory)

Configure xvfb for Vivid (15.04) as a service using systemd:
http://askubuntu.com/questions/621246/configure-xvfb-for-vivid-15-04-as-a-service-using-systemd

Tor Browser environment variables controlling how Tor starts and exits:
https://trac.torproject.org/projects/tor/ticket/13005

I'm assuming we are completely replacing Debian repo's system Tor with TBB Tor.

I didn't. But forgot to make explicit, that this should be optional. At least for its initial implementation. Due to the volatileness of TBB. (Upstream is moving files around and making otherwise bold changes.) I think this whole thing is a hack. Should therefore just be optional. It's also more likely to break. Too experimental to make it the default for everyone. And cumbersome. Instructions would likely include steps such as:
sudo cp /usr/share/tor/tor-service-defaults-torrc /home/debian-tor/.tb/tor-browser_en-US/Browser/TorBrowser/Data/Tor/torrc-defaults
Or somehow this would need to be scripted.

~/.tb/tor-browser_en-US/Browser/TorBrowser/Data/Tor/torrc gets edited by tor-launcher. See comments.

# This file was generated by Tor; if you edit it, comments will not be preserved
# The old torrc file was renamed to torrc.orig.1 or similar, and Tor will ignore it

Bridge scramblesuit [...]

~/.tb/tor-browser_en-US/Browser/TorBrowser/Data/Tor/torrc-defaults is static, but also very likely be updated by upstream in future.

Merging in Whonix's /usr/share/tor/tor-service-defaults-torrc is also required. Difficult in the absence of torrc.d-style configuration directories.

Implementing this ticket is hard. Start with T116 first.

I think this whole thing is a hack. Should therefore just be optional. It's also more likely to break. Too experimental to make it the default for everyone. And cumbersome.

Then for a stable based OS like Whonix this is not an option. Eventually more pluggable transport binaries will make it into their repos upstream so we don't really have to go thru all this to eventually use them.

We still support obfsproxy at the moment and its not just a PT but a framework that supports many types of pluggable transports.

But then we'll don't get a pluggable transports gui (tor-launcher) within the next how many years. And no access to recent (working!) pluggable transports.

At least T116 is good to have. When T116 is done, this one will be will be more in reach.

Another option would be documenting how to use TBB running on the host or in another VM and having Whonix-Gateway use it. But that should become another ticket.

Another blocker problem: TBB will refuse to run as root. Not going to be possible to run it as system Tor. Cannot use debian-tor group.

Its a sound design decision because why would they want users running a vulnerable browser as root? but its a showstopper unless upstream changes their core development plans to include target users other than jut desktop users for TBB.

https://tor.stackexchange.com/questions/3224/the-tor-browser-bundle-should-not-be-run-as-root-why

https://tor.stackexchange.com/questions/6705/make-tor-browser-bundle-use-debian-tor-tor-and-not-tbb-tor

I succeeded starting TBB as user debian-tor.

sudo mkdir /home/user/debian-tor
sudo chown --recursive debian-tor:debian-tor /home/user/debian-tor
sudo usermod debian-tor -s /bin/bash

Moved the TBB folder to /home/user/debian-tor and started it there.

All tested only on regular Debian for now. Should not be that hard to do the same on Whonix.

I succeeded starting TBB as user root. Just commented out the root check in the startup script.

File: Browser/start-tor-browser
Comment out:

if [ "`id -u`" -eq 0 ]; then
        complain "The Tor Browser Bundle should not be run as root.  Exiting."
        exit 1
fi

This could be doable with sed. Also submitting a patch upstream would not be hard. Getting it merged a little harder. But judging from when I talked last time with Isis on tor-talk, I guess she could be supportive. Simple patch after all.

But I don't think we'd need to run as user root anyhow. The only port below 1024 we're listen on is the DNS port. And there is no reason to not move that DNS port up.

Nice progress!

I looked for ways to enable TBB's automatic update functionality when its running headless. It can be done by shipping a custom pref.js/user.js settings file. We can configure when updates happen and if they happen automatically without having to interact with a GUI notification to permit it. This way we can bundle TBB with Whonix Gateway and not have to rely on any updater mechanisms outside TBB itself.

App.update.auto

http://kb.mozillazine.org/Category:Preferences
http://kb.mozillazine.org/App.update.auto
http://kb.mozillazine.org/About:config_entries#Update._and_Update_notifications.

http://kb.mozillazine.org/Editing_configuration

While it's easier to use about:config for a single profile, it may be easier to use a user.js file if you need to make the same changes in many profiles (see the linked article for more information).

http://kb.mozillazine.org/About:config

about:config is a feature of Mozilla applications which lists application settings (known as preferences) that are read from the profile files prefs.js and user.js, and from application defaults. Many of these preferences are not present in the Options or Preferences dialog. Using about:config is one of several methods of modifying preferences and adding other "hidden" ones.

https://support.mozilla.org/en-US/questions/965842#answer-459525
https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/A_brief_guide_to_Mozilla_preferences

User pref. files

In the profile directory are two user pref files: prefs.js and user.js. prefs.js is automatically generated by the application and should not be edited manually, whereas user.js is an optional file the user can create to override preferences initialized by other preferences files.

On stability problems, I think the stable branch of Tor Browser is very reliable and solid they are based on upstream Firefox long term releases.

TBB 5+ enables automatic updates by default. No custom modifications needed.

Starting with this release, Tor Browser will now also download and apply upgrades in the background, to ensure that users upgrade quicker and with less interaction. This behavior is governed by the about:config pref app.update.auto, but we do not recommend disabling it unless you really know what you're doing.

https://blog.torproject.org/blog/tor-browser-50-released

In T118#6361, @HulaHoop wrote:

TBB 5+ enables automatic updates by default. No custom modifications needed.

Starting with this release, Tor Browser will now also download and apply upgrades in the background, to ensure that users upgrade quicker and with less interaction. This behavior is governed by the about:config pref app.update.auto, but we do not recommend disabling it unless you really know what you're doing.

https://blog.torproject.org/blog/tor-browser-50-released

Yes. Nice. That helps. So in theory we start TBB headless to have it upgrade itself. The same would likely apply if we could implement what this ticket was originally for.

One problem I can see is the need to restart TBB for updates to take effect. Without a GUI it's not possible for a user to know this information directly but needrestart can detect updated daemons by open file descriptors and restart them.

In T118#6510, @HulaHoop wrote:

One problem I can see is the need to restart TBB for updates to take effect. Without a GUI it's not possible for a user to know this information directly but needrestart can detect updated daemons by open file descriptors and restart them.

Reference? I am skeptical about that. TBB is not a daemon in the sense that needrestart knows daemons. And TBB upgrades itself in a unique way from perspective of needrestart. For regular daemons that come as deb packages it's easier. Because their dependencies also come as deb packages. And when the dependencies are updated by apt-get/dpkg, needrestart has a chance to notice that and to recommend restart of the daemon.

https://gehrcke.de/2014/06/good-to-know-checkrestart-from-debian-goodies/

Checkrestart is a Python application wrapping lsof (“list open files”). It tries to identify files used by processes that are not in the file system anymore. How so?

Note that during an update a certain binary file becomes replaced: the new version is first downloaded to disk and then rename()ed in order to overwrite the original. During POSIX rename() the old file becomes deleted. But the old file is still in use! The standard says that if any process still has a file open during its deletion, that file will remain “in existence” until the last file descriptor referring to it is closed. While these files that are still held “in existence” for running processes by the operating system, they are not listed in the file system anymore. They can however easily be identified via the lsof tool. And this is exactly what checkrestart does.

Hence, checkrestart “compares” the open files used by running processes to the corresponding files in the file system. If the file system contains other (e.g. newer) data than the process is currently using, then checkrestart proposes to restart that process. In a tidy server environment, this usually is the case only for updated shared library files.

I see. Sounds interesting. It's still unclear if checkrestart is capable of monitoring arbitrary processes that are not managed by dpkg/apt-get. And if it would be easier to replicate this lsof based check rather than bending checkrestart to do that. (Also undesirable to have both checkrestart and needrestart installed at the same time, see T324.)

Anyhow. I don't think that this lsof based approach would work. Apparently TBB does not replace it's own binaries before it's restarted. In comparison, also deb packages shipping daemons do not replace their own binaries. dpkg does that. TBB asks users to restart. Even if the latest version automatically updates, it does not automatically restart itself. A not asked for restart would be undesirable from a user perspective, so that's unlikely. But the restart is required.

Did some analysis, had the tor-browser_en-US folder managed by git before/during/post update.

Early during upgrade download:

• Folder Browser/updates/0/ is empty. [No update ever before.]
• File Browser/updates.xml is created.
• cat Browser/active-update.xml

<updates xmlns="http://www.mozilla.org/2005/app-update"><update appVersion="5.5a1" buildID="20000101000000" channel="alpha" displayVersion="5.5a1" extensionVersion="5.5a1" installDate="1440016394100" isCompleteUpdate="false" isOSUpdate="false" name="Tor Browser 5.5a1" serviceURL="https://www.torproject.org/dist/torbrowser/update_2/alpha/Linux_x86_64-gcc3/5.0a3/en-US?force=1" showNeverForVersion="false" showPrompt="false" promptWaitTime="172800" type="minor" version="5.5a1" detailsURL="https://www.torproject.org/projects/torbrowser.html.en" platformVersion="38.2.0" previousAppVersion="5.0a3" foregroundDownload="true"><patch type="complete" URL="https://www.torproject.org/dist/torbrowser/5.5a1/tor-browser-linux64-5.5a1_en-US.mar" finalURL="https://dist.torproject.org/torbrowser/5.5a1/tor-browser-linux64-5.5a1_en-US.mar" hashFunction="SHA512" hashValue="a5a8225a287c81bd5756120abbd85042bc12094e8dc737cc9dd2670df43f1f883de580a1e0a83cdf9a25064dde50e54bbdd367d0a817892a9c34f46781f65c16" size="78383226" selected="true" state="downloading"/></update></updates>

• Above contains state="downloading"

Later during download:

• File Browser/updates/0/update.status is created.

cat Browser/updates/0/update.status

downloading

After download finished, but not yet restarted:

cat Browser/updates/0/update.status

applied

cat Browser/updates/0/update.version

5.5a1

After clicking "restart browser":

• File Browser/updates/0/update.version no longer exists.
• File Browser/updates/0/update.status no longer exists.
• File Browser/updates/last-update.log contains a lot, including a line UPDATE TYPE complete.
• cat Browser/updates.xml

<updates xmlns="http://www.mozilla.org/2005/app-update"><update appVersion="5.5a1" buildID="20000101000000" channel="alpha" displayVersion="5.5a1" extensionVersion="5.5a1" installDate="1440015600400" isCompleteUpdate="false" isOSUpdate="false" name="Tor Browser 5.5a1" serviceURL="https://www.torproject.org/dist/torbrowser/update_2/alpha/Linux_x86_64-gcc3/5.0a3/en-US" showNeverForVersion="false" showPrompt="false" promptWaitTime="172800" type="minor" version="5.5a1" detailsURL="https://www.torproject.org/projects/torbrowser.html.en" platformVersion="38.2.0" previousAppVersion="5.0a3" statusText="The Update was successfully installed" foregroundDownload="true"><patch type="complete" URL="https://www.torproject.org/dist/torbrowser/5.5a1/tor-browser-linux64-5.5a1_en-US.mar" finalURL="https://dist.torproject.org/torbrowser/5.5a1/tor-browser-linux64-5.5a1_en-US.mar" hashFunction="SHA512" hashValue="a5a8225a287c81bd5756120abbd85042bc12094e8dc737cc9dd2670df43f1f883de580a1e0a83cdf9a25064dde50e54bbdd367d0a817892a9c34f46781f65c16" size="78383226" selected="true" state="succeeded"/></update></updates>

• Above contains state="succeeded".

After another browser restart.

cat Browser/active-update.xml

<updates xmlns="http://www.mozilla.org/2005/app-update"/>

In T90#6837, @troubadour wrote:

A note about T118. Been on and off on that one. It looks like tor-launcher is merely settings variables, and that the actual work (like editing torrc or starting meek-client) is done downstream, in firefox or wherever. So I'm not sure it's even possible achieve the bridges settings that way, without starting Tor browser.

I don't think it's important for the implementation of this ticket. However, this is as I guess things internally work...

By the time tor-launcher [a Firefox is visible, Tor Browser is already running "in the background". Just not visible. Not popping up at that point. It's Tor Browser's (Firefox's) code that processes tor-launcher's [Firefox addon's] code. Therefore showing the tor-launcher gui.

https://gitweb.torproject.org/tor-launcher.git/tree/src/components/tl-process.js uses Tor ControlPort command SAFECONF to store settings.

grep -r extensions.torlauncher.default_bridge_type

Browser/TorBrowser/Data/Browser/profile.default/extensions/tor-launcher@torproject.org/components/tl-process.js:  kPrefDefaultBridgeType: "extensions.torlauncher.default_bridge_type",
Browser/TorBrowser/Data/Browser/profile.default/extensions/tor-launcher@torproject.org/chrome/content/network-settings.js:const kPrefDefaultBridgeType = "extensions.torlauncher.default_bridge_type";
Browser/TorBrowser/Data/Browser/profile.default/extensions/tor-launcher@torproject.org/modules/tl-util.jsm:    const kPrefName = "extensions.torlauncher.default_bridge_type";
Browser/TorBrowser/Data/Browser/profile.default/prefs.js:user_pref("extensions.torlauncher.default_bridge_type", "meek-amazon");

grep -r awsstatic.com

Browser/TorBrowser/Data/Browser/profile.default/preferences/extension-overrides.js:pref("extensions.torlauncher.default_bridge.meek-amazon.1", "meek 0.0.2.0:2 4EE0CC769EB4B15A872F742EDE27D298A59DCADE url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com");
Browser/TorBrowser/Data/Tor/torrc:Bridge meek 0.0.2.0:2 4EE0CC769EB4B15A872F742EDE27D298A59DCADE url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com

File Browser/TorBrowser/Data/Browser/profile.default/preferences/extension-overrides.js might not be shipped by tor-launcher (perhaps by Tor Browser or TorButton), but it's processed by tor-launcher.

It seems, that Tor writes torrc.

grep -r "comments will"

Binary file Browser/TorBrowser/Tor/tor matches
Browser/TorBrowser/Data/Tor/torrc:# This file was generated by Tor; if you edit it, comments will not be preserved

But only because tor-launcher issues SAFECONF. And I would reasonably speculate, that tor-launcher sets any [bridge] settings in Tor using the Tor control protocol. After its ready, it uses SAFECONF.