Tor port 5300 is easily answered. It's Tor's DnsPort.

https://github.com/Whonix/anon-gw-anonymizer-config/blob/49ce21f97965609e4fe06af20005637878324fcc/usr/share/tor/tor-service-defaults-torrc.anondist#L153

dhclient is a dependency of anon-gw-dhcp-conf.

https://github.com/Whonix/anon-gw-dhcp-conf/blob/b4999b1a6f79c280afbdecc521a1a05feb84526a/debian/control#L17

It is used to obtain a lan ip address for eth0 from the virtualizer. It should not be reachable from Whonix-Workstations since it is blocked by Whonix-Gateway firewall.

Anyhow. It would be useful to make dhclient listen on eh0 only. Can you figure out how to configure isc-dhcp-client that way? We could also switch the dhcp client while we are at it.

Perhaps its not a problem:

Even though netstat shows dhcpd has a socket open that is listening, but
it only processes dhcp requests on the named interface.

You can also use the local-address statement, but beware the special requirements.

https://lists.isc.org/pipermail/dhcp-users/2010-July/011973.html

However a couple of replies down:

I've found that version 3.0.4-13+etch2 (debian) listens and steals traffic not intended for it on interfaces which it is configured to not use.

https://lists.isc.org/pipermail/dhcp-users/2010-July/011974.html

I don't know much to disprove it either way but as long as it only listens (firewall takes care of that) and doesn't attempt talking to a dhcp server on the WS, it should be OK.

An alternative could be the pydhcplib in Debian:

http://pydhcplib.tuxfamily.org/pmwiki/index.php?n=Site.OptionsList

However I'm leaning towards keeping everything as it is since you seem to have paid great attention to blocking DNS resolution setting using script hooks, and changing to something else might need more effort to reach the same safety level.

Even a functional /etc/resolv.conf on Whonix-Gateway would not be scary. As long as there is no local Tor DnsPort, it wont work.

The listener on all interfaces is more scary.

How is dhclient called? -> http://stackoverflow.com/questions/14720571/which-program-invokes-dhclient-on-debian-squeeze

The way it is called influences our options to replace it.

And DHCP involves raw packages that circumvent iptables. Which is even more scary. A difficult topic.

https://mailman.boum.org/pipermail/tails-dev/2014-December/007553.html

A question I thought that @HulaHoop asked. (Because I confused myself in T239#10445.)

Why not remove the DHCP client on Whonix-Gateway to reduce attack surface?

Good question. Has long not been reconsidered. In past perhaps because of physical isolation, but that would not count anymore nowadays.

Virtualizer specific. Some comments here:

https://github.com/Whonix/whonix-gw-network-conf/blob/master/etc/network/interfaces.d/30_non-qubes-whonix

• VirtualBox: as per comments, seams doable.
• KVM: no idea
• Qubes: does not need this

So if I make it work for VirtualBox, I wouldn't know if the same config would work for KVM. In case of VirtualBox, I don't remember if it caused conflicts with real LAN, other VMs or cloned Whonix-Gateways.

@Patrick:

I don't remember if it caused conflicts with real LAN, other VMs or cloned Whonix-Gateways.

@HulaHoop:

What kind of conflicts?

Let's suppose we no longer had a DHCP client installed and used static IPs.

VirtualBox:

That this is non-trivial, is mentioned here:
https://forums.virtualbox.org/viewtopic.php?f=1&t=49066

• Whonix-Gateway 1 using static IP: 10.0.2.15
• Whonix-Gateway 2 using static IP: 10.0.2.15
• Some non-Whonix VM using NAT, getting IP 10.0.2.15 from VirtualBox DHCP server.

Would they conflict within the VirtualBox NAT network? That has to be tested.

Des VirtualBox static networking work reliable? That has to be tested.

Could you fool the Gateway filtering rules with spoofed IPs?

No. Imagine several physically isolated Whonix-Gateways behind the same physical router. Whonix-Workstations still cannot connect to other Whonix-Gateway which they are not directly connected to or otherwise circumvent them.

The way it is called influences our options to replace it.

Here is some similar info on what invokes it and how to change it:

https://ubuntuforums.org/showthread.php?t=1773210&s=3926512bb7f3f9044dd8ed5b6a553d1f&p=10893136#post10893136

Do you know what happens if no clients listed in ifupdown are installed?

Good question. Has long not been reconsidered. In past perhaps because of physical isolation, but that would not count anymore nowadays.

The effect it has on different hypervisors and how to change it is very hard to know and might cause even more usability friction. I'm for replacing dhclient with something else that we can configure to more sensible defaults.

Choices:

• The pydhcplib package contains the client pydhcp that can be called from commandline to parse DHCP packets the way we would like. Seems to have the ability to alter interfaces on demand. Let me know if this is useful for our purposes:

http://manpages.ubuntu.com/manpages/precise/man8/pydhcp.8.html

"Option up tells pydhcp to set the network interface up if

not.  noup  tells  not  to  set  up the interface. up and noup are only
useful in combination with the device type. Default is noup."

• https://packages.debian.org/jessie/udhcpc

Very barebones busybox implementation. Listed in ifup list as one of the options it cycles thru if dhclient is not available.

In T559#10454, @HulaHoop wrote:

Do you know what happens if no clients listed in ifupdown are installed?

Didn't test. Would speculate, no DHCP, no connectivity then.

The effect it has on different hypervisors and how to change it is very hard to know and might cause even more usability friction. I'm for replacing dhclient with something else that we can configure to more sensible defaults.

Yes, that would be great. Ideally written in a memory safe language. And ideally neither listening for nor sending raw packages.

Far best solution and to be sorted out first however is a static network configuration without any dhcp. Seems less effort to me than reinventing a new dhcp solution.

Confirmed. You are right that it uses raw sockets which do bypass iptables. For the record Shellshock directly affected dhcpclient. I think something this dangerous should be ripped out then questions asked later if it doesn't completely break things.

https://diablohorn.com/2013/11/28/qp-raw-sockets-iptables/

Potential solution is a "derooted DHCPclient that does not need CAP_NET_RAW"

Likely possible with systemd because it can whitelist CAPS. Not sure if it can be run non-root however.

Apparmor can block raw socket access. AFAIK the version in Debian cannot control network access yet?

Its worst than I thought. This behavior is part of an RFC so likely this is how other clients might behave... There is pushback when someone suggested to change this:

https://kb.isc.org/article/AA-00378/0/Why-does-DHCP-use-raw-sockets.html

You can indirectly disable raw sockets by removing the raw cap with an Apparmor profile for dhcpclient.

To check if this measure works show all raw sockets on system:

https://stackoverflow.com/a/17523444

cat /proc/net/raw

Switched to static network configuration.

• https://github.com/Whonix/anon-gw-dhcp-conf/commit/ce5b8e0c4e09f24379a09748524a5fb58812170c
• https://github.com/Whonix/whonix-gw-network-conf/commit/670b928231327fc858e4cc588c5be58d281b4f25

• Works with VirtualBox.
• Likely breaks KVM. A solution for that has to be found. @HulaHoop
• Will make Physical Isolation configuration more difficult.

With libvirt a user can create another NAT network besides the default - with the same IP range. So another GW would have its own dedicated NAT without conflicts.

Can you emulate these changes, use that static IP? What will need changes? KVM documentation?

Can you emulate these changes, use that static IP?

Tested. Not working but thats expected since the ip range on the "default" network supports 192.168.122.2 - 192.168.122.254

What will need changes? KVM documentation?

Yes. I'll have to add steps for creating new NAT networks. I hope we can ship a static IP that works for both instead of adding another network creation step to KVM whonix default GW-WS install pair.

Works with VirtualBox.

By working you mean in multi-GW usecase too?

By working you mean in multi-GW usecase too?

Yes.

So can we move to something static in the 192.168.122.2 - 192.168.122.254 range (depends on VBox choking or not) or should I include another network file with the whonix-libvirt package?

192... will be a huge generator of FUD "conflicts with my router". Long time ago we moved away from that exactly for that reason.

Shipping a network configuration file in whonix-libvirt won't work, since that would break VirtualBox.

Remember https://forums.whonix.org/t/non-qubes-whonix-13-0-0-1-0-x-issues/2443? Relevant summary:

• downloadable builds created by me
• raw image contains both VirtualBox and KVM packages
• therefore it was not possible to ship a KVM specific gui config package and not break VirtualBox at the same time

Same here. Unless you find a way to active that network configuration file when it detects being run inside KVM only. You'd also have to conditionally - at run time - hide whonix-gw-network-conf /etc/network/interfaces.d/30_non-qubes-whonix. Very complex and hacky.

Changing VirtualBox static IP range may be possible, but would break connectivity for all users who do Whonix 13 -> Whonix 14 upgrades and they'd have to run some obscure commands on the host to fix this. Would be a support hell.

My mistake I was not clear. By network configuration I mean yet another XML to create a new separate network as an alternative to "default" (like how I do it now with whonix internal network for KVM). It has nothing to do with GW files at all. No changes have to be made there.

Now my only question is: What is the subnet range you would like me to include for that custom network?

Same as VirtualBox.

https://github.com/Whonix/whonix-gw-network-conf/blob/670b928231327fc858e4cc588c5be58d281b4f25/etc/network/interfaces.d/30_non-qubes-whonix#L27

It has to work if Whonix-Gateway tries to use this static configuration:

auto eth0
iface eth0 inet static
        address 10.0.2.15
        netmask 255.255.255.0
        gateway 10.0.2.2

What I meant was subnet range using the CIDR calculator:

http://jodies.de/ipcalc?host=10.0.0.0&mask1=16&mask2=255.255.255.0

Network: 10.0.2.0/24 00001010.00000000.00000010 .00000000 (Class A)
Broadcast: 10.0.2.255 00001010.00000000.00000010 .11111111
HostMin: 10.0.2.1 00001010.00000000.00000010 .00000001
HostMax: 10.0.2.254 00001010.00000000.00000010 .11111110
Hosts/Net: 254

I created a test network with 10.0.2.0/24 and no matter what I did the GW could not connect.

I enabled broadcasts in interfaces.d, disabled and enabled the test network DHCP with no result. So the current code will completey break KVM Whonix connectivity unfortunately.

The only sane solution IMO is to reinstate dhcp-client while wrapping it in an apparmor profile that forbids raw socket creation by denying the cap. That way iptables can blocks connections to it on internal-net.

I doubt it is possible to successfully use a dhcp client with raw sockets disabled. It may be possible to develop such a thing in theory, but I don't think it exists.

Scratch the "dhcp kvm" search term. The proper search term is "kvm static network configuration". Start with a Debian VM, then port the working solution to Whonix for debugging purposes.

http://serverfault.com/questions/627238/kvm-libvirt-how-to-configure-static-guest-ip-addresses-on-the-virtualisation-ho

http://serverfault.com/questions/627238/kvm-libvirt-how-to-configure-static-guest-ip-addresses-on-the-virtualisation-ho

These steps were not needed at all. Once I selected non-conflicting settings everything worked. Some changes to the netmask and gateway will need to be made to interfaces.d

This works:

address 10.0.2.15
netmask 255.255.252.0
gateway 10.0.0.1

netmask calculator:

http://www.jodies.de/ipcalc?host=10.0.2.15&mask1=255.255.252.0&mask2=

Address:   10.0.2.15             00001010.00000000.000000 10.00001111
Netmask:   255.255.252.0 = 22    11111111.11111111.111111 00.00000000
Wildcard:  0.0.3.255             00000000.00000000.000000 11.11111111
=>
Network:   10.0.0.0/22           00001010.00000000.000000 00.00000000 (Class A)
Broadcast: 10.0.3.255            00001010.00000000.000000 11.11111111
HostMin:   10.0.0.1              00001010.00000000.000000 00.00000001
HostMax:   10.0.3.254            00001010.00000000.000000 11.11111110
Hosts/Net: 1022

I compared it to internal network's netmask to make sure no conflicts:

Address:   10.152.152.10         00001010.10011000.10 011000.00001010
Netmask:   255.255.192.0 = 18    11111111.11111111.11 000000.00000000
Wildcard:  0.0.63.255            00000000.00000000.00 111111.11111111
=>
Network:   10.152.128.0/18       00001010.10011000.10 000000.00000000 (Class A)
Broadcast: 10.152.191.255        00001010.10011000.10 111111.11111111
HostMin:   10.152.128.1          00001010.10011000.10 000000.00000001
HostMax:   10.152.191.254        00001010.10011000.10 111111.11111110
Hosts/Net: 16382                 (Private Internet)

The nat network settings that worked. What other name do you suggest besides "test"?

<network>
  <name>test</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr2' stp='on' delay='0'/>
  <domain name='test'/>
  <ip address='10.0.0.1' netmask='255.255.252.0'>
    <dhcp>
      <range start='10.0.2.0' end='10.0.3.254'/>
    </dhcp>
  </ip>
</network>

Various documentation changes:

Add import steps for new network

Change the side by side instructions to account for static addresses and vibr numbers.

Include a warning somewhere on the multi-WS pages against using the same gateway for different WS trust-levels

In T559#10494, @HulaHoop wrote:

address 10.0.2.15
netmask 255.255.252.0

This works for VirtualBox.

gateway 10.0.0.1

This breaks VirtualBox.

What other name do you suggest besides "test"?

Whonix-Gateway? Perhaps even more specifically, it being for the external network interface.

Looks like libvirt supports a gateway= keyword. Does that work?

Looks like libvirt supports a gateway= keyword. Does that work?

No. Its applicable to static routing networks not NAT, however I will test with another subnet range. It will likely work but the choice of 10.0.2.15 must be changed.

10.0.2.0/24

Works:

address 10.0.2.128
netmask 255.255.255.0
gateway 10.0.2.1


<network>
  <name>nat</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr2' stp='on' delay='0'/>
  <domain name='nat'/>
  <ip address='10.0.2.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.0.2.128' end='10.0.2.254'/>
    </dhcp>
  </ip>

In T559#10500, @HulaHoop wrote:

address 10.0.2.128
netmask 255.255.255.0

Works with VirtualBox.

gateway 10.0.2.1

Changing the gateway IP does not work for VirtualBox.

Then we have reached an impasse because nothing I can put in the network configuration can change the gateway IP. Its not KVM's fault as its the norm to have gateway IPs of x.x.x.1 for a given subnet. Because some idiot on the VBox team chose .2 compatibility is impossible.

The only network type where it can be changed is in static routing but it would require users to change their physical LAN settings to run Whonix - not gonna happen.

A very ugly hack:

Use a realtime network packet editor like netsed to change any outgoing packet ip destination headed for 10.0.2.2 to 10.0.2.1.

netsed uses scapy which uses raw sockets but at least you can bind it to a single interface instead of being forced to have this on all interfaces as with dhcpclient.

https://packages.debian.org/jessie/netsed

Can you redirect these packages using route? (Try in a Debian VM first to exclude Whonix firewall from interfering.)

Can you redirect these packages using route? (Try in a Debian VM first to exclude Whonix firewall from interfering.)

Thought about it some more.

My last idea doesn't make sense since it will do this for both versions so it will still break VBox.

I see no easy solution to this...

We're using ConditionVirtualization=kvm elsewhere already.
(shared-folder-help systemd unit file) Should be doable to reuse it for
the route command also.

We're using ConditionVirtualization=kvm elsewhere already.(shared-folder-help systemd unit file) Should be doable to reuse it for the route command also.

Since this is an option how about shipping a 30_non-qubes-vbox or just 30_vbox and a 30_kvm and have a service that conditiobally activates one or the other based on detecting the vm type - it can do this by deleting the unnecessary file upon boot. This makes sure that even if the package updates reintroduce the file it gets cleaned up again.

The advantage is I can include a static ip config in 30_kvm that just works with the default network without needing yet more steps to setup kvm whonix.

Tested that multiple GWs using the same static ip works. No documentation changes needed for that.

Seems like an awful hack. Last resort. If it somehow by some update (by ifupdown) is run after ifupdown, it breaks connectivity.

What about ConditionVirtualization=kvm + route?

OK I will try route but I need some help with commands.

Is it possible to overwrite everything to be in the 192.168.122.0/24 subnet?

No idea. But we should probably stay on the subnet we have.

Great news! This config works without hacks. You can keep 10.0.2.15 unchanged too. Turns out the gateway ip address was just called "ip address"...

<network>
  <name>external</name>
  <forward mode='nat'/>
  <bridge name='virbr1' stp='on' delay='0'/>
  <ip address='10.0.2.2' netmask='255.255.255.0'/>
</network>

Must change Whonix (internal) to virbr2

Thinking about changing naming of network files to internal/external to be more clear.

subnet range not conflicting:

Address:   10.0.2.0              00001010.00000000.00000010 .00000000
Netmask:   255.255.255.0 = 24    11111111.11111111.11111111 .00000000
Wildcard:  0.0.0.255             00000000.00000000.00000000 .11111111
=>
Network:   10.0.2.0/24           00001010.00000000.00000010 .00000000 (Class A)
Broadcast: 10.0.2.255            00001010.00000000.00000010 .11111111
HostMin:   10.0.2.1              00001010.00000000.00000010 .00000001
HostMax:   10.0.2.254            00001010.00000000.00000010 .11111110
Hosts/Net: 254

That's great! So https://github.com/Whonix/whonix-gw-network-conf/blob/master/etc/network/interfaces.d/30_non-qubes-whonix can stay as is?

But existing KVM users need to change their config after upgrading to Whonix 14?

Yes it can stay as it is.

But existing KVM users need to change their config after upgrading to Whonix 14?

With Whonix 14 there will be no direct upgrade path on KVM at least since the interfaces.d will break connectivity. Also with 64bit compatibility this means the repo paths have changed.

Not a big problem as long as we warn people about it. The inevitable support threads can be redirected to a sticky topic.

Also with 64bit compatibility this means the repo paths have changed.

No such issue. 32 bit users do not have to do anything. Just users who
download Whonix 14 images will be 64 bit users. Users who used Whonix 13
or before that was 32 bit can continue to use that. This is because
Whonix packages are platform _all.

https://forums.whonix.org/t/state-of-offical-64-bit-builds/399/16?u=patrick

At some point however when Debian kills 32 bit, they will have to
download a new image. Not sure if it will be possible to upgrade i386 in
place to amd64 or if that would result in a too huge mess.