Lately I started running Tor on an OpenWrt router (see my previous article). This turned out to be quite a nice experience - Tor on the router has many benefits for Tor-network clients.
Moving Tor from your computer(s) to the router can change the Tor-network landscape. Routers have the most direct Internet connection and they run 24/7. This opens up the opportunity to donate bandwidth to the Tor-network, and while a single router can probably donate little bandwidth there's strength in numbers. In this article I will do some calculations that will hopefully convince you that the potential bandwidth contribution of routers running Tor-firmware is very interesting.
Can many low-bandwidth client routers make a significant contribution to the bandwidth of the Tor-network? To answer this question we must first look at the current bandwidth of the network, and how it is used. Thanks to the Tor metrics project we have data about the bandwidth of the Tor network:
In this graph the bandwidth is the sum of both incoming and outgoing traffic of all relays in the Tor-network. The advertised bandwidth is the sum of the bandwidth each relay claims it can provide. The bandwidth history is the sum of the bandwidth that is reported to have been used.
I think we all agree that the graph shows some impressive numbers. But without understanding the way the Tor-network works you might get the wrong picture. From the clients perspective the bandwidth history is not ~75 Gbit/s but ~12 GBit/s. Why is that? To understand this we need to look at the bandwidth a single client uses:
Client --> OR1 --> OR2 --> OR3 --> Server
One client uses the bandwidth of three onion routers, and for each onion router the client uses both upload and download bandwidth. For a client to upload 1 MB of data, roughly 6 MB of Tor-network bandwidth is needed.
In order to get an idea of the average bandwidth a Tor router can contribute to the Tor-network I use data from Akamai. As you probably know, Akamai is a firm that operates a huge content delivery network. Every quarter they publish a lengthy report with metrics on the observed download speed in different countries. This report contains two metrics of interest: Average connection speed, and Average peak connection speed.
Most publications about the speed of the Internet only mention the average connection speed, which does not represent the connection speed of the users. This has to do with the way Akamai measures it. For Akamai, the average connection speed is the average speed of a http file download. If you request a web-page however, many parallel downloads are taking place which all share the same connection. So, the average speed in the Akamai report is only a fraction of the real speed of the Internet connection. The average peak connection speed only uses the maximum observed download speed of an IP address. This is most likely the speed with no parallel downloads, and thus best represents the speed of the Internet connection.
To get a good idea about the bandwidth of Tor-network users we cannot simply use the average peak Internet connection speed of 32 Mbit/s mentioned on the Akamai website. The Tor metrics show that most Tor users are living in relatively rich countries with good Internet speeds. Therefore I decided to calculate the bandwidth using the data of the 10 counties with the most Tor users.
Top 10 countries: 2016-02-11 to 2016-05-11
Country Mean daily users Avg download speed(Mbit/s)
United States 364015 (19.38 %) 61.5
Russia 214670 (11.43 %) 60.0
Germany 187220 (9.97 %) 51.3
France 112798 (6.01 %) 43.2
United Kingdom 84199 (4.48 %) 56.8
Italy 54166 (2.88 %) 32.0
Spain 53045 (2.82 %) 59.9
Brazil 49302 (2.62 %) 30.3
Japan 47675 (2.54 %) 82.9
Canada 42031 (2.24 %) 54.9
1209121 (64.37 %) 55.6 (weighted)
Weighted Avg download speed:
364015 * 61.5
+ 214670 * 60.0
+ 187220 * 51.3
+ 112798 * 43.2
+ 84199 * 56.8
+ 54166 * 32.0
+ 53045 * 59.9
+ 49302 * 30.3
+ 47675 * 82.9
+ 42031 * 54.9
67191202.8 / 1209121 = 55.6 Mbit/s
Mean daily users all countries: 1209121 / 0.6437 = 18800000
As you can see, taking the global distribution of Tor users into account leads to a much higher average download speed. What I find interesting is that this speed is rapidly improving as the next graph shows:
(Graph made with: Akamai data visualization)
Until now I have only talked about the download speed. What about the upload speed? Most Internet connections are asymmetric having an upload speed which is typically 1/10 of the download speed. This upload speed becomes the limiting factor for the bandwidth a Tor router can contribute to the Tor-network. On average I think one Tor router can contribute 5 Mbit/s in both directions, a total of 10 Mbit/s.
There's one question left to answer: does this bandwidth contribution hurts the normal Internet use of the user? Well, not for downloading because it only uses 1/10 of the download speed. For uploading I looked at my Internet statistics: My peak average daily upload speed was 60 Kbit/s. Many days had a much lower daily average. I think that users who are not doing much uploading will probably not notice the sharing of the upload bandwidth.
Now we have an idea about the average bandwidth contribution of a single Tor router we can start some calculations.
First: How many Tor routers are needed to provide the same advertised bandwidth of the current Tor-network?
170000 Mbit/s / 10 Mbit/s = 17000 routers
17000/1880000 = 0.9% of the Tor users
Second: How many Tor routers are needed to provide the same observed bandwidth of the current Tor-network?
75000 Mbit/s / 10 Mbit/s = 7500 routers
7500/1880000 = 0.4% of the Tor users
Wow, it only takes a very small percentage of Tor users donating bandwidth to provide the same bandwidth that the Tor-network has today. Even better, if the number of Tor users increase so does the donated bandwidth.
Of course, at the moment there is no "Tor router", and the Tor software is not ready to make use of bandwidth donated by clients. I hope this article convinces you that having an official "Tor router" would not only benefit the Tor users, but it will also create new opportunities to expand and strengthen the Tor-network. What do you think?
In my previous article I shared my thoughts on running Tor on the router. I described an ideal Tor router configuration and argued that having Tor on the router benefits both security and usability.
This article is about that ideal Tor router configuration. How did I configure my router, and why did I choose the configuration? The interesting part is that it really is “just configuration”. No programming involved. Even more interesting, it's easy too!
All the configuration is for a GL-AR150 portable router. A very cheap router ($25 on Amazon) that has special Tor firmware that you can install after you bought the device. To be honest, when I bought the device I did not expect much of it. It just seemed too cheap to be more than a toy. Boy was I wrong - after playing with it for some time I realized that it is the most powerful router I have ever owned.
So, what are the specs? (more details here)
Chip: Atheros 9331 Soc at 400MHz
OS: OpenWrt Chaos Calmer 15.05
RAM: 64 MB
ROM: 16 MB
USB: 1*USB 2.0
Ethernet: 2 * 100 Mbps
Wireless: 802.11b/g/n (150 Mbps)
In the default configuration one of the Ethernet ports (eth0) must be connected to the Internet modem (WAN). The second Ethernet port (eth1) and the wireless interface are used for the LAN. Because I was interested in the Tor firmware I have not played much with the factory configuration.
After installing the manufacturers Tor firmware the router has the following configuration:
eth0 - Configured as WAN interface
eth1 - Transparently routes all LAN traffic through the Tor-network
SSID OpenWrt - Acts as a normal LAN wifi interface (no Tor routing)
SSID tor - Transparently routes all LAN traffic through the Tor-network.
To prevent repeating myself I will discus the details of this configuration when I compare it with my own configuration. Let's get started:
My Tor configuration is as follows:
eth0 - Used as WAN interface
eth1 - Limits Internet access to programs using the Tor SOCKS interface
SSID tor_off - Normal, unrestricted LAN wifi interface
SSID tor_medium - Transparently routes all LAN traffic through the Tor-network.
SSID tor_max - Limits Internet access to programs using the Tor SOCKS interface.
As you can see I configured an extra way to connect to the Internet. Eth1 and SSID tor_max only allow programs that are configured to use the Tor SOCKS interface to connect to the Internet. Why have I done this? The reason is that the transparent Tor interface does not offer the best possible privacy protection. With transparent routing many chatty background services are also routed through Tor and their invisible traffic can de-anonymize the user. An example is a mail program that checks the mail every 10 minutes over an unencrypted connection.
The SOCKS-only interfaces do not have a route to the WAN interface, so it's not possible to connect to the Internet directly. To make connected computers aware of this, gateway and name server addresses are not set by the router's DHCP server. Eth1 and the tor_max AP are part of a bridged network. Computers connected to eth1 can access computers connected to the tor_max AP (and vice versa).
One of the things I did not like of the original configuration was the fact that the transparent Tor routing blindly forwarded all TCP traffic to the Tor daemon. This prevented accessing the router's UI and SSH server. My setup only forwards non-local TCP traffic so that the UI and SSH server can always be reached.
I think we all know that the Internet can be dangerous with regard to privacy and security. Spy-ware, Trojans etc. all need the Internet to function. So why do we allow our computers to be connected to the Internet all the time? It would be better if we could choose which programs can use the Internet and when. With this in mind I came to the following wired network configuration:
computer(s) – network switch – eth1[GL-AR150]eth0 – Cable modem – The Internet
As said before, the eth1 interface has no Internet connection. This is not as inconvenient as you might think because my web-browser can still connect to the Internet by using the Tor SOCKS proxy provided by the router.
What if a computer needs an Internet connection? Then there are two options:
First I can plug-in a handy USB wifi adapter and connect to the tor_off access point. The DHCP server on this access point will set the gateway and name server addresses making it possible to use the Internet. If you have an integrated wifi adapter (laptops) things are of course simpler - configure the adapter to use the tor_off access point by default, and only switch on wifi if you need an Internet connection.
The second option would be to use the router's UI to reconfigure eth1 into a normal forwarding configuration. This involves changing the network, DHCP and firewall settings and restarting these services. The disadvantage of this solution is that all computers connected to the switch get full Internet access too.
I did not like the second option. Making it safe would involve MAC filtering so that only one computer at a time gets connected. This can be done but it is complicated. I like simple, so I did not even try to implement the second option.
One big advantage of my setup is that it's now safe to use devices that are designed to spy on you (I mean Android devices of course). I remember that I was quite shocked when I first browsed the Google Play store. Almost all the apps were spy-ware! A simple flashlight app would need access to your contacts, location, history, birth-certificate etc. On my Android tablet I read a lot of books and I am worried that the book reader apps would report on everything I read. Something like: Rob is reading "The hitchhikers guide to the galaxy" and made a note on page 42 about the importance of towels. These worries made me decide to disable wifi on my tablet which is of course a major PITA. With my new setup I connect my tablet to the tor_max access point. This gives apps on the tablet access to to my internal network, but not to the Internet. They cannot send their reports or download helpful adds! Also, by configuring Firefox to use the SOCKS proxy, I can use my tablet for web surfing.
Being a programmer I could not resist to explore how the GL-AR150 could be programmed. This turned out to be quite a disappointment. It was way too easy! No kidding, the OpenWrt people did an excellent job, programming an OpenWrt device is fun!
I enjoyed myself making a small status page (example here) similar to the home page of the original GL-AR150 firmware. The difference is that there is no need to log in to see the router's status. It also displays much faster.
You can access the new status page by surfing to /torbox on the routers IP address:
http://192.168.100.1/torbox (eth1 and tor_max)
Note: torbox is just an internal name I use for my router. I do not intend to release a project/product under the same name.
Installing the status page does not replace the web-UI of the manufacturer. You can still reach it by surfing to the bare IP-address. Remember however that the web-UI of the manufacturer was designed for the manufacturer's configuration. It contains some hard-coded values and is incompatible with my configuration. Choose the more generic LUCI web-UI from the manufacturer's UI if you want to change a setting (note: I find it easier to edit the files in the etc/config/ directory)
I hope you like my configuration enough to give it a try. If you do, remember that I can give you no warranties. Use my configuration at your own risk. As far as I know it's not possible to brick your device. It should always be possible to restore the factory configuration by pressing the reset button for 10 seconds.
My configuration requires the manufacturer's Tor firmware (version 1.3 at the time of writing). You can download and install this firmware by following the instructions in this article: new-tor-firmware-for-gl-ar150-and-gl-inet6416. The firmware link in the article is not the latest version. You can download the latest version at http://www.gl-inet.com/firmware/ar150/tor/
After you installed the Tor firmware you must use its UI to set the board-name (use torbox), root password and timezone of the router. You can access the router's UI by connecting to http://192.168.8.1 (if you are asked for a password use goodlife)
Before installing my configuration it's important to check if the manufacturers Tor firmware works. If not, my configuration will not function too.
Installing the configuration is a simple copy-paste process. Replacing some files and adding others. The file containing the configuration has the same directory structure as the router and can be downloaded here. Inside the archive you find the following directories and files:
Files for changing the network configuration:
A new Tor configuration file:
Files for adding an extra status-page:
After you unpacked the archive you might want to change the wireless passwords in etc/config/wireless. It's best to use a different password for the tor_max interface because this interface has access to your LAN computers/devices. If you want you can also change the ssid's. Do not change the other options!
The best way to copy-paste the files is by making a SFTP connection to the router. When I tried this, it did not work. The dropbear SSH server of the router did not support SFTP. I had to install the openssh-sftp-package to get SFTP working:
opkg install openssh-sftp-server
It is also possible to use the LUCI web-UI of the router to install the package. You can access the web-UI by connection to the OpenWrt wifi access point and then surf to http://192.168.8.1/
On Debian you can now use Nautilus's “File->Connect to server” to establish a SSH connection to the router (use IP address 192.168.8.1). After copy-pasting you have to restart the router for the configuration to take effect. The configuration changes IP addresses and SSID's to the following:
eth1: IP 192.168.100.1 network 192.168.100.0/24
SSID tor_off: IP 172.16.1.1 network 172.16.1.0/24
SSID tor_medium: IP 172.16.50.1 network 172.16.50.0/24
SSID tor_max: IP 192.168.100.1 network 192.168.100.0/24
If you did not change the password in the wireless configuration file, the password for the wifi access points is goodlife.
If you like my setup, I hope you will give it a try. The hardware is very cheap, so for most of you buying the device needed will not be a problem. Here are some links to shops:
The GL-iNet webshop
GL-AR150 on Amazon
Same with external antenna
If you are living in The Netherlands and have no credit-card, you can buy one from Deal Extreme where you can pay with iDeal.
GL-AR150 on Deal Extreme
I recently came across a small travel-router that claimed to provide transparent access to the Tor-network. Just make a wifi connection to its Tor access point and all the TCP and DNS traffic is routed through the Tor-network. For a price of only $25 I could not resist and bought one. Over the last month I have been playing with the device and... it worked flawlessly, it was fun!
So, running the Tor software on cheap router hardware works, but is it a good idea? I believe it is. In this article I will argue that having Tor on the router benefits both security and usability. It opens up new possibilities for expanding the Tor-network and can provide a much needed source of income for the Tor-project.
In this article I will refer to an ideal Tor router. This router will provide 3 wifi access points (AP) with the following SSID's:
- TorOff : This AP does not route through the Tor-network.
- TorTransparent: Provides a transparent Tor proxy.
- TorSocks: Provides access to the Tor SOCKS interface of the router's Tor daemon. Except for DHCP, all other traffic is blocked.
The software running on the router is based on OpenWRT and signed by the Tor-project. It should not be possible to add additional software.
I strongly believe that running the Tor daemon on the router improves security. This is why:
If a user connects to the TorSocks AP instead of a local Tor daemon, it becomes impossible to bypass the Tor-network. Software that is not configured to use Tor will not be able to use the Internet.
The Tor daemon on the router runs in a much more secure environment. On the router just a few carefully selected processes are running. Processes that are controlled by the Tor-project that signed the firmware. Compare this with the Windows environment where there is no control. If Microsoft wants to replace your Tor daemon with its own version it can do so.
While using the TorSocks AP benefits security, the story for the TorTransparent AP is more complicated. The transparent AP routes all TCP and DNS traffic through the Tor-network. This includes traffic from many chatty background services which may leak information that can de-anonymize the user. An example is a mail program that checks the mail every 10 minutes over an unencrypted connection. To protect users who are unaware of this background traffic I think the transparent AP should only allow HTTP and HTTPS by default.
For strong anonymity the user should use special privacy enhanced software like the Tor browser in combination with the TorSocks AP.
Having the Tor daemon running on the router improves usability in the following ways:
- Better maintainability.
- Easier client configuration.
- Better client support.
Running Tor on the router means that Tor does not have to be configured and maintained on every device in ones home. Because the router uses an OS that is controlled by the Tor project, it can automatically update the Tor software (improves security too).
No special client configuration is needed if the TorTransparent AP is used. This means that programs that are not Tor-aware can use the Tor-network to gain a privacy benefit. Of course, it must be made clear to the user that the transparent AP does not provide the best possible privacy protection. For the best privacy protection the user should use the TorSocks AP in combination with privacy enhanced software like Tor-browser.
Playing with Tor router hardware convinced me that its time has come. It can improve both security and usability for the users of the Tor-network. But what can it do for the Tor-project? I think the Tor-project can benefit in the following areas:
- Resistance against parties that can influence or even shutdown the Tor-network.
- A source of independent funding
- New possibilities to strengthen the network and improve the anonymity of the users.
The first point has probably surprised you. Which parties am I talking about? How can they influence or even shutdown the Tor-network? The answers are simple. Most Tor software runs on an OS that is under the control of Google (Android), Microsoft or Apple. These companies have the power to remove or modify the Tor software. Will they do this? Probably not, but remember that risk = chance * effect. The effect of removed or modified Tor software is simply not acceptable which makes it a high risk. Currently the Tor-project has no defense against a hostile act of the OS manufacturers.
One of the weak points of the Tor-project is its funding. Roughly 75 percent comes from grants from US government related institutions. These grants are not stable income, and also make people wonder if the Tor-project is not secretly working for the US. The Tor-project needs to diversify its funding.
I am pretty sure that developing Tor router firmware can become a stable source of income for the Tor-project. Why? Because a there is already a company that sells Tor router hardware. The story of this company is very interesting. They misled their customers about the hardware, claiming it was developed in-house while instead they used of-the-shelf hardware from China. For this their crowdfunding campaign on Kickstarter was suspended. Later analysis of their software revealed some serious security flaws. All this did not finish the company, today they are selling different versions of their AnonaBox for prices ranging from $80 to $120 (of which a generous $1 is donated to the Tor-project!)
The last benefit for the Tor-project that I want to mention are the possibilities to strengthen the network. Think about it; each Tor router adds hardware to the Tor-network that runs 24/7. Hardware that is under the control of the Tor-project. What can you do with this hardware? I don't have all the answers here. In the past I have showed that you can use a router as a Tor-bridge with some simple WAN-NAT iptables rules. Another interesting possibility would be to integrate Tor-browser into the router and access it using a remote desktop connection. This may not be possible today, but it will certainly be possible tomorrow.
I had (and have) a lot of fun playing with my Tor router hardware. Here are some links for those who are interested:
First my device. I bought a GL-Inet GL-AR150
GL-AR150 Tor firmware:
You can buy it from:
GL-AR150 on Amazon
If you are living in The Netherlands and have no creditcard, you can buy one from deal extreme where you can pay with iDeal.
GL-AR150 on Deal Extreme
Note: Hardware with a much faster processor and twice the amount of RAM is coming soon (two weeks?):
This article is about a Linux program called avne, a program that runs other programs inside a virtual network environment. All network traffic from this environment is intercepted and forwarded to the Tor network. Interception is transparent and is guaranteed to be 100% effective.
In order to run a program inside the virtual network environment the user can simply type avne followed by the program name.
avne ip link
avne iceweasel -P avne -no-remote
avne chromium --disable-cache
avne wget https://blog.torproject.org/
Note: Iceweasel = Firefox, renamed for (tm) reasons by Debian
Interception of the network traffic takes place at the IP-level. Because of this, the program started by avne does not have to be Tor-aware. It does not have to have a special Tor configuration and it does not have to be SOCKS compatible. With avne more communications programs can use the Tor network.
AVNE is currently in the alpha phase of development. This means that it is not intended for end users. Alpha versions are for developers and people interested in the technologies behind the program. Although avne works quite well it is not fully developed and tested.
So, what do I hope to achieve with this release?
First of all I want to show you some new technologies that can greatly improve the security and usability of the client to Tor interface. I want to get you thinking: hey this stuff is cool and less difficult than I first thought, maybe now we could.... Please don't be shy to share your ideas!
Second I want your opinion on what should be included in the first beta release. I want avne to be a program to "connect and protect". At the moment I am mainly working on the connect part of the program, but there are plenty of possibilities to add extra protection to the program. Also, connect and protect may not always be compatible. Should I limit the connect part to gain protection?
Last but not least. With this release I hope to get some feedback in the form of bug reports.
Because avne is still in the alpha phase of development there is no install package for the program. Fortunately installation is easy. It involves the following steps:
Download the source code: avne-0.5.tar.gz
Extract the avne-0.5.tar.gz file.
Compile the source code.
gcc -Wall -o avne avne.c
Note: if you get an error about the setns function then simply remove this function from the sourcecode.
Change ownership of the executable to root and set the suid bit.
chown root avne && chmod u+s avne
Make a symbolic link to the executable in /usr/local/bin
ln -s /home/rob/projects/avne/avne /usr/local/bin/avne
AVNE expects Tor to be located at localhost:9050. You can specify another address in the avne.conf file which should be in the same directory as the executable.
There are two ways avne can run a program: it can create a brand new virtual network environment, or use an existing environment. Creating a new network environment is the normal way to start a program. It is as simple as typing:
avne chromium --disable-cache
avne iceweasel -P avne -no-remote
Important: When Iceweasel (Firefox) is already running Iceweasel forks the running process. This forked process runs inside the network environment of its parent and not inside a network environment created by avne. You can prevent this behavior by first closing all running Iceweasel instances, or by starting Iceweasel with another profile (see example above). To create a new profile close all running Iceweasel instances and start Iceweasel with the option -Profilemanager.
Always check if avne is enabled by requesting its status page at IP address 10.10.10.10.
Using an existing virtual network environment is needed for debugging. It allows you to run a second program (for example Wireshark) inside the network environment of another program. If you want to attach to an existing network environment you need the PID of a program that is running inside the network environment. This PID is reported by avne shortly after the program starts.
First start Iceweasel:
avne iceweasel -P avne -no-remote
avne reports the PID of the program running inside the virtual environment:
avne: child pid is 
Use this PID to inject Wireshark inside the same network environment:
avne --use-namespace 1234 wireshark
Note: you can also choose to start Wireshark first and then Iceweasel.
After a program is started you can check if avne is active by requesting its status page located at IP address 10.10.10.10. At the moment the status page displays quite a lot of debugging information. This information is not guaranteed to be correct, development is faster than reporting! The debugging information will be removed in future versions.
During development of a program like avne logging is essential. You can find its logfile at /var/log/avne. Be aware that this logfile is very detailed and can grow quite large.
The avne program uses a combination of two technologies: User Space Networking and lightweight virtualization.
With User Space Networking network protocols like IP, UDP, TCP and others are implemented by code running in User Space. A normal (non-kernel) program can use this to get almost total control of the network traffic. How? Lets give you an example of how avne uses User Space Networking to intercept and forward TCP/IP traffic. If avne receives a TCP SYN packet (connect request) for lets say IP address 126.96.36.199 it sends back a TCP ACK packet (acknowledge) to the communications program. The communications program then thinks it is connected to a server at 188.8.131.52 while in reality it is connected to the avne program. Once the connection with the communications program is fully established avne connects to the Tor network and starts forwarding network traffic.
Running network protocols in User Space requires a network interface that is not connected to the network protocol stack of the kernel. The tuntap device of the Linux kernel can create such an interface. Network interfaces created by a tuntap device are connected to a raw socket in User Space. Depending on the mode of the tuntap device (tun or tap), I/O on the raw socket consists of IP-datagrams (tun mode) or Ethernet packets (tap mode). avne uses the tun mode to process the network traffic of the client program.
At the moment avne implements (=intercepts) two network protocols: TCP/IP and DNS.
One problem that plagues Tor since its start is that even if a program is configured to use its SOCKS interface, there is no guarantee that all network traffic of the program uses the SOCKS interface. The normal non-SOCKS interface is still accessible, and client programs can still use it. Bad client implementations and add-ons have been bypassing the SOCKS interface in the past.
AVNE uses lightweight virtualization to create a network environment with only one network interface, a tun interface that is connected to the avne program. In this network environment the normal system network interface (eth0) is not available. All network traffic is guaranteed to pass through avne which in turn passes it to the Tor network.
The virtualization technology that avne uses is called Linux Kernel Namespaces. Inside the Linux kernel all the important resources are capable of having multiple independent instances. Programs running in User Space use one of the available instances of a resource and are referred to as “running inside a resource name namespace”.
Named after the resource there are several different kernel namespaces:
PID namespace: A PID namespace encapsulates a process tree. Starting a new PID namespace creates a new process tree which only contains processes that are started inside the newly created namespace. The first process running inside a PID namespace has a PID of 1, and acts as the “init-process” of the namespace.
Network namespace: A network namespace represents a completely independent network stack. This includes interfaces, IP addresses, routing and iptables rules. A newly created network namespace is spic and span - not even the lo interface is configured...
Mount namespace: A mount namespace consists of a set of mount points. A new mount namespace starts with a copy of all the mount points at the moment of creation. This copy is independent - mount actions inside the namespace do not affect mounts outside the namespace.
IPC namespace: The IPC namespace isolates the System V interprocess mechanisms like message queues, semaphore sets and shared memory.
UTS namespace: The UTS namespace encapsulates the settings for host name and domain name.
User namespace: A User namespace has its own user and group ID's. These ID's can - and normally do - overlap global ID's.
Kernel Namespaces are a great idea. Lightweight and easy to use! At the moment avne only uses the network and UTS namespaces. Future versions can significantly improve the protection of the user by using additional namespaces. For example: using a mount namespace can present a different rootfs to the program started by avne. One with no access to the home directory (or a fake home directory), different hosts, passwd, hostname files, etc.
The avne program is designed to be extensible. It is easy to add new network protocols, filters, or add support for other overlay networks than Tor. This section gives you a high-level overview of the avne implementation.
AVNE has a simple object orientated design which basically consists of two objects. A connection object takes care of the connection with the client (for example Firefox), and a upstream object connects to a server (Tor). For TCP traffic this looks like:
client <-- fd_tun --> tcp_connection
upstream_connection <-- fd_upstream --> server
(input_buffer == ref output_buffer of tcp_connection)
(output_buffer == ref input_buffer of tcp_connection)
upstream_functions (connect, disconnect, read, write, prepare_events, handle_events)
In the diagram above the tcp and upstream connection share input- and output buffers. Data exchange between the tcp and upstream connection only takes place via these shared buffers. The tcp connection can further use a set of functions to control the upstream connection.
Like all good object orientated design, the tcp and upstream objects know nothing of each-others implementation. This makes the design flexible. For a tcp connection object it does not matter what the upstream object does, as long as the upstream object accepts its input and output buffers and implements a known set of functions. To illustrate this, avne has the following types of upstream connections:
- tcp_upstream_connection: A simple pass-trough connection (Tip: handy for testing!).
- socks5_upstream_connection: Connects to the Tor socks interface.
- http_upstream_connection: Implements an internal status server at IP-address 10.10.10.10:80
Note that this design also makes it possible to chain multiple upstream connections. You can for example put a filter_adds object between the tcp connection and a SOCKS connection.
All I/O is event-driven and uses a simple select loop. Pseudo code for TCP looks like:
clear fdset_read and fdset_write
add fd_tun to fdset_read
for each tcp_connection do
for each tcp_connection do
ack bytes that have been removed from buffer by upstream connection
check if upstream connection changed to closed state
In the io_loop, file descriptors are added to the read and write sets by calling the prepare_events function on the active tcp connections. This function in turn calls the prepare_events of the upstream connection it owns. The interesting part is that a tcp connection does not have a private file descriptor, but shares the file descriptor of the tun interface with all other I/O on the tun interface. So, the prepare_events function of a tcp connection can only add the file descriptor of the owned upstream connection to the read or write sets.
After the select function returns the function handle_events is called for each tcp connection. This function calls the handle_events function of the owned upstream connection and on return checks for changes in the state of the upstream connection. If the upstream connection has written bytes from its output buffer, those bytes are ack-ed to the client. If the upstream connection changed its state to disconnected, the tcp connection starts the TCP disconnect sequence.
When I started my avne project I had some worries about implementing the TCP protocol. Its a complex protocol. This complexity is mostly due to the fact that TCP must work over unreliable connections. Connections that can damage packets, drop packets, duplicate packets, change the sequence of packets etc.
The tun interface that avne uses communicates over a very reliable connection (user space - kernel). With this in mind I decided to do a partial implementation of the TCP protocol. From the testing I have done it seems like this implementation works very well. There are still some loose ends, but I am confident these will be easy to fix.
Implementing DNS support proved to be more interesting than I thought. I ended up implementing two ways to do DNS queries.
The first way only supports DNS resolve queries. AVNE extracts the domain name from an incoming DNS message and passes it to Tor using its SOCKS resolve extension. When Tor has resolved the query, a reply DNS datagram is constructed and sended to the client.
The second implementation uses TCP for DNS queries. DNS supports both the UDP and TCP protocol to submit queries. Because a TCP query only differs from an UDP query by a two byte length field, an UDP query can be copied blindly into a TCP message. This message can then passed to the Tor network using a normal SOCKS TCP connection. The reply is converted back into an UDP reply and sended to the client.
I really like the TCP queries solution. It does support the full DNS protocol and it does not rely on a non-standard SOCKS protocol extension. There are however problems with this solution. For a TCP query you need to specify a DNS server. In my code I use Google Public DNS, which has an IP address of 184.108.40.206. This worked fine for my tests, but it probably will not scale. For a DNS server a TCP query uses considerable more resources than an UDP query. It would not surprise me if the number of simultaneous TCP connections from a client to the DNS server has a limit. If so, this will be a problem for a Tor exit node. When loading a web page you can easily have say 20 simultaneous DNS connections. If 100 users connected to an exit node do the same, the DNS server sees a total of 2000 simultaneous request coming from one IP address. Will it honor these requests? Probably not. For TCP queries to scale the Tor exit node code needs to intercept TCP traffic to port 53 and redirect the query to its own internal DNS resolver, or even better to a full caching name-server on the same machine.
In a normal network environment DNS queries are fast and cheap. If a client does not get a quick reply it assumes the query got lost or was dropped and resubmits the query. The Tor network does not have the fast response that is expected, leading to extra unnecessary queries. To counter this I have implemented a simple client-side DNS cache.
For a quick look at the source code of the avne program you can follow this link: source code of avne.c
This has been a long article so if you are reading this you must be interested in avne and its technologies. I hope you want to give the program a try and if you encounter problems you will report them to me. Ideas are of course welcome too. Please keep in mind that it is alpha software that is not fully developed and tested. Do not use it if you need anonymity!
AVNE is new technology that can significantly enhance usability and security for Tor users. Like all new technology the effect of the technology must first be carefully studied before it is used. This is especially true for technology that is to be used in combination with anonymity software like Tor where a flaw can have serious consequences.
Here are some questions I have:
Currently AVNE works with all TCP client software. Is this safe or should I exclude some types of programs? For example: I think it's a bad idea to allow email clients that do not use encryption. Without encryption the password of the user can be snooped at the exit node. What other programs (protocols) should not be used in combination with Tor?
How much and what type of access should a program running under avne have to the users system? The virtualization technology that is used is very powerful. It can be (and is) used to do full userland virtualization. At the moment only the network and UTS subsystems are virtualized. The next important subsystem to virtualize would be the file system. What files/directories should be visible and what files/directories should be faked?
What should be in the first beta release? For the first beta release I want the network part of the program to be safe to use with Tor. Because proper testing will take time I don't think it would be wise to add too much extra functionalities in the program.
I developed avne with Tor in mind. Therefore I think most technical discussions about avne should take place at the tor-dev mailing list. Hope to see you there!