HOEVENSTEIN

Can the Tor-network benefit from client routers?

Rob van der Hoeven
Fri May 13 2016

Lately I started running Tor on an OpenWrt router (see my previous article). This turned out to be quite a nice experience - Tor on the router has many benefits for Tor-network clients.

Moving Tor from your computer(s) to the router can change the Tor-network landscape. Routers have the most direct Internet connection and they run 24/7. This opens up the opportunity to donate bandwidth to the Tor-network, and while a single router can probably donate little bandwidth there's strength in numbers. In this article I will do some calculations that will hopefully convince you that the potential bandwidth contribution of routers running Tor-firmware is very interesting.

The bandwidth of the Tor network

Can many low-bandwidth client routers make a significant contribution to the bandwidth of the Tor-network? To answer this question we must first look at the current bandwidth of the network, and how it is used. Thanks to the Tor metrics project we have data about the bandwidth of the Tor network:

Total relay bandwidth

In this graph the bandwidth is the sum of both incoming and outgoing traffic of all relays in the Tor-network. The advertised bandwidth is the sum of the bandwidth each relay claims it can provide. The bandwidth history is the sum of the bandwidth that is reported to have been used.

I think we all agree that the graph shows some impressive numbers. But without understanding the way the Tor-network works you might get the wrong picture. From the clients perspective the bandwidth history is not ~75 Gbit/s but ~12 GBit/s. Why is that? To understand this we need to look at the bandwidth a single client uses:

Client --> OR1 --> OR2 --> OR3 --> Server

One client uses the bandwidth of three onion routers, and for each onion router the client uses both upload and download bandwidth. For a client to upload 1 MB of data, roughly 6 MB of Tor-network bandwidth is needed.

Potential Tor router bandwidth

In order to get an idea of the average bandwidth a Tor router can contribute to the Tor-network I use data from Akamai. As you probably know, Akamai is a firm that operates a huge content delivery network. Every quarter they publish a lengthy report with metrics on the observed download speed in different countries. This report contains two metrics of interest: Average connection speed, and Average peak connection speed.

Most publications about the speed of the Internet only mention the average connection speed, which does not represent the connection speed of the users. This has to do with the way Akamai measures it. For Akamai, the average connection speed is the average speed of a http file download. If you request a web-page however, many parallel downloads are taking place which all share the same connection. So, the average speed in the Akamai report is only a fraction of the real speed of the Internet connection. The average peak connection speed only uses the maximum observed download speed of an IP address. This is most likely the speed with no parallel downloads, and thus best represents the speed of the Internet connection.

To get a good idea about the bandwidth of Tor-network users we cannot simply use the average peak Internet connection speed of 32 Mbit/s mentioned on the Akamai website. The Tor metrics show that most Tor users are living in relatively rich countries with good Internet speeds. Therefore I decided to calculate the bandwidth using the data of the 10 counties with the most Tor users.

Top 10 countries: 2016-02-11 to 2016-05-11

Country         Mean daily users     Avg download speed(Mbit/s)
=================================================
United States   364015  (19.38 %)    61.5
Russia          214670  (11.43 %)    60.0
Germany         187220   (9.97 %)    51.3
France          112798   (6.01 %)    43.2
United Kingdom   84199   (4.48 %)    56.8
Italy            54166   (2.88 %)    32.0
Spain            53045   (2.82 %)    59.9
Brazil           49302   (2.62 %)    30.3
Japan            47675   (2.54 %)    82.9
Canada           42031   (2.24 %)    54.9
=================================================
               1209121  (64.37 %)    55.6 (weighted)

Weighted Avg download speed:

   364015 * 61.5 
 + 214670 * 60.0 
 + 187220 * 51.3 
 + 112798 * 43.2 
 +  84199 * 56.8 
 +  54166 * 32.0 
 +  53045 * 59.9
 +  49302 * 30.3
 +  47675 * 82.9
 +  42031 * 54.9
   =============
 67191202.8 / 1209121 = 55.6 Mbit/s

Mean daily users all countries: 1209121 / 0.6437 = 18800000

As you can see, taking the global distribution of Tor users into account leads to a much higher average download speed. What I find interesting is that this speed is rapidly improving as the next graph shows:

Average peak connection speed of Tor top 10 countries

(Graph made with: Akamai data visualization)

Until now I have only talked about the download speed. What about the upload speed? Most Internet connections are asymmetric having an upload speed which is typically 1/10 of the download speed. This upload speed becomes the limiting factor for the bandwidth a Tor router can contribute to the Tor-network. On average I think one Tor router can contribute 5 Mbit/s in both directions, a total of 10 Mbit/s.

There's one question left to answer: does this bandwidth contribution hurts the normal Internet use of the user? Well, not for downloading because it only uses 1/10 of the download speed. For uploading I looked at my Internet statistics: My peak average daily upload speed was 60 Kbit/s. Many days had a much lower daily average. I think that users who are not doing much uploading will probably not notice the sharing of the upload bandwidth.

Some calculations

Now we have an idea about the average bandwidth contribution of a single Tor router we can start some calculations.

First: How many Tor routers are needed to provide the same advertised bandwidth of the current Tor-network?

170000 Mbit/s / 10 Mbit/s = 17000 routers 
17000/1880000 = 0.9% of the Tor users

Second: How many Tor routers are needed to provide the same observed bandwidth of the current Tor-network?

75000 Mbit/s / 10 Mbit/s = 7500 routers
7500/1880000 = 0.4% of the Tor users

Wow, it only takes a very small percentage of Tor users donating bandwidth to provide the same bandwidth that the Tor-network has today. Even better, if the number of Tor users increase so does the donated bandwidth.

Of course, at the moment there is no "Tor router", and the Tor software is not ready to make use of bandwidth donated by clients. I hope this article convinces you that having an official "Tor router" would not only benefit the Tor users, but it will also create new opportunities to expand and strengthen the Tor-network. What do you think?

Comments: 0

My OpenWrt Tor configuration

Rob van der Hoeven
Thu Apr 21 2016

In my previous article I shared my thoughts on running Tor on the router. I described an ideal Tor router configuration and argued that having Tor on the router benefits both security and usability.

This article is about that ideal Tor router configuration. How did I configure my router, and why did I choose the configuration? The interesting part is that it really is “just configuration”. No programming involved. Even more interesting, it's easy too!

The router

All the configuration is for a GL-AR150 portable router. A very cheap router ($25 on Amazon) that has special Tor firmware that you can install after you bought the device. To be honest, when I bought the device I did not expect much of it. It just seemed too cheap to be more than a toy. Boy was I wrong - after playing with it for some time I realized that it is the most powerful router I have ever owned.

So, what are the specs? (more details here)

Chip:     Atheros 9331 Soc at 400MHz
OS:       OpenWrt Chaos Calmer 15.05
RAM:      64 MB
ROM:      16 MB
USB:      1*USB 2.0
Ethernet: 2 * 100 Mbps 
Wireless: 802.11b/g/n (150 Mbps)

In the default configuration one of the Ethernet ports (eth0) must be connected to the Internet modem (WAN). The second Ethernet port (eth1) and the wireless interface are used for the LAN. Because I was interested in the Tor firmware I have not played much with the factory configuration.

Original Tor configuration

After installing the manufacturers Tor firmware the router has the following configuration:

Wired:

eth0 - Configured as WAN interface
eth1 - Transparently routes all LAN traffic through the Tor-network

Wireless:

SSID OpenWrt - Acts as a normal LAN wifi interface (no Tor routing)
SSID tor     - Transparently routes all LAN traffic through the Tor-network.

To prevent repeating myself I will discus the details of this configuration when I compare it with my own configuration. Let's get started:

My Tor configuration

My Tor configuration is as follows:

Wired:

eth0 - Used as WAN interface
eth1 - Limits Internet access to programs using the Tor SOCKS interface

Wireless:

SSID tor_off    - Normal, unrestricted LAN wifi interface
SSID tor_medium - Transparently routes all LAN traffic through the Tor-network.
SSID tor_max    - Limits Internet access to programs using the Tor SOCKS interface.

As you can see I configured an extra way to connect to the Internet. Eth1 and SSID tor_max only allow programs that are configured to use the Tor SOCKS interface to connect to the Internet. Why have I done this? The reason is that the transparent Tor interface does not offer the best possible privacy protection. With transparent routing many chatty background services are also routed through Tor and their invisible traffic can de-anonymize the user. An example is a mail program that checks the mail every 10 minutes over an unencrypted connection.

The SOCKS-only interfaces do not have a route to the WAN interface, so it's not possible to connect to the Internet directly. To make connected computers aware of this, gateway and name server addresses are not set by the router's DHCP server. Eth1 and the tor_max AP are part of a bridged network. Computers connected to eth1 can access computers connected to the tor_max AP (and vice versa).

One of the things I did not like of the original configuration was the fact that the transparent Tor routing blindly forwarded all TCP traffic to the Tor daemon. This prevented accessing the router's UI and SSH server. My setup only forwards non-local TCP traffic so that the UI and SSH server can always be reached.

Usage scenario

I think we all know that the Internet can be dangerous with regard to privacy and security. Spy-ware, Trojans etc. all need the Internet to function. So why do we allow our computers to be connected to the Internet all the time? It would be better if we could choose which programs can use the Internet and when. With this in mind I came to the following wired network configuration:

computer(s) – network switch – eth1[GL-AR150]eth0 – Cable modem – The Internet

As said before, the eth1 interface has no Internet connection. This is not as inconvenient as you might think because my web-browser can still connect to the Internet by using the Tor SOCKS proxy provided by the router.

What if a computer needs an Internet connection? Then there are two options:

I did not like the second option. Making it safe would involve MAC filtering so that only one computer at a time gets connected. This can be done but it is complicated. I like simple, so I did not even try to implement the second option.

One big advantage of my setup is that it's now safe to use devices that are designed to spy on you (I mean Android devices of course). I remember that I was quite shocked when I first browsed the Google Play store. Almost all the apps were spy-ware! A simple flashlight app would need access to your contacts, location, history, birth-certificate etc. On my Android tablet I read a lot of books and I am worried that the book reader apps would report on everything I read. Something like: Rob is reading "The hitchhikers guide to the galaxy" and made a note on page 42 about the importance of towels. These worries made me decide to disable wifi on my tablet which is of course a major PITA. With my new setup I connect my tablet to the tor_max access point. This gives apps on the tablet access to to my internal network, but not to the Internet. They cannot send their reports or download helpful adds! Also, by configuring Firefox to use the SOCKS proxy, I can use my tablet for web surfing.

A new status page

Being a programmer I could not resist to explore how the GL-AR150 could be programmed. This turned out to be quite a disappointment. It was way too easy! No kidding, the OpenWrt people did an excellent job, programming an OpenWrt device is fun!

I enjoyed myself making a small status page (example here) similar to the home page of the original GL-AR150 firmware. The difference is that there is no need to log in to see the router's status. It also displays much faster.

You can access the new status page by surfing to /torbox on the routers IP address:

http://192.168.100.1/torbox   (eth1 and tor_max) 
http://172.16.1.1/torbox      (tor_off)
http://172.16.50.1/torbox     (tor_medium)

Note: torbox is just an internal name I use for my router. I do not intend to release a project/product under the same name.

Installing the status page does not replace the web-UI of the manufacturer. You can still reach it by surfing to the bare IP-address. Remember however that the web-UI of the manufacturer was designed for the manufacturer's configuration. It contains some hard-coded values and is incompatible with my configuration. Choose the more generic LUCI web-UI from the manufacturer's UI if you want to change a setting (note: I find it easier to edit the files in the etc/config/ directory)

Installing the configuration

I hope you like my configuration enough to give it a try. If you do, remember that I can give you no warranties. Use my configuration at your own risk. As far as I know it's not possible to brick your device. It should always be possible to restore the factory configuration by pressing the reset button for 10 seconds.

My configuration requires the manufacturer's Tor firmware (version 1.3 at the time of writing). You can download and install this firmware by following the instructions in this article: new-tor-firmware-for-gl-ar150-and-gl-inet6416. The firmware link in the article is not the latest version. You can download the latest version at http://www.gl-inet.com/firmware/ar150/tor/

After you installed the Tor firmware you must use its UI to set the board-name (use torbox), root password and timezone of the router. You can access the router's UI by connecting to http://192.168.8.1 (if you are asked for a password use goodlife)

Before installing my configuration it's important to check if the manufacturers Tor firmware works. If not, my configuration will not function too.

Installing the configuration is a simple copy-paste process. Replacing some files and adding others. The file containing the configuration has the same directory structure as the router and can be downloaded here. Inside the archive you find the following directories and files:

Files for changing the network configuration:

etc/config/dhcp
etc/config/firewall
etc/config/network
etc/config/wireless

A new Tor configuration file:

etc/tor/torrc

Files for adding an extra status-page:

etc/config/uhttpd
www/torbox.css
www/torbox.lua
www/bumblebee.jpg

After you unpacked the archive you might want to change the wireless passwords in etc/config/wireless. It's best to use a different password for the tor_max interface because this interface has access to your LAN computers/devices. If you want you can also change the ssid's. Do not change the other options!

The best way to copy-paste the files is by making a SFTP connection to the router. When I tried this, it did not work. The dropbear SSH server of the router did not support SFTP. I had to install the openssh-sftp-package to get SFTP working:

ssh root@192.168.8.1

opkg update
opkg install openssh-sftp-server

It is also possible to use the LUCI web-UI of the router to install the package. You can access the web-UI by connection to the OpenWrt wifi access point and then surf to http://192.168.8.1/

On Debian you can now use Nautilus's “File->Connect to server” to establish a SSH connection to the router (use IP address 192.168.8.1). After copy-pasting you have to restart the router for the configuration to take effect. The configuration changes IP addresses and SSID's to the following:

eth1:              IP 192.168.100.1   network 192.168.100.0/24
SSID tor_off:      IP 172.16.1.1      network 172.16.1.0/24
SSID tor_medium:   IP 172.16.50.1     network 172.16.50.0/24
SSID tor_max:      IP 192.168.100.1   network 192.168.100.0/24

If you did not change the password in the wireless configuration file, the password for the wifi access points is goodlife.

Give it a try!

If you like my setup, I hope you will give it a try. The hardware is very cheap, so for most of you buying the device needed will not be a problem. Here are some links to shops:

The GL-iNet webshop

GL-AR150 on Amazon
Same with external antenna

If you are living in The Netherlands and have no credit-card, you can buy one from Deal Extreme where you can pay with iDeal.

GL-AR150 on Deal Extreme

Comments: 0

Thoughts on Tor router hardware

Rob van der Hoeven
Tue Feb 23 2016

I recently came across a small travel-router that claimed to provide transparent access to the Tor-network. Just make a wifi connection to its Tor access point and all the TCP and DNS traffic is routed through the Tor-network. For a price of only $25 I could not resist and bought one. Over the last month I have been playing with the device and... it worked flawlessly, it was fun!

So, running the Tor software on cheap router hardware works, but is it a good idea? I believe it is. In this article I will argue that having Tor on the router benefits both security and usability. It opens up new possibilities for expanding the Tor-network and can provide a much needed source of income for the Tor-project.

Setting the stage

In this article I will refer to an ideal Tor router. This router will provide 3 wifi access points (AP) with the following SSID's:

The software running on the router is based on OpenWRT and signed by the Tor-project. It should not be possible to add additional software.

Security

I strongly believe that running the Tor daemon on the router improves security. This is why:

For strong anonymity the user should use special privacy enhanced software like the Tor browser in combination with the TorSocks AP.

Usability

Having the Tor daemon running on the router improves usability in the following ways:

Running Tor on the router means that Tor does not have to be configured and maintained on every device in ones home. Because the router uses an OS that is controlled by the Tor project, it can automatically update the Tor software (improves security too).

No special client configuration is needed if the TorTransparent AP is used. This means that programs that are not Tor-aware can use the Tor-network to gain a privacy benefit. Of course, it must be made clear to the user that the transparent AP does not provide the best possible privacy protection. For the best privacy protection the user should use the TorSocks AP in combination with privacy enhanced software like Tor-browser.

How the Tor-project can benefit

Playing with Tor router hardware convinced me that its time has come. It can improve both security and usability for the users of the Tor-network. But what can it do for the Tor-project? I think the Tor-project can benefit in the following areas:

The first point has probably surprised you. Which parties am I talking about? How can they influence or even shutdown the Tor-network? The answers are simple. Most Tor software runs on an OS that is under the control of Google (Android), Microsoft or Apple. These companies have the power to remove or modify the Tor software. Will they do this? Probably not, but remember that risk = chance * effect. The effect of removed or modified Tor software is simply not acceptable which makes it a high risk. Currently the Tor-project has no defense against a hostile act of the OS manufacturers.

One of the weak points of the Tor-project is its funding. Roughly 75 percent comes from grants from US government related institutions. These grants are not stable income, and also make people wonder if the Tor-project is not secretly working for the US. The Tor-project needs to diversify its funding.

I am pretty sure that developing Tor router firmware can become a stable source of income for the Tor-project. Why? Because a there is already a company that sells Tor router hardware. The story of this company is very interesting. They misled their customers about the hardware, claiming it was developed in-house while instead they used of-the-shelf hardware from China. For this their crowdfunding campaign on Kickstarter was suspended. Later analysis of their software revealed some serious security flaws. All this did not finish the company, today they are selling different versions of their AnonaBox for prices ranging from $80 to $120 (of which a generous $1 is donated to the Tor-project!)

The last benefit for the Tor-project that I want to mention are the possibilities to strengthen the network. Think about it; each Tor router adds hardware to the Tor-network that runs 24/7. Hardware that is under the control of the Tor-project. What can you do with this hardware? I don't have all the answers here. In the past I have showed that you can use a router as a Tor-bridge with some simple WAN-NAT iptables rules. Another interesting possibility would be to integrate Tor-browser into the router and access it using a remote desktop connection. This may not be possible today, but it will certainly be possible tomorrow.

Playtime

I had (and have) a lot of fun playing with my Tor router hardware. Here are some links for those who are interested:

First my device. I bought a GL-Inet GL-AR150

GL-AR150 Specs
GL-AR150 Tor firmware:

You can buy it from:

GL-AR150 on Amazon

If you are living in The Netherlands and have no creditcard, you can buy one from deal extreme where you can pay with iDeal.

GL-AR150 on Deal Extreme

Note: Hardware with a much faster processor and twice the amount of RAM is coming soon (two weeks?):

GL-MT300A Specs

Comments: 0

Another Virtual Network Environment

Rob van der Hoeven
Wed Oct 14 2015

This article is about a Linux program called avne, a program that runs other programs inside a virtual network environment. All network traffic from this environment is intercepted and forwarded to the Tor network. Interception is transparent and is guaranteed to be 100% effective.

In order to run a program inside the virtual network environment the user can simply type avne followed by the program name.

Examples:

    avne bash
    avne ip link
    avne iceweasel -P avne -no-remote
    avne chromium --disable-cache
    avne wget https://blog.torproject.org/

Note: Iceweasel = Firefox, renamed for (tm) reasons by Debian

Interception of the network traffic takes place at the IP-level. Because of this, the program started by avne does not have to be Tor-aware. It does not have to have a special Tor configuration and it does not have to be SOCKS compatible. With avne more communications programs can use the Tor network.

AVNE is currently in the alpha phase of development. This means that it is not intended for end users. Alpha versions are for developers and people interested in the technologies behind the program. Although avne works quite well it is not fully developed and tested.

So, what do I hope to achieve with this release?

Installation

Because avne is still in the alpha phase of development there is no install package for the program. Fortunately installation is easy. It involves the following steps:

  1. Download the source code: avne-0.5.tar.gz

  2. Extract the avne-0.5.tar.gz file.

    tar xfvz avne-0.5.tar.gz
    
  3. Compile the source code.

    gcc -Wall -o avne avne.c
    
    Note: if you get an error about the setns function then simply remove this function from the sourcecode.
    
  4. Change ownership of the executable to root and set the suid bit.

    chown root avne && chmod u+s avne
    
  5. Make a symbolic link to the executable in /usr/local/bin

    ln -s /home/rob/projects/avne/avne /usr/local/bin/avne
    

AVNE expects Tor to be located at localhost:9050. You can specify another address in the avne.conf file which should be in the same directory as the executable.

Usage

There are two ways avne can run a program: it can create a brand new virtual network environment, or use an existing environment. Creating a new network environment is the normal way to start a program. It is as simple as typing:

avne chromium --disable-cache
avne iceweasel -P avne -no-remote

Important: When Iceweasel (Firefox) is already running Iceweasel forks the running process. This forked process runs inside the network environment of its parent and not inside a network environment created by avne. You can prevent this behavior by first closing all running Iceweasel instances, or by starting Iceweasel with another profile (see example above). To create a new profile close all running Iceweasel instances and start Iceweasel with the option -Profilemanager.

Always check if avne is enabled by requesting its status page at IP address 10.10.10.10.

Using an existing virtual network environment is needed for debugging. It allows you to run a second program (for example Wireshark) inside the network environment of another program. If you want to attach to an existing network environment you need the PID of a program that is running inside the network environment. This PID is reported by avne shortly after the program starts.

Example:

First start Iceweasel:

    avne iceweasel -P avne -no-remote

avne reports the PID of the program running inside the virtual environment:

    avne: child pid is [1234]

Use this PID to inject Wireshark inside the same network environment:

    avne --use-namespace 1234 wireshark

Note: you can also choose to start Wireshark first and then Iceweasel.

After a program is started you can check if avne is active by requesting its status page located at IP address 10.10.10.10. At the moment the status page displays quite a lot of debugging information. This information is not guaranteed to be correct, development is faster than reporting! The debugging information will be removed in future versions.

During development of a program like avne logging is essential. You can find its logfile at /var/log/avne. Be aware that this logfile is very detailed and can grow quite large.

Technology

The avne program uses a combination of two technologies: User Space Networking and lightweight virtualization.

User Space Networking

With User Space Networking network protocols like IP, UDP, TCP and others are implemented by code running in User Space. A normal (non-kernel) program can use this to get almost total control of the network traffic. How? Lets give you an example of how avne uses User Space Networking to intercept and forward TCP/IP traffic. If avne receives a TCP SYN packet (connect request) for lets say IP address 1.2.3.4 it sends back a TCP ACK packet (acknowledge) to the communications program. The communications program then thinks it is connected to a server at 1.2.3.4 while in reality it is connected to the avne program. Once the connection with the communications program is fully established avne connects to the Tor network and starts forwarding network traffic.

Running network protocols in User Space requires a network interface that is not connected to the network protocol stack of the kernel. The tuntap device of the Linux kernel can create such an interface. Network interfaces created by a tuntap device are connected to a raw socket in User Space. Depending on the mode of the tuntap device (tun or tap), I/O on the raw socket consists of IP-datagrams (tun mode) or Ethernet packets (tap mode). avne uses the tun mode to process the network traffic of the client program.

At the moment avne implements (=intercepts) two network protocols: TCP/IP and DNS.

Lightweight virtualization

One problem that plagues Tor since its start is that even if a program is configured to use its SOCKS interface, there is no guarantee that all network traffic of the program uses the SOCKS interface. The normal non-SOCKS interface is still accessible, and client programs can still use it. Bad client implementations and add-ons have been bypassing the SOCKS interface in the past.

AVNE uses lightweight virtualization to create a network environment with only one network interface, a tun interface that is connected to the avne program. In this network environment the normal system network interface (eth0) is not available. All network traffic is guaranteed to pass through avne which in turn passes it to the Tor network.

The virtualization technology that avne uses is called Linux Kernel Namespaces. Inside the Linux kernel all the important resources are capable of having multiple independent instances. Programs running in User Space use one of the available instances of a resource and are referred to as “running inside a resource name namespace”.

Named after the resource there are several different kernel namespaces:

Kernel Namespaces are a great idea. Lightweight and easy to use! At the moment avne only uses the network and UTS namespaces. Future versions can significantly improve the protection of the user by using additional namespaces. For example: using a mount namespace can present a different rootfs to the program started by avne. One with no access to the home directory (or a fake home directory), different hosts, passwd, hostname files, etc.

Implementation

The avne program is designed to be extensible. It is easy to add new network protocols, filters, or add support for other overlay networks than Tor. This section gives you a high-level overview of the avne implementation.

Data flow

AVNE has a simple object orientated design which basically consists of two objects. A connection object takes care of the connection with the client (for example Firefox), and a upstream object connects to a server (Tor). For TCP traffic this looks like:

client <-- fd_tun --> tcp_connection
                          input_buffer
                          output_buffer
                          upstream_connection <-- fd_upstream --> server
                              (input_buffer == ref output_buffer of tcp_connection)
                              (output_buffer == ref input_buffer of tcp_connection)
                              upstream_functions (connect, disconnect, read, write, prepare_events, handle_events)

In the diagram above the tcp and upstream connection share input- and output buffers. Data exchange between the tcp and upstream connection only takes place via these shared buffers. The tcp connection can further use a set of functions to control the upstream connection.

Like all good object orientated design, the tcp and upstream objects know nothing of each-others implementation. This makes the design flexible. For a tcp connection object it does not matter what the upstream object does, as long as the upstream object accepts its input and output buffers and implements a known set of functions. To illustrate this, avne has the following types of upstream connections:

Note that this design also makes it possible to chain multiple upstream connections. You can for example put a filter_adds object between the tcp connection and a SOCKS connection.

Event handling

All I/O is event-driven and uses a simple select loop. Pseudo code for TCP looks like:

io_loop()
    clear fdset_read and fdset_write
    add fd_tun to fdset_read

    while client_running
        for each tcp_connection do
            tcp_connection_prepare_events(tcp_connection)

        select(fdset_read,fdset_write)

        for each tcp_connection do
            tcp_connection_handle_events(tcp_connection)

        tun_handle_events()

tcp_connection_prepare_events(tcp_connection)
    call tcp_connection->upstream.functions->prepare_events(tcp_connection->upstream)

tcp_connection_handle_events(tcp_connection)
    call tcp_connection->upstream.functions->handle_events(tcp_connection->upstream)

    ack bytes that have been removed from buffer by upstream connection
    check if upstream connection changed to closed state

In the io_loop, file descriptors are added to the read and write sets by calling the prepare_events function on the active tcp connections. This function in turn calls the prepare_events of the upstream connection it owns. The interesting part is that a tcp connection does not have a private file descriptor, but shares the file descriptor of the tun interface with all other I/O on the tun interface. So, the prepare_events function of a tcp connection can only add the file descriptor of the owned upstream connection to the read or write sets.

After the select function returns the function handle_events is called for each tcp connection. This function calls the handle_events function of the owned upstream connection and on return checks for changes in the state of the upstream connection. If the upstream connection has written bytes from its output buffer, those bytes are ack-ed to the client. If the upstream connection changed its state to disconnected, the tcp connection starts the TCP disconnect sequence.

Tcp protocol implementation

When I started my avne project I had some worries about implementing the TCP protocol. Its a complex protocol. This complexity is mostly due to the fact that TCP must work over unreliable connections. Connections that can damage packets, drop packets, duplicate packets, change the sequence of packets etc.

The tun interface that avne uses communicates over a very reliable connection (user space - kernel). With this in mind I decided to do a partial implementation of the TCP protocol. From the testing I have done it seems like this implementation works very well. There are still some loose ends, but I am confident these will be easy to fix.

Dns protocol implementation

Implementing DNS support proved to be more interesting than I thought. I ended up implementing two ways to do DNS queries.

I really like the TCP queries solution. It does support the full DNS protocol and it does not rely on a non-standard SOCKS protocol extension. There are however problems with this solution. For a TCP query you need to specify a DNS server. In my code I use Google Public DNS, which has an IP address of 8.8.8.8. This worked fine for my tests, but it probably will not scale. For a DNS server a TCP query uses considerable more resources than an UDP query. It would not surprise me if the number of simultaneous TCP connections from a client to the DNS server has a limit. If so, this will be a problem for a Tor exit node. When loading a web page you can easily have say 20 simultaneous DNS connections. If 100 users connected to an exit node do the same, the DNS server sees a total of 2000 simultaneous request coming from one IP address. Will it honor these requests? Probably not. For TCP queries to scale the Tor exit node code needs to intercept TCP traffic to port 53 and redirect the query to its own internal DNS resolver, or even better to a full caching name-server on the same machine.

In a normal network environment DNS queries are fast and cheap. If a client does not get a quick reply it assumes the query got lost or was dropped and resubmits the query. The Tor network does not have the fast response that is expected, leading to extra unnecessary queries. To counter this I have implemented a simple client-side DNS cache.

Source code

For a quick look at the source code of the avne program you can follow this link: source code of avne.c

AVNE needs your I/O

This has been a long article so if you are reading this you must be interested in avne and its technologies. I hope you want to give the program a try and if you encounter problems you will report them to me. Ideas are of course welcome too. Please keep in mind that it is alpha software that is not fully developed and tested. Do not use it if you need anonymity!

AVNE is new technology that can significantly enhance usability and security for Tor users. Like all new technology the effect of the technology must first be carefully studied before it is used. This is especially true for technology that is to be used in combination with anonymity software like Tor where a flaw can have serious consequences.

Here are some questions I have:

I developed avne with Tor in mind. Therefore I think most technical discussions about avne should take place at the tor-dev mailing list. Hope to see you there!

Comments: 0