Universal Relay

Universal Relay can be used as a gateway for an arbitrarily complex network topology, separated by a packet barrier.

Universal Relay is a multi-use SOCKS and transparent TCP proxy server. Prominent features of Universal Relay are as follows:

  • Tunnel TCP->TCP, SOCKS->TCP, TCP->SOCKS, and SOCKS->SOCKS
  • SOCKS server supports version 4, 4a, and 5. SOCKS client supports version 5.
  • Thanks to the internal IP to domain mapping with the custom programmable DNS server, TCP->SOCKS can reliably preserve the originally requested domain even where there should not be an IP address associated with the domain, such as with the Tor client's SOCKS server for .onion domains
  • Highly customizable programmatic DNS server for other purposes (implemented as a PowerDNS backend)
  • Transparent Happy Eyeballs
  • IP address and domain name matching and rewriting facility: can recognize IP prefixes or domain name suffixes, and extract lower components from matching domain names or IP addresses to convert to new domain names or IP addresses, select alternative paths or upstream interfaces, and/or outright block them
  • IPv6-only "private side", which isolates designated clients and controls their Internet connection in a programmatic manner using the matching and rewriting facility
  • IPv6-only features on the private-side network can still be used even if there is no upstream IPv6 connectivity or if the destination website does not support IPv6
  • Private side network can have conflicting IP ranges with the Internet side without loss of reachability
  • Private side and Internet side can be in different VRFs or network namespaces (requires ctrtool ns_open_file)
  • Can select path used to connect to the Internet (requires Socket Enhancer; see also Universal Relay/napi-socket)
  • Transparent proxy can live "off-path" (i.e. not on the default route, or on the path of the default gateway)

History

In July and August 2020, I registered an ASN and provider independent IPv4 and IPv6 addresses at the American Registry for Internet Numbers. This was particularly useful for me, as it allowed me to realize many of the advantages that comes with having static, routed IPv6 prefixes, such as reverse DNS, the guaranteed ability to realize Snippets:Nginx geo local server address (required for things like IPv6 Things and Socketbox), and the ability to create a network with multiple downstream routers using static routes and/or prefix delegation.

However, what I did not like about this was that I had to spend a lot of money doing all of this. The ARIN fees already cost me over one thousand dollars. I also tried to sign up for various business Internet services, but either they did not serve my home (which was in a residential location, not a commercial one, due to COVID), or they were way too expensive for me as an individual. This was certainly not very nice if I wanted to talk about the advantages of IPv6 and allow others to realize those advantages too.

I had already set up a network using the BGP addresses at home using a VPN tunnel. This allowed me to use the BGP addresses directly on the network, but it was overall not very nice, simply because of inefficiencies with using the VPN tunnel in circumstances where the ISP addresses would already suffice, as well as potential privacy issues (due to the ARIN whois).

This caused a rather interesting dilemma: should I use BGP addresses, which has the advantage of being larger and static, at the expense of privacy issues and inefficiencies with using a VPN tunnel, or should I use the ISP addresses, which are more efficient to use, but are dynamic and smaller? To have the best of both worlds, I used IPv6 NAT on my main router to translate the static 2001:db8:XXXX::/48 addresses that I numbered my network with to the dynamic ISP addresses, using the ip6tables NETMAP target described in Linux Networking Primitives.

Eventually, I considered switching to the other ISP I had in my area, which did not have IPv6 the last time I used it several years before this consideration. This was very problematic since that would effectively nullify any advantage that I previously had with using IPv6 addresses on my home network. I was not willing to use my BGP addresses in such a situation.

This ultimately meant that in order for a network with IPv6 enabled to be sustainable, I had to have an IPv6-enabled ISP. Changing ISPs would therefore be a much bigger ordeal compared to the average consumer since IPv6-only subnets on my network would have to go back to having IPv4 addresses, which ends up creating a bottleneck from the address space being very small compared to IPv6. This was certainly not acceptable for me, as doing so severely limited the ability to have IPv6-only subnets.

You might be wondering, "why not just use a Hurricane Electric tunnel?" Well, there are people on the Internet with complaints about those tunnels, such as Netflix being blocked, peering issues with Cogent, not being compatible with CGNAT, the reputation of the addresses being less favorable compared to ISP addresses, and no longer offering free BGP tunnels.

These issues, especially the address reputation issue, ultimately led me to the big problem of wanting to have an IPv6 network with an ISP that does not have IPv6, but I cannot use tunnel or VPN services.

And so Universal Relay was created to realize the advantages of IPv6 without requiring anything other than your normal IPv4-only, dual-stack, or IPv6-only Internet connection. If you only have IPv4, then it won't grant you access to IPv6-only sites (no software can do that, and Universal Relay is not intended to change that), but if you do switch to an ISP which has IPv6, then Universal Relay can make the most out of it, and if you switch back to an IPv4-only ISP, then you can still sustain the IPv6-only private side network.

There were a number of other circumstances that also led to the creation of Universal Relay, such as

  • DNS irregularities
  • Transparent Happy Eyeballs
  • Wanting to stay safe on public WiFi (using the SOCKS server to the Throwaway Box), as a potential alternative to the WireGuard netns bridge.
  • Being able to use it without root access (this was useful for the Throwaway Box)
  • The difficulty of DIY-ing a switch. This is because most commercially available switches use hardware packet forwarding, whereas a DIY switch would have to use software forwarding. However, a software switch would have many more features, such as the filters made available in ebtables, as well as layer 3 routing, so adding more features into a software switch would make up for the inefficiency.

In addition to the above, I remember having written an old text document which had address and subnet mappings to effectively "partition" the IPv6 address space. The text below is not exactly the contents of that text document, but is very similar in spirit: 

fd00:1200:4500:7800::/64 -> fe80::/64 on eth0
fd00:1200:4500:7801::/64 -> fe80::/64 on eth1
fd00:1200:4500:7802::1 port 80 -> unix /run/nginx.sock

Summary: Registered IPv4/IPv6 space and ASN with RIR. Initially used it to support home network. Unfortunately, not sustainable in the long run, so switched back to ISP addresses with IPv6 NAT workaround. Workaround fell apart as I contemplated switching ISPs. Tunnels were found problematic too. Created Universal Relay as a consequence of realizing those issues.

Functionality deemed "out of scope"

  • Any means of bypassing censorship or existing web filters.
  • Maintaining block lists or categories of web sites (like with Pi-Hole). This is not to be confused with the collection of domain names (with the dump_data interface) for the purpose of building a relay_map (see Universal Relay/Static map manager), or where the filtering is done for operational or technical reasons (such as to classify certain domain names which may require special treatment on the network layer, such as "captive" (don't use a proxy or VPN), "ocsp" (use HTTP instead of HTTPS), or "ntp" (redirect to central NTP server))
  • Completely blocking all access to a certain web site; the domain filters will only filter out patterns of domains if the programmatic DNS server were to be used. If the client connected to the IP address directly (using the ip4/ip6-[...].u-relay.home.arpa zone) and used some domain fronting technique, then Universal Relay does not really have the means of blocking that without also causing collateral blocking. The only safe place where you can ensure that a particular IP address is blocked is by using the transform_resolved_endpoint user hook, or by ensuring that clients cannot provide arbitrary parameters to a particular upstream connection (like a static C-list implemented using the resolve_map user hook). Although the filters can be used to block IP addresses or websites like with Pi-Hole or pfBlocker-NG, they are much more suited for redirection or facilitation of additional connectivity, rather than to block connectivity.
  • Universal Relay can be thought of as supporting "deep packet inspection", as it is theoretically able to modify TCP streams on the fly. However, it is not deep "packet" inspection per se, since it still operates on the TCP stream level, not the packet level.
  • SSL decryption and content inspection. While this may be a useful feature and is already supported by TLS sockets in Node.js, it is out of scope due to the complexity this feature may have. Currently, TLS streams pass through in encrypted form without decrypting.

Detailed Description

This article or section needs to be updated.
Describe the mode of operation of the fake DNS server, and how it allows us to recover the original domain name, as well as the Internet and ISP-independent properties of Universal Relay's private side.

Universal Relay is a simple TCP->TCP relay for (isolated) network namespaces. It supports a variety of modes of operation, such as tunneling all traffic through a SOCKS server, as well as custom routing of network destinations.

Universal Relay was originally intended to be a means of causing all network traffic in a network namespace to be tunneled through a SOCKS proxy (such as the one created by ssh -D). Similar in overall spirit to slirp4netns, but instead of creating a TUN device and using a userspace TCP/IP stack, we reuse the kernel's TCP/IP stack using the following redirection in the newly created network namespace: 

ip link set lo up
ip route add local 0.0.0.0/0 dev lo table main
ip route add local ::/0 dev lo table main

and use ctrtool ns_open_file to create listening sockets in the network namespace.

TCP/IP connections are read from the namespace's listening sockets; when a new connection is made, the destination is read (using the same technique as previously used in IPv6 Things and Socketbox) and translated into SOCKS5 CONNECT commands on the host.

Unlike slirp4netns, Universal Relay specifically accounts for the fact that there might be a whole network that it should be applied to, so you can create subnets, static routes, and even BGP sessions in that network namespace to other network namespaces to make all your VMs and (rootless) Docker containers use it, for example. It's also a very effective way to experiment with various routing protocols (BGP, OSPF, RIP, etc.) using network namespaces. Unlike a traditional address translator, this is possible even if the IP addresses conflict with the host network.

Also intended to be a NAT64 CLAT / PLAT by rewriting IPv4 destinations into their IPv6 equivalent with NAT64 prefix, or vice versa. Quite difficult to do on the layer 3, but rather trivial on the application layer. Unix domain socket destinations will also be supported.

Also intended to be used in conjunction with a protocol that will be developed for use with Socket Enhancer, to take "transparent proxying" off the latter's to do list.

Also intended to be a more flexible and general-purpose version of Snippets:Node.js NAT64 TCP relay, inet-relay, and socketbox-relay.

TCP port forwarding as implemented in slirp4netns will not be supported in Universal Relay itself due to fundamental complexities, as Node.js would need to be able to create sockets in multiple network namespaces (which it doesn't really support). Instead, the recommended solution is to use ctrtool ns_open_file or socketbox to create a listening socket in the host network namespace, then use HAProxy (or a similar tool) in the foreign network namespace with bind fd@${CTRTOOL_NS_OPEN_FILE_FD_n} (if using ctrtool ns_open_file) or bind sockpair@${CTRTOOL_NS_OPEN_FILE_FD_n} (if using Socketbox) in a frontend to relay data from the host network namespace over to the foreign namespace.

The name "Universal Relay" comes from the fact that it is technically a socket relay (similar to socat); the term "universal" refers to the fact that it can handle relaying for the entire Internet, or "universe", as well as the "universal" applicability of its features across many different scenarios, unlike socat which can only handle one IP address and port, which makes socat useful only for relaying a single service. Universal Relay is not a Tor relay (though it can be used in conjunction with Tor's SOCKS server).

Universal Relay has a fake DNS server that dynamically maps domain names to "phantom" IP addresses. This allows us to implement features like transparent Happy Eyeballs, since the transport medium is now independent of the actual IP addresses on the Internet.
Anatomy of "cookie" IPv6 addresses generated by Universal Relay's programmatic DNS server.
With the fake DNS server, it is normal that all "public" IP addresses used on the Internet are unroutable on the private-side network.

With the "transparent_server" enabled, Universal Relay is technically still a NAT device, since it "collapses" a series of private IP addresses into a single IPv4 and IPv6 address, though it still strives to be a much better version of e.g. the NAT in iptables. For example, instead of sending the network traffic directly out into the outbound/WAN interface, it can arbitrarily rewrite network destinations to other places of choosing, such as through a SOCKS server or a Unix domain socket, or a different IP address (including (untested) a link-local IPv6 address), or even from IPv4 to IPv6 or vice versa (there is an ipRewrite module that implements the equivalent of NAT64/464XLAT).

Other advantages of Universal Relay over iptables include:

  • No root access required on host, as long as you can create a new user and network namespace (e.g. with unshare -r -n); very useful for "rootless" containers.
  • No need to have iptables support in your kernel (except for the TPROXY target, which is recommended, but not required.)
  • Logging of connections (for auditing purposes); if you don't like this, just redirect stdout to /dev/null.
  • Implemented in a way that actually blocks inbound connections to the private-side network.
  • Reuses kernel TCP/IP stack for speed, simplicity and security.
  • Deals with IP address conflicts gracefully (i.e. the relaying on the host is as if the private-side network were to be completely disregarded in terms of the routing table). This is also useful if one of the upstreams is a DN42 or Tailscale VPN where you might have an unbounded set of private IP addresses (IPv4 or IPv6) on that upstream.
  • No raw network protocol access from the private side (this prevents header spoofing in case there are untrusted clients on the private side)
  • When used with Socket Enhancer, it is possible to set the source IP address for individual outbound connections to Internet destinations on-the-fly; this can be combined with policy-based routing.
  • Can still be used with only a subset of the IPv4/IPv6 address space (useful if there is already a default gateway on the private-side network, and Universal Relay is used primarily for address rewriting or NAT64 purposes); in this case, you would set up a (static) route covering that subset of address space that goes to the network namespace associated with Universal Relay's listening socket, and all clients that wish to use it would have to recognize that route (either directly, by adding the route on the device itself, or indirectly, by adding the route on the device's default gateway).
  • If the two sides of Universal Relay have different path MTUs and/or different TCP maximum segment sizes (TCP MSS), then Universal Relay makes those differences invisible between the two sides.

Disadvantages include:

  • Only relays TCP. UDP is not supported.
- For DNS, this is worked around by (untested) putting Unbound on the private side that listens on both TCP and UDP and forwards all DNS requests to the upstream server (through Universal Relay) over TCP. Alternatively, for a single host, it is possible to put options use-vc in /etc/resolv.conf.
- For NTP, this is worked around by (untested) putting an NTP server that has 127.127.1.0 (local/undisciplined clock) as a clock source on the private side, but which is on the same host as Universal Relay itself, and provided that the host already has correct time synchronization (it will appear to clients as a stratum 1 server with refid .LOCL.).
- See also Universal Relay/napi-socket.
  • Not possible to pass through special TCP options, sequence number offsets, or urgent data.
  • Pings on the private side will appear to always respond, even if the ultimate target host on the Internet does not. This also means that traceroutes to Internet destinations from the private side will only see the private-side routers, and a traceroute to an Internet-side Traceroute Text Generator will not show the actual traceroute text.
  • No loop detection (the TTL does not decrement for connections that pass through Universal Relay; rather, it is reset to the host's default.)
  • May be a little bit slower and take up more memory (due to socket buffers and bookkeeping) than iptables connection tracking, which is in-kernel. An iperf test on the local host shows speeds of about 6-7 Gbps.
  • Can only forward a discrete set of port numbers (if not using TPROXY; TPROXY is still available even on rootless containers (if using iptables-nft) because it only requires CAP_NET_ADMIN in the user-namespace-owned network namespace, and not on the host); without TPROXY, the suggested starting list of port numbers to allow is: 53, 80, 443, 853, 993, 8080. TPROXY is still preferred, however.

The TPROXY redirection is as follows, to be done in the same network namespace as Universal Relay's listening sockets: 

iptables-nft -t mangle -A PREROUTING ! -i lo -p tcp -j TPROXY --on-ip 127.0.0.1 --on-port 53
ip6tables-nft -t mangle -A PREROUTING ! -i lo -p tcp -j TPROXY --on-ip ::ffff:127.0.0.1 --on-port 53

Note that this redirection only applies to other network namespaces and physical or virtual clients connected to the namespace through veth or tun/tap devices. It does not apply to the original network namespace itself; you can remove ! -i lo if you'd like, but this could result in potential glitches; if not, then you will have to bind each port number explicitly to intercept connections in the same network namespace as the listening sockets.

Hint for development: The SOCKS server created by the OpenSSH client (when using -D) supports Unix domain sockets on the client side: 

ssh user@host -D /home/sebastian/gitprojects/universal-relay/urelay.sock

(Obviously, if your name is not Sebastian, then you would have to change that path :))

TCP sockets can be thought of as bidirectional pipes over the network (like Unix pipes and the pipes created with the '|' operator in Bash or other POSIX shells). What is sent to the server socket comes out of the client socket and vice versa. TCP is reliable, so that ensures that all of the original data is preserved, just like a Unix pipe. If the read end of a Unix pipe is closed, then writing to the pipe will fail with an error and signal the process to terminate. But if the write end is closed, then reading from the pipe will result in an EOF (read returns 0 bytes read). TCP sockets are in the same way, as if they were the read end of one pipe and the write end of another pipe. To close just the read end or write end, you would use the shutdown() system call.

Known bugs

  • The SOCKS server to direct mode has been tested successfully on Windows. However, the transparent proxy feature is not compatible with Windows Subsystem for Linux (WSL) due to lack of iptables TPROXY support. As mentioned before, it might be possible to bind each port individually.
  • Many virtualization and containerization solutions like Docker, VirtualBox, and WSL do not support IPv6 connections from virtual machines and containers out of the box. However, recent versions of QEMU in SLIRP mode do support IPv6.

To do

Development

  • Multicast DNS resolver module (using the avahi-daemon simple interface).
  • Universal Relay + PowerDNS + dnsmasq + IPv6 Things + Socketbox ctrtool container template script
  • UDP (using python asyncio)
  • SNI/Host header reading -- done! See SNI proxying (Universal Relay). Host header reading will likely be done in some HTTP module instead, due to the possibility of changing the Host header for different requests in a single keep-alive connection.
  • PROXY protocol header sending and possibly reading. Currently expecting to support sending a PROXY protocol header for both v1 and v2, and receiving a PROXY protocol header for v2 only, using the socket 4-tuple from the private-side network (which could be exposed to the Internet, thus forming a reverse proxy). In the case of v2, the domain name recovered by stage 2 of Transparent Happy Eyeballs may be sent as the domain name in the PP2_TYPE_AUTHORITY value.
  • Happy Eyeballs with SOCKS client in IP address mode
  • SQL table based IP to domain mapping (MySQL, PostgreSQL, SQLite, etc.) (using CREATE SEQUENCE to assign "phantom" IP addresses to domains as a unique primary key)
  • SRV record to domain endpoint helper function
  • fedb:1200:4500:7800::/64 -> ULA prefix
  • Generator for example-static-map.json -- combine the maps of multiple discrete files, auto-assignment of static offsets in relay_map -- Done! See Universal Relay/Static map manager
  • Endpoint fromJSON and toJSON functions -- note that !connFuncType ("direct", "socks", or "directTLS") will need to be used in the options map instead of !connFunc
  • Plugin system
  • Manually emitting a connection event (for connReadAttributes.socketAcceptor) does not work on an Express app directly. However, you can wrap the Express app in http.createServer or https.createServer. This may eventually make its way into the telnet and HTTP(S) servers of IPv6 Things and IPv6 Bible. (connReadAttributes.socketAcceptor is a means of connecting incoming sockets into "internal functions", such as a controller web server, instead of making a TCP, SOCKS, or Unix domain socket connection somewhere else.)
  • For the transparent encryption feature, the set of domain names that are allowed to be connected to (and for which the server certificate would be verified) can be limited to just a few domain names. In any case, it can be programmed to be presented as a mathematical set.
  • DNS cache

Operations

  • Consider "partial" private side networks:
    • Instead of creating a new network altogether, provide the listening sockets as a service on an existing network.
    • It is possible to use dnsmasq to map domains into the static region (using --address= and/or a hosts file) or use the programmatic DNS server as an upstream DNS server (using --server=), which Universal Relay can convert back to the original domain and use Happy Eyeballs to connect to the domain on the Internet.
    • It is possible to use the DNS64 module from Unbound and set the NAT64 prefix to the Universal Relay NAT64 region (or any other 32-bit group substitution static region with ip_subst set to 0.0.0.0/0)
    • The main disadvantage here is that we lose the characteristic that we are able to sustain our private side network even if the upstream ISP changes. This can be mitigated using an IPv6-enabled VPN that supports prefix delegation for the private side "upstream" network, or possibly by using slirp4netns.
    • We also run the risk of potentially looping back onto itself. This can be mitigated by blocking connections to the ipv6_prefix on the Internet side.
    • The easiest way to realize this for right now is to use the WireGuard netns bridge to set up the isolated WireGuard "domain", and then apply the TPROXY rules in the newly created namespace, then create the listening sockets in that namespace (using ctrtool ns_open_file)

For example, you could have in example-static-map.json: 

"relay_map": [["www.google.com", 1], ["www.apple.com", 2]]

and, in the DNS server's static AAAA or host file configuration: 

fedb:1200:4500:7800:5ff:7001:0:1 www.google.com
fedb:1200:4500:7800:5ff:7001:0:2 www.apple.com

If a user browses www.google.com, then the DNS server, by virtue of the AAAA and host file configuration, will return the IPv6 address ending in :0:1. This routes to Universal Relay, which by virtue of relay_map, detects that it's for www.google.com, and therefore proxies the connection there. The same is true for www.apple.com and the IPv6 address ending in :0:2. But all other domains go directly to the Internet as usual, without going through Universal Relay (unless a DNS64 server is configured to use the Universal Relay NAT64 region)

The IPv6 address and the relay_map offset are not related to the actual IP address of the domain names and are therefore safe to hard code, since the resolution of the domain name on the Internet is only performed at the final step (see Transparent Happy Eyeballs)

  • The PROXY protocol v2 could be used to provide access to an IPv6 network without having IPv6. This is done by running an instance on the Internet server side, and then limiting the set of possible destinations to specific local networks only. Another instance runs client side, and it just intercepts connections to that network and sends it to the TCP port of the server with a PROXY protocol v2 header with the original destination IP address and port number.
  • Consider using the source IP address to determine whether outbound connections should be made directly or through a different upstream interface. (This is intended to allow a client to be able to select which upstream interface to use by way of its network configuration.)

Links

See also

Retrieved from "https://website.peterjin.org/w/index.php?title=Universal_Relay&oldid=2829"