Transparent Happy Eyeballs

Transparent Happy Eyeballs assigns "cookie" IPv6 addresses to domain names used by a website; despite the addresses not likely correlating to the actual IP address of the domain name, the website still functions normally.

Transparent Happy Eyeballs is a feature of Universal Relay that allows the Happy Eyeballs algorithm for fast IPv6 to IPv4 fallback (RFC6555, RFC8305) to be implemented on the network layer, even if the end application doesn't support it natively.

Rationale

If you attempted to access website.peterjin.org using the IP address instead of the domain name, or with any IP address or domain name other than website.peterjin.org, you would see a certificate error. Thus, it is necessary to keep the original domain name of website.peterjin.org. However, website.peterjin.org can point to any IP address you want. The Universal Relay equivalent of accessing that IP address is https://ip4-23-161-208-254.u-relay.home.arpa, though that will also result in an SSL error.
  • To access a website normally, a web browser first uses DNS to look up the IP address of the domain name you are trying to visit. The web browser then connects to that IP address to access the website.
  • Normally, if you query e.g. www.google.com, the IP addresses will be some IP address range owned by Google, or for other websites, the ISP or hosting provider.
  • If the IP address returned by a DNS query to www.google.com is usually not associated with www.google.com, then you wouldn't be able to access Google.
  • But what if we put a forwarding proxy on that IP address, which listens on port 443 and connects to the real www.google.com?
  • The web browser first looks up the IP address of www.google.com. Unbeknownst to the web browser, the IP address of www.google.com is not that of Google, but that of the forwarding proxy. The web browser then makes a connection to port 443 on that IP address. The connection is received by the forwarding proxy, which looks up the real IP address of www.google.com and connects to the real www.google.com. Once the socket connection is established, the server socket from the web browser and the socket to www.google.com are piped (relayed) to and from each other.
  • socat can be used to create such a forwarding proxy:
socat TCP-LISTEN:100.90.0.1:443,reuseaddr,fork TCP:www.google.com:443
  • In this way, the browser is communicating with the proxy, but the TCP stream is that of a socket connection to www.google.com. Thus, the web browser looks like it's connected to www.google.com when in fact it's actually through the proxy.
  • The web browser sees that the IP address of www.google.com is 100.90.0.1, which we can do with /etc/hosts or a DNS server. The web browser makes a connection to 100.90.0.1 port 443. This is picked up by the forwarding proxy, which connects to the real www.google.com. The TCP stream is relayed between the two connections, so the browser appears to still be able to communicate with www.google.com.
  • If you wanted to add another domain name, then you would have to use another IP address:
socat TCP-LISTEN:100.90.0.2:443,reuseaddr,fork TCP:www.apple.com:443
  • "Transparent Happy Eyeballs" is basically this, but much more systematic and extends to all domain names on the Internet. Instead of assigning the IP addresses manually, we use a custom DNS server. Upon query of a domain name, it will assign a random IP address to that domain name, and return that IP address as an AAAA record. The IP address is mapped back to the domain name upon receiving the TCP connection. The so-called "AnyIP" trick is used to allow a network namespace to be able to receive TCP connections for an entire subnet (see Snippets:Nginx geo local server address), and a call to getsockname() (exposed as socket.localAddress in net.createServer) allows us to retrieve the exact IP address specified in the TCP SYN packet. This is much more efficient than running multiple instances of socat. The IP address is mapped back to the domain name, and Universal Relay makes a TCP connection to the real IP address of the domain name.
  • The main rationale for modifying DNS here, instead of using something that may more accurately describe the proxy server's endpoint, is that it allows us to use a proxy server while the web browser or other client is still using the original domain name and port number. Requiring the web browser to use a different domain name and port number is difficult, as they are effectively hard coded in URLs that may be referenced by other websites in the HTML code (including but not limited to hyperlinks, images, style sheets, scripts, and XMLHttpRequest), as well as the fact that the domain name cannot change because that would cause the Host header or SNI to be incorrect or not recognized by the web server (a similar scenario exists at [1]).
  • This procedure requires a very large number of IP addresses, as one IP address is required per domain name, and devices are expected to access many different websites. Thus, the implementation is IPv6 only. But it is possible for the device to connect via IPv6 to the proxy and the connection to the real website is over IPv4.

Method of operation

This article or section needs to be updated.
Explain why it is necessary to use a pool of IPv6 addresses, namely due to the fact that we need to establish a bijective mapping, and IP addresses on the Internet won't suffice since multiple domains may have the same IP address. Also explain why u-relay.home.arpa was chosen over simply "passing through" IPv4/IPv6 literals to Universal Relay's listening sockets using the default route.

Explain how the domain to IP mapping is managed and how it is similar to an LRU cache.

Transparent Happy Eyeballs is implemented using a DNS server coupled to a TCP relay which allows domain names to be recovered from TCP connections in a transparent and L7-independent manner. Without Transparent Happy Eyeballs, the client has to manage the IPv4 fallback, but Transparent Happy Eyeballs allows the IPv4 fallback to be shifted from the client to Universal Relay instead, so that the benefits of Happy Eyeballs can be realized by all clients on the private side network, as long as it generally supports IPv6 connections. From the perspective of the client, the connection to the "cookie" IPv6 address appears as a single, unbroken connection, which transparently falls back to IPv4 if the IPv6 connection fails.
Anatomy of "cookie" IPv6 addresses generated by Universal Relay's programmatic DNS server.
The client might request a domain name which does not actually exist on the Internet. In this case, the programmatic DNS server will still assign a "cookie" IPv6 address, but the real DNS resolution will fail, so Universal Relay will simply reset and terminate the connection; this error message is different from the normal "Server not found" error message if a domain does not exist on the Internet. Even if the domain name does not exist in real DNS, the client's requested domain name may still be interpreted in other ways, such as to extract the IPv4 or IPv6 literal from a u-relay.home.arpa subdomain.
Top: The programmatic DNS server is implemented as a root name server which answers authoritatively for all queried domains. Bottom: Comparison with a regular DNS server on the Internet.

A programmatic DNS server is internally set up to return "cookie" IPv6 addresses for every domain that it receives queries, and subsequently record the cookie IPv6 to domain mapping. The returned IPv6 addresses point to a ip route add local region (see also Snippets:Nginx geo local server address) which Universal Relay listens on. One address is allocated per domain name queried (see also Universal Relay/Domain Mapping), so that Universal Relay can recover the original DNS name that was associated with the "cookie" IP address, even if the protocol does not have the equivalent of SNI or a Host header (e.g. SMTP or SSH). Now that we have the original DNS name, it can be resolved to real IPv4 and IPv6 addresses, and Universal Relay can then make the upstream connection using a variant of the standard Happy Eyeballs algorithm.

This is actually very similar to NAT64 and DNS64, in that we generate IP addresses to point to a fixed region, and then route that fixed region to an application which can handle those IP addresses.

We have to use "cookie" IPv6 addresses instead of the real IPv6 addresses because the latter fails to account for the fact that there could be two different domains with the same IPv6 address, but different IPv4 addresses, and the common IPv6 address is unreachable; without any other observable flags or indicators telling the domain names apart, Universal Relay could not reliably fall back to the correct IPv4 address.

Only one cookie IPv6 address is allocated for any given domain, even if the domain is IPv4-only or has multiple IP addresses on the Internet. With the exception of domains under a few special-purpose zones such as u-relay.home.arpa, no two domains have the same cookie IPv6 address, even if they have the same set of IP addresses on the Internet (but this can be overridden using domain_canonicalizer)

This procedure is very similar to Tor's AutomapHostsOnResolve feature, but it is IPv6 only on the client-facing side, not specific to Tor onion domains, is able to use Happy Eyeballs to connect to the ultimate destination, restricts cookie IP addresses to within a /64 (so only that /64 needs to be routed, rather than the default route), and implements a static region for applications that cannot handle dynamic address assignment to domain names.

This procedure can work well with domain names, but it does not work well with bare IPv4/IPv6 literals. To mitigate this shortcoming, Universal Relay recognizes the special u-relay.home.arpa zone, which is used to, among other things, represent IPv4/IPv6 literals. Example domains under that zone are ip6-2001-db8--1.u-relay.home.arpa and ip4-192-0-2-1.u-relay.home.arpa. These domains represent the IPv6 address 2001:db8::1 and IPv4 address 192.0.2.1. Universal Relay allocates a cookie IPv6 address for those domains just like with any other domain, but resolves the domains in the final stage to the fixed IPv4 or IPv6 address that is embedded within the first level subdomain of u-relay.home.arpa.

Transparent Happy Eyeballs was built upon a number of premises:

  • The vast majority of applications do not care about the IP addresses of domain names embedded within the application. If google.com resolved to a different IP address from a different perspective, then applications don't really care about that, as long as it's actually reaching a google.com server. With the rising use of SSL certificates attached to domain names rather than IP addresses, as well as name-based virtual hosting with the Host header and SNI, the vast majority of applications no longer hard code IP addresses. The effect of all this is that it is possible to assign any IP address to any domain name, provided that the IP address can be routed to, and the ultimate connection to that IP address leads to an actual Internet connection to that domain name on the Internet.
  • Happy Eyeballs is a feature that ought to exist in all TCP-based applications. Adding Happy Eyeballs support requires that all applications be modified to support it. If there are many applications, then this can be burdensome.
  • In regards to the u-relay.home.arpa domain, web servers should treat an IP address in the Host header as if it were an unknown domain; thus, it can be expected that accessing a web server on a given IP address but with an unknown Host header domain would be equivalent to accessing the web server directly using the IP address. There are some web servers like 1.1.1.1 where that is not true, but the majority of web servers on the Internet should behave in that way.
  • In regards to the u-relay.home.arpa domain, the majority of applications that accept IP addresses as a remote hostname argument will also accept domain names.

Transparent Happy Eyeballs also has a couple of other advantages:

  • It is now possible to read any domain name that the client requests, in its exact, original form, and apply transformations and/or filters based on that domain name, even if that domain name does not exist on the Internet, that domain name shares an IP address or set of IP addresses with another domain (by way of CNAME or otherwise), or the client uses a non-HTTP and non-SSL protocol like SMTP, SSH, or FTP (extended passive mode only) to connect to the domain name. For example, www.example.com and www.example.org could have the same IPv4 and IPv6 address, which is problematic if we want to use network layer software to route on only one of those domains. By using the programmatic DNS server to assign different IPv6 addresses to those two domains, the domains can now be differentiated using the fact that they are destined towards a different IPv6 address.
  • It is possible to statically assign a fixed IPv6 address to a particular domain name (using relay_map, see Universal Relay/Static region allocations), which allows network-level filters to be constructed on that domain name, even if the IPv4 or IPv6 address of that domain name changes on the Internet.

For example, it is possible to construct a user hook filter with code such as:

if (ep.getDomainString() === 'www.example.com') {
    ep.options_map_.set("!bind_addr", "192.168.1.1");
}

And the conditional bind address of 192.168.1.1 for connections to www.example.com would be honored by all applications on any operating system for systems connected to the private side network, provided that the application supports DNS resolution using the system DNS resolver, provided that it is pointed to the programmatic DNS server; this can be automatically configured using the DNS server option in DHCPv4 and DHCPv6, as well as the RDNSS option (see RFC 8106) in SLAAC RAs. The intended effect is that the filters could be used in any context involving the domain name (e.g. typing the domain name in a web browser, wget https://www.example.com, ssh root@www.example.com, or nc www.example.com 80[1]), thus making the filters "universal".

  • The Happy Eyeballs algorithm is applied to all connections, so it is much more feasibly possible to use a less-reliable IPv6 transition mechanism like 6to4 or a Hurricane Electric tunnel (which has peering issues with Cogent) for IPv6 Internet access.
  • The Happy Eyeballs implementation allows rewriting of addresses to apply a NAT64 prefix on the Internet side, which allows Happy Eyeballs to be realized even if the DNS64 server doesn't synthesize AAAA records for the IPv4 addresses on dual-stack destinations.
  • Domain names can now connect to destinations that cannot be represented in a DNS A or AAAA record (see Out-of-band DNS record), including, but not limited to, IPv6 link local addresses with scope ID, alternate ports or port numbers, Unix domain sockets, Tor onion domains, and resources only available through a proxy or VPN in circumstances where they have the same IP address as some other unrelated resource on a different network.
  • An actual internet connection is not required to assign cookie IP addresses to domains within the programmatic DNS server. Universal Relay does not intercept pings (they just stop in the network namespace of Universal Relay's listening sockets, and the network namespace always responds), so a ping to www.google.com will always appear to work even if there is no Internet connection.
  • Because the programmatic DNS server only returns IP addresses limited to its configured prefix, the systems on the private side network are immune to DNS rebinding attacks which attack systems on the private side network. DNS rebinding is still possible on the Internet side, however it is possible to use the IP address matching and rewriting facility to block connections to private IP addresses if the domain name is one where a private IP address is not expected.
  • The default route is freed up and can be used for other purposes, such as to connect to alternative Internets like DN42 or to peer with other instances of Universal Relay's private-side network. (It is also possible to use a DN42 VPN as an upstream, where domains under *.dn42 and ULAs represented in domain names like ip4-172-20-0-1s-dn42.u-relay.home.arpa or ip6-fd42-d42-d42--1s-dn42.u-relay.home.arpa cause the DN42 VPN to be selected instead of the regular IANA/ICANN internet. This is especially useful if your normal Internet connection uses the same IP address ranges as DN42 networks, and you want to access both DN42 and something on your local network without having to change network or proxy settings.)
  • From the client's perspective, all domain names appear to have exactly one IPv6 address, which allows the private-side network to be IPv6 only and still have access to IPv4-only websites, just like a DNS64/NAT64 system. No separate DNS64/NAT64 is needed. Alternatively, there is a NAT64 region within the static region, and this NAT64 region is made aware to clients with a CLAT by generating AAAA records on the ipv4only.arpa domain. Though it is seldom used unless the client used a DNS resolver on the Internet, since real IPv4 addresses of Internet destinations are not passed through the programmatic DNS server. It was miraculously used without even realizing it, if DNS-over-HTTPS is enabled in Firefox (disabled by default since use-application-dns.net is blocked)
  • Because DNS queries are not actually passed through to the Internet in the first stage (where the client makes a DNS query), DNS tunneling attacks can be mitigated with a domain name filter in the third stage (where the client makes a TCP connection and Universal Relay makes a DNS query to the Internet).

Technical details

Universal Relay uses an LRU cache (implemented using the JavaScript Map object) to manage the domain to cookie IP mapping. If the domain name does exist in the cache, then the corresponding IP is returned, and the cache entry is updated to be the most recent. If a domain name does not already exist in the cache, then a mapping to a random "cookie" IP address is created. The mapping is indexed both by IP address and domain name, and is inserted into the mapping table. If there are 10,000 entries or more already in the mapping, the least recently used mapping is removed from both indexes. Subsequently, the corresponding "cookie" IP address will no longer be valid for the least recently used entry's domain name, and if the domain name corresponding to the least recently used entry is queried again, then a new IP address will be generated for that domain name.

Cookie IP addresses bind to the domain names themselves, not the Internet IP addresses associated with the domain names. An entry mapping a domain name to a cookie IP address can be kept as long as needed without expiration, regardless of the TTL of the domain name's IP addresses on the Internet.

See also

  1. ping www.example.com would resolve www.example.com using the programmatic DNS server as usual. However, ICMP echos are not relayed and are instead simply responded to using the ping reply functionality inherent in the network namespace containing Universal Relay's listening sockets, such that the application appears to receive a ping reply, even though the host on the Internet is not actually pinged. This means that an application could check for Internet connectivity using a command such as ping www.google.com, and it would appear to receive ping replies and therefore pass the check, regardless of whether or not there is an actual Internet connection. Similarly, traceroute and tracert would show the routers on the private side network, but appear to "stop" at Universal Relay's listening sockets.