Oasis: An Overlay-Aware Network Stack

Harsha V. Madhyastha, Arun Venkataramani, Arvind Krishnamurthy and Thomas Anderson
harsha@cs.washington.edu, arun@cs.umass.edu, {arvind, tom}@cs.washington.edu
Overlays have enabled several new and popular distributed applications such as Akamai, Kazaa, and Bittorrent. However, the lack of an overlay-aware network stack has hindered the widespread use of general purpose overlay packet delivery services. In this paper, we describe the design and implementation of Oasis, a system and toolkit that enables legacy operating systems to access overlay-based packet delivery services. Oasis combines a set of ideas -- network address translation, name resolution, packet capture, dynamic code execution -- to provide greater user choice. We are in the process of making the Oasis toolkit available for public use, specifically, to ease the development of PlanetLab-based packet delivery services.

Design Goals

The following objectives guide the design of Oasis.
    1. Fine-grained control
      Oasis should enable the user to exercise fine-grained control in determining preferred Overlay Service Providers (OSPs) or performance goals for different portions of traffic. Note that, with the existing ISP model, user-choice is undermined because of the high overhead of switching carriers or employing multiple carriers simultaneously.
    2. Do no harm
      Oasis should not worsen performance or fault-tolerance compared to the existing Internet, i.e., an overlay should be employed only when it optimizes user-specified metrics beyond the underlay. Further, the computational overhead  that Oasis itself imposes upon all traffic should be minimal.
    3. Extensibility
      Oasis should also expose a generic interface to overlays to support diverse overlay services and protocols. To allow for Oasis, overlay services, and the division of labor between them to evolve independently, Oasis should be equipped to safely execute untrusted code supplied by OSPs.
    4. Deployability
      Oasis should not require any modification to legacy applications and operating systems.

Architecture
Basic architecture of Oasis


Oasis has three main tasks:
  1. to provide a facility to the user to specify routing goals
  2. to intercept relevant packets and determine the optimal carrier dynamically
  3. to interface with overlays to send and receive encapsulated packets
The above figure shows the core architecture of Oasis that we choose so as to fulfil these goals while conforming to our design guidelines of fine-grained control, minimal overhead, extensibility and deployability. The components in white with bold boundaries are those present in legacy operating systems. The shaded pieces, User Control and OSP Portal are the basic components that Oasis introduces. The components in dotted boxes are overlay-specific modules that are dynamically employed based on the user's preference. This decoupling between the OSP portal and the individual overlay modules has two benefits. First, Oasis is extensible to new overlay services as it does not restricting overlays to abide by a fixed API. Second, the native underlay (denoted by ISP/IP) can be used when no overlay can improve user-specified metrics beyond the underlay.

Design
   
Outgoing packet path Incoming packet path
Outgoing packet path Incoming packet path


Our realization of the architecture of Oasis consists of the following components as illustrated in the above figure.
  • VNI is a virtual network interface used to intercept, modify, and resend network packets from user-space
  • DNS Resolver maintains mappings needed to implement name-based routing preferences
  • Flow Map maintains per-flow state for address translation and routing
  • Monitor is a brokering agent that maintains a database of end-to-end path metrics for different OSPs
  • Overlay Manager manages the list of available OSPs
  • RT is an operating system-specific module for inserting routes into the routing table on the host machine
  • OC1, OC2, etc. are custom overlay code modules
  • OC0 simply routes the packet to the underlay by default

Components of Toolkit

We will soon be putting up for download some components of our toolkit that we hope would aid in the development of overlays.
  • Oasis: We will make available versions of Oasis that work on Windows and Linux.
  • Exit Node: We will also provide a flexible exit-node module that runs on PlanetLab. The principle features that it provides are NATing to interface with legacy destinations, efficient handling of raw sockets to enable this, and priority-based queueing of packets received from clients and other overlay nodes.