synergy Distributed Stream Processing Middleware

Research Projects

• Sharing-Aware Component Composition
  

  Distributed stream processing systems need to satisfy application QoS requirements despite the shared, dynamic, and large-scale environment. Designed and implemented in Synergy techniques for composing distributed stream processing applications with QoS demands. Synergy enhances QoS provision and reduces resource load by efficiently reusing both data streams and processing components. To achieve this goal Synergy provides a set of fully distributed algorithms to discover and evaluate the reusability of available data streams and processing components when instantiating new stream applications. For QoS provision, Synergy performs QoS impact projection to examine whether the shared processing can cause QoS violations on currently running applications. Projection is based on queueing theoretical models for both regular and bursty traffic. For initial component deployment Synergy employs a decentralized replica placement protocol that aims to maximize availability, while respecting resource constraints, and making performance-aware placement decisions.

• Load Prediction and Hot-Spot Alleviation
  

  Managing the load of the nodes of a large-scale, dynamic distributed stream processing system in real-time and without centralized supervision is challenging. Designed and implemented in Synergy a self-managing resource monitoring architecture for identifying and relieving overloaded nodes. Monitoring responsibilities are shared among all nodes using a completely decentralized DHT-based architecture. Nodes proactively predict application QoS violations using a statistical forecasting framework based on auto- and cross-correlation, as well as linear regression. To alleviate hot-spots, nodes autonomously migrate the execution of stream processing components using a non-disruptive migration protocol. Synergy's performance evaluation over PlanetLab involves the implementation of a network traffic monitoring application operating on real streaming data.

• Managing Large-Scale Distributed Real-Time Applications
  

  Managing large-scale distributed real-time applications. Satisfying end-to-end QoS requirements in large-scale distributed real-time applications is challenging, due to the unpredictability and heterogeneity of the environment. Focusing on media streaming and transcoding as an example, co-designed in C++ a decentralized architecture that enables nodes to collaborate to offer composite applications. A resource allocation algorithm distributes the processing and communication loads fairly, while satisfying application end-to-end QoS demands. Nodes adapt to variable resource conditions by optimizing their resource usage locally, selecting from discrete QoS output levels. Multiple nodes participating in a composite application coordinate their QoS through feedback from the service receiver.

• Data Dissemination in Peer-to-Peer Systems
  

  Efficiently locating data or services in a fully decentralized, self-organizing, unstructured overlay network is challenging, due to the large scale and the lack of global view. Designed in Java a content-driven routing mechanism that guides queries to peers most probable to provide results. Intelligently routing queries is based on Bloom filter synopses of peers' content. Adaptive dissemination algorithms propagate synopses according to peers' interactions. A decentralized trust management architecture enables peers to evaluate reputation information and avoid lying and colluding.

• Consistent Replication in Distributed Multi-Tier Architectures
  

  Designed, implemented in Java, evaluated, and documented a replication middleware for distributed, multi-tier, server architectures. Proposed and incorporated in the middleware an efficient, distributed strong consistency protocol. Quantified the server replication and data partitioning performance benefits, as well as the consistency overhead, using the TPC-W transactional web commerce benchmark.

• Collaborative Spam Filtering Robust To Sybil Attacks
  

  Designed, implemented in C++, and documented an event-driven software prototype of a collaborative spam filter that employs a distributed protocol to defend against sybil attacks.

• Distributed Logging for Asynchronous Replication
  

  Designed, implemented in C++, and documented a logging mechanism used for asynchronous replication in a distributed disk array.

• Dynamic Page Migration for Software DSMs
  

  Designed and implemented in C a protocol for dynamic memory page migration across the nodes of a Software Distributed Shared Memory System. Dynamic page migration improves performance by increasing locality and adaptability, while remaining transparent to the application programmer.

• Narrow-Band Voice Over IP
  

  Analyzed the H.323 protocol family, used for multimedia applications (VoIP) in packet switched networks, and summarized the results in a technical report, including detailed protocol description and performance evaluation of applications under IPv6 in Solaris.

Course Projects

• Caching in Mobile Ad Hoc Networks
  Design of a two-level architecture for caching the results of spatial queries in mobile, ad hoc, peer-to-peer networks. [2005]


• Adaptive Data Propagation in Peer-to-Peer Systems
  Design and implementation in Java of a protocol for propagating Bloom Filter summaries of peers' data content and comparison to Breadth First Search by simulation. [2004] report presentation


• Reputation-Based Trust Management
  Design of a decentralized protocol for reputation-based trust management in unstructured peer-to-peer systems and evaluation of its performance by simulation. [2004] report


• Distributed Trust Management
  Survey on peer-to-peer trust management, presenting and comparing EigenTrust, P2PRep, RCertPX, P-Grid, and TrustMe. [2004] report


• SSL Offloading for High Performance Data Center Servers
  Design, implementation in C, and evaluation of the offloading of the SSL processing part of a web-server on a dedicated processor of an SMP. The SSL processing was done by pthreads, which communicated with the web-server threads using i)message queues and polling, ii) shared memory and signals. [2004]


• TCP Offloading
  Survey on offloading TCP processing on dedicated processors in SMPs, clusters, and NICs. [2004] presentation


• DNS Poisoning
  Analysis of attacks based on inserting bogus resource records in a victim's DNS cache to achieve host impersonation. [2004] presentation


• Model Driven Middleware
  Survey on Model Driven Architectures for the design of Distributed Real-Time Systems. [2004] presentation


• Data Dissemination in Wireless Sensor Networks
  Analysis of a data dissemination protocol used for reprogramming the nodes of a wireless sensor network and brief comparison with data dissemination in mobile peer-to-peer networks. [2005] presentation


• Hyper-Threading
  Analysis of the Simultaneous Multithreading implementation of Pentium 4. [2004] presentation


• Instruction Prediction
  Survey on instruction repetition and reuse in modern microprocessors. [2003] report


• Routing Protocols for Large-Scale Wireless Mobile Ad Hoc Networks
  Presentation of DSR, AODV, TORA, and LAR and performance comparison by simulation in ns-2 and QualNet. [2003] report


• User-Level Thread Library
  Implementation in C of a run-time user-level thread library and performance evaluation. The library's API provides threads, locks, and barriers, and all the functions common in thread libraries (e.g. create, exit, join, yield, self). [2002] report (in Greek) code


• Library of Synchronization Algorithms
  Implementation in C of a run-time library of 4 synchronization algorithms (test and test and set lock, ticket lock, array based queue lock, sense reversing centralized incremental barrier) and performance comparison. [2002] report (in Greek) code


• Industrial Networks
  Analysis and comparison of the real-time response and operation reliability of the industrial networks Profibus, WorldFIP and LonWorks in the MAC sublayer. [2002] report (in Greek)


• Reinforcement Learning
  Survey on Reinforcement Learning and its applications on intelligent industrial control. [2002] report (in Greek)


• Activity Theory
  Survey on the Activity Theory for Human-Computer Interaction and presentation of a case-study using the Activity Checklist for the design and evaluation of an e-learning application. [2002] report (in Greek)


• Bootstrap Loader
  Implementation of a bootstrap loader, executed on a PC's power-on, in 80x86 assembly. [2001] report (in Greek) code


• Parallel Matrix Multiplication
  Development of an application for parallel matrix multiplication in C using a Software DSM and performance comparison of different data allocation strategies. [2001] report (in Greek) code


• Aspect-Oriented Programming
  Survey on Aspect-Oriented Programming and presentation of a case-study using AspectJ for Java. [2001] report (in Greek) code


• TCP Agent
  Development of a multithreaded TCP agent in Java, that connects to time and chargen servers and forwards their outputs. [2000] report (in Greek) code


• Text Search Engine
  Development of a text search engine in C using hashing. [2000] report (in Greek) code

Fun Projects

• Templatized Factory Method Design Pattern
  Implementation in C++ of a deck of cards, demonstrating the use of the factory method design pattern using templates. [2008] code


• FreeBSD on T40
  Tips on running FreeBSD on IBM Thinkpad T40. [2003]


• Binary Sequence Decoder
  Development of a decoder in C, for translating binary sequences to ASCII text. [2001] code


• Internet Gateway
  Development of a client/server infrastructure in Java, to control an Internet gateway. [2001] code


cd /home