The Experiment Orchestration Toolkit (ExOT)

Researchers are required to support their claims with experimental evidence and provide results that are reproducible, comparable and exhaustive. However, the effort required to conduct exhaustive experimental analyses, or reproduce and compare different results, has proven to be high. To tackle these issues, we present the Experiment Orchestration Toolkit (ExOT).

ExOT was developed in the Computer Engineering Group, which is part of the Computer Engineering and Networks Laboratory at ETH Zürich. All components of the ExOT project are publicly available and all vital links to get started can be found on this website. ExOT is released under the 3-clause BSD license.

Feature Overview

ExOT supports reproducibility by using the TOML configuration and environment descriptor files to capture relevant experiment parameters. Figure 1 shows the structure of ExOT and the interaction of the different components of the experiment setup.

The main components of an experimental setup using ExOT are:

The experiment engine for data processing and experiment orchestration.
The devices under test, which are divided into different configurable zones.
Source, sink and jammer applications.

While source and sink applications should be implemented using the ExOT application library to be able to be controlled by the experiment engine, any application that can be run in the desired zone can be used as a jammer application. Jammer applications are optional, and can be used to simulate disruptive influences on the covert channel, which is useful to either

understand the influence of external factors on the covert channel, or to
evaluate possible mitigation strategies.

Figure 1: ExOT experiment structure

All applications are mapped to a zone, whereas one zone defines the runtime environment of the applications. For example, one zone in a measurement setup might be a virtual machine on a server, or a single node in a wireless sensor network.

The experiment engine

The experiment engine will setup the environment by configuring the zones(s) and applications as well as copying the necessary data. It controls the experiment execution, fetches the data and cleans up the environment after the experiment execution has finished. The experiment engine also offers a variety of debug outputs in the form of log-messages during execution, or plots for data preview during analysis. The complete flow from generating an experiment to analysis can be written in one Python script, which makes experiments easier to version and maintain. Moreover, the Data Processing also contains a variety of different drivers. These drivers allow the experiment engine to interact with different kinds of platforms, for example Android based smartphones or Linux based laptops.

Data Processing Model

The data processing model is based on a layered information flow model, illustrated in Figure 2. Similar to the well known OSI model, information travels from the highest layer to the lowest, and then up to the highest again.

Layer 6 describes how input data is generated and how metrics are calculated from the measurement data. In layer 5 and 4, the source and line coding is defined, which is used to compress and shape the data stream depending on the channel specifications. Layer 3 describes the data format required by the applications, while layer 2 defines file I/O. The two bottom layers describe the source (sending) and sink (receiving) applications and the channel and are not part of the data processing.

Figure 2: Data Processing Stack

Application Library

The Application Library facilitates the creation of sending, receiving and custom jamming applications that interoperate with the entire Experiment Orchestration Toolkit.

The library is implemented in C++17, and takes advantage of many modern language features. It broadly uses generic programming and compile-time code generation as well as avoids complex class hierarchies. The library can be rather easily extended. It has been successfully applied in creating sender and receiver applications for, among others, cache-based, thermal, power and frequency scaling covert channels.

The library provides a way of creating application based on the process networks model of computation, where individual nodes are connected together and pass data via queues. An example of such process network based applications is illustrated in Figure 3. We provide examples for such applications for Linux based systems in a separate repository.

Figure 3: ExOT example deployment aps

Compilation suite

The compilation suite is a docker based build environment, which minimises the effort for cross-compilation of the deployment applications. This allows researchers to apply their analysis to different platforms. Furthermore, it increases the reproducibility of deployment applications.

Applications for Android

We provide Android wrappers for all deployment applications, integrating the Android Library with the Android NDK. Using the wrappers, applications can be ported as background service or application with a user interface. Furthermore, we provide a service which operates as an intent proxy. The intent proxy is a background service which translates and relays all intents sent to it to the deployment applications.

Getting Started

ExOT Repositories»

ExOT Wiki»

In order to enjoy all features of ExOT, please follow the steps bellow to get a basic setup:

Make sure you have the following software installed:
Create an ExOT directory and clone all the ExOT repositories into this directory and initialise the submodules, using following script:

for repo in eengine app_unx app_apk compilation; do
  git clone https://gitlab.ethz.ch/tec/public/exot/${repo}.git
  cd ${repo}
  git checkout v1.1.0
  git submodule --init --recursive update
  cd ..
done

Now, you can use the ExOT Wiki to and the examples provided in the repository, familiarise yourself with ExOT.

Related Publications

Original Publication: Increased reproducibility and comparability of data leak evaluations using ExOT

As computing systems are increasingly shared among different users or application domains, researchers have intensified their efforts to detect possible data leaks. In particular, many investigations highlight the vulnerability of systems w.r.t. covert and side channel attacks. However, the effort required to reproduce and compare different results has proven to be high. Therefore, we present a novel methodology for covert channel evaluation. In addition, we introduce the Experiment Orchestration Toolkit (ExOT), which provides software tools to efficiently execute the methodology.

Our methodology ensures that the covert channel analysis yields expressive results that can be reproduced and allow the comparison of the threat potential of different data leaks. ExOT is a software bundle that consists of easy to extend C++ libraries and Python packages. These libraries and packages provide tools for the generation and execution of experiments, as well as the analysis of the experimental data. Therefore, ExOT decreases the engineering effort needed to execute our novel methodology. We verify these claims with an extensive evaluation of four different covert channels on an Intel Haswell and an ARMv8 based platform. In our evaluation, we derive capacity bounds and show achievable throughputs to compare the threat potential of these different covert channels.

Paper

Slides

@inproceedings{exot2020,
  title        = {{Increased reproducibility and comparability of data leak evaluations using ExOT}},
  author       = {Philipp Miedl and Bruno Klopott and Lothar Thiele}, 
  booktitle    = {2020 Design, Automation \& Test in Europe Conference \& Exhibition (DATE)},
  pages        = {},
  year         = {2020},
  organization = {IEEE}
}

Whitepaper: The Experimentation Orchestration Toolkit

In this whitepaper, we give a detailed overview of the design and implementation strategies used during the development of the first public version of ExOT. ExOT is designed to be easily extended and can be used to easily include a variety of different platforms in a measurement setup. It helps to automate the process of setting up, executing and analysing measurements.

Paper

@techreport{whiteexot2020,
  title        = {{The Experimentation Orchestration Toolkit}},
  author       = {Philipp Miedl and Bruno Klopott and Lothar Thiele}, 
  group        = {Computer Engineering and Networks Laboratory (TIK)},
  year         = {2020},
  institution  = {ETH Zürich},
  month        = {05},
}

Use-Case: Threat potential assessment of power management related data leaks

Alhtough the research community has increased their focus on side and covert channel attacks, several challenges remain, e. g. high required investement of resources for reproducible, comparable and exhaustive analyses. Furthermore, while many data leaks have been discovered in recent years, little attention has been given to security implications of the power management in multicore systems. In this thesis, we attempt to solve these challenges and investigate the threat potential of power management related data leaks in multicore systems.

Thesis

Poster

@phdthesis{miedl2020assessment,
  title={{Threat potential assessment of power management related data leaks}},
  author={Philipp Miedl},
  year={2020},
  school={ETH Zurich}
}