Weevil: a Distributed System Experiment Automation Tool
Overview
This research project addresses the problem of experimenting with
highly distributed systems. We use the term highly distributed
system to refer to a system capable of delivering a service to
many clients through a large number of distributed access points. A
highly distributed system usually consists of a network of components,
executing independent and possibly heterogeneous tasks, that
collectively realize a coherent service. Examples of such systems are
various forms of general-purpose communication systems,
application-level overlays, networks of caching servers, peer-to-peer
file-sharing systems, distributed databases, replicated file systems,
and distributed middleware systems in general.
One difficulty software engineers face later in distributed system
development process is how to evaluate and tune the alternative
designs and implementations. The only solution is systematic,
repeatable experimentation with executable prototypes in realistic
execution environments. Thus, many large-scale or emulated networks
come into being to provide the testbed for such large-scale
experiments. However, the large-scale and complicated nature of
large-scale network environments and distributed system usage
scenarios makes the control of such experiments difficult.
Our tool, Weevil, is to ease the difficulty in programming
distributed experiment control scripts by generating them based on
some easy-to-write configuration files. It covers a two-phase
experimentation process.
- Workload generation: A distributed system workload is a
time-ordered list of service calls from clients to system access
points. While there are many ways to create such workloads, they
are still not flexible enough and unavailable for some scenarios.
Our approach to workload generation is to perform an offline
simulation of expected client behavior and record the resulting
service calls. During an experiment run, these service calls are
issued as inputs into the subject system.
- Experiment deployment and execution: This activity
applies a workload to a system and gathers runtime data. To achieve
this, the subject system, the workload, and the framework codes are
first deployed across an experimentation testbed and then
configured. Next, the system is started and stimulated by the
execution of service calls at the times and locations dictated by
the workload. When the experiment terminates, output and diagnostic
data are collected from across the testbed to facilitate efficient
data analysis and reduction.
Weevil consists of a synthetic-workload generator coupled
with an environment for managing the deployment and execution of
experiments. Weevil is intended to facilitate experimentation
activities for distributed systems by providing engineers with a
flexible, configurable, automated and, thus, repeatable process for
evaluating their software on a networked testbed. Further, Weevil is
not targeted at a specific system or application model, but rather it
is a generic, programmable tool. This documentation describes a
prototype implementation of Weevil.
Licence and Credits
Copyright © 2004-2006 University of Colorado.
Weevil is free software. you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.
Weevil is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
People
Documents
Workload Generator Reference Manual
Experiment Environment Reference Manual
Publications
-
Yanyan Wang
Automating
Experimentation with Distributed Systems Using Generative
Techniques.
Ph.D. Thesis, Department of Computer Science, University of
Colorado. Aug. 2006.
- Yanyan Wang,
Matthew J. Rutherford,
Antonio Carzaniga,
Alexander L. Wolf,
Automating
Experimentation on Distributed Testbeds, In Proceedings of
the 20th IEEE/ACM International Conference on Automated Software
Engineering (ASE 2005), Long Beach, California, November 7-11,
2005.
- Yanyan Wang,
Antonio Carzaniga,
Alexander L. Wolf,
Four
Enhancements to Automated Distributed System Experimentation
Methods. In Proceedings of the Thirtieth International
Conference on Software Engineering (ICSE'08). Leipzig, Germany,
May 2008.
- Yanyan Wang,
Matthew J. Rutherford,
Antonio Carzaniga,
Alexander L. Wolf,
Weevil:
a Tool to Automate Experimentation with Distributed
Systems. Technical Report CU-CS-980-04. Department of
Computer Science, University of Colorado. Oct. 2004.
Acknowledgements
We extended a parallel
tool, vxargs, to allow
more arguments included in an arbitrary command (instead of only
allowing IP address as arguments in the original vxargs). We thank Yun Mao, for providing this useful tool. You can download our
extended version of vxargs here.
Feedback
Your feedbacks are very valuable to us. Please don't hesitate to send
us email
(Yanyan.Wang@colorado.edu)
if you have any problems, corrections, or suggestions. Thanks!
Copyright 2004-2006 Yanyan Wang