+++ /dev/null
-BASE = report
-TEX_FILE = $(BASE).tex
-PDF_FILE = $(TEX_FILE:.tex=.pdf)
-PDFLATEX = pdflatex
-BIBTEX = bibtex
-
-$(PDF_FILE): $(BASE).tex
- $(PDFLATEX) $<
- $(BIBTEX) $(BASE)
- $(PDFLATEX) $<
- $(PDFLATEX) $<
- rm -f *~ *.aux *.log *.blg *.bbl
-
-clean:
- -rm -f *~ *.aux *.log *.blg *.bbl
- -rm -f $(PDF_FILE) *~
+++ /dev/null
-
-
-During the last few years, BitTorrent has become one of the most heavily used protocol on the Internet.
-Our current project is focused on evaluation of the behavior and performance of BitTorrent implementations and on the possible
-improvements to these applications. This report will present the tools we developed in order to automate and
-improve our research about BitTorrent swarms. The modular and extensible design of these tools create
-a fresh starting point for an extensive logging information analysis and testing architecture.
+++ /dev/null
-This report presents the development of an environment suitable for generating BitTorrent file-sharing experiments and evaluating
-them. We created an easily deployable solution that enables automated testing of different BitTorrent
-clients in real world situations. This solution involves building virtualization environments and controlling the BitTorrent clients
-using a Client-Server architecture, with a Commander that controls the nodes involved in an experiment by communicating with a Server
-installed on these remote systems. Two clients were considered for further study on transfer performance: Tribler and libtorrent/hrktorrent. The first one had to be modified in order to output the necessary logging information. Furthemore, a status messages parser and a verbose messages parser were developed for each client. The parsed data is stored in a newly designed sqlite database and a graphical interface application provides visualization using graphs and other similar structures.\\
-
-One of the next steps in our project is to perform extensive testing using files of different types, sizes and number of peers. The resulted logs would then be compared and anaylized.
-
-\subsection{Experimental Results}\label{results}
-\input{results.tex}
-
+++ /dev/null
-In our infrastructure the database stores all the parsed logging data. For its implementation we choose sqlite because it provides an easy and fast way to manage a database: the entire database is stored as a single file. During the experiment all the peers store their status and verbose information in a local sqlite database file. All the files eventually get merged to form our database.
-
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 3.2in, height = 6in]{img/DB.png}
-\end{center}
-\caption{Database Schema for BitTorrent Experiments}
-\label{db}
-\end{figure}
-
-In Figure \ref{db} we have a detailed look on the database schema. The tables in the database store the following information:
-\begin{itemize}
- \item swarms: information about a BitTorrent session(swarm)
- \item btclients: information about the tested the BitTorrent clients
- \item client\_sessions: client sessions information
- \item status\_messages: client status information extracted from status log files
- \item verbose\_messages: information about sent/received BitTorrent messages; extracted from verbose log files
-\end{itemize}
-
-
-\subsubsection{Database Helper Objects}
-
-Our infrastructure also provides database helper objects to make database operations easier:
-\begin{itemize}
- \item db\_init: a simple shell script that creates a SQLite BitTorrent message logging database file.
- \item DatabaseAccess.py:
- \begin{itemize}
- \item implements low-level Python operations for managing a BitTorrent message logging database (created through the use of the db\_init script)
- \item exports methods allowing insertion, selection and deletion of table entries in the database. This methods are directly used by the DatabaseCommander.py and DatabaseWriter.py objects.
- \item can be called from the command line. This enables a small test case of the exported functions and fills the database initial entries.
- \end{itemize}
- \item DatabaseCommander.py:
-\begin{itemize}
- \item provides command line access to and interaction with the BitTorrent information database.
- \item allows adding, deleting and showing entries in the swarms and client\_sessions table.
-\end{itemize}
- \item DatabaseWriter.py:
-\begin{itemize}
- \item enables access to the status\_messages and verbose\_messages tables.
- \item offers a programmer interface which exports six methods for adding, deleting and listing entries.
- \item offers a command line interface similar to the one belonging to the DatabaseCommander.
-\end{itemize}
-\end{itemize}
-
-
+++ /dev/null
-Once we have collected all the logging and verbose data from the experiment, the next step will be to analyze it. Our testing infrastructure provides a graphical viewer tool for inspecting peer behavior. \\
-
-The viewer is implemented in python with the help of two libraries: \textit{matplotlib} - for plotting graphs and \textit{TraitsUi} - for handling the widgets. It offers several important plotting options which show the user how a peer behaved and how it interacted with other peers during the experiment:
- \begin{itemize}
- \item download/upload speed - shows the evolution of download/upload speed for the peer
- \item acceleration - shows how fast the download/upload speed of the peer increases/decreases
- \item statistics - shows what type and amount of messages the peer exchanged with other peers
- \end{itemize}
-
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 6.0in, height = 3.5in]{img/gui1.png}
-\end{center}
-\caption{Viewer in Single Client Mode}
-\label{gui1}
-\end{figure}
-
-The last two options are very important for our project because they provide valuable information about the performance of the BitTorrent client, with respect to live streaming, and how this performance is influenced by the messages that the BitTorrent client exchanges. The acceleration option shows how fast a BitTorrent client is able to download data at the beginning of the torrent. This is a basic requirement in live streaming, because it basically means to start playback of a torrent file with no delays. The statistics option shows what flow of messages makes a BitTorrent client have a fast acceleration. Using our viewer, we discovered in our experiments that the hrktorrent BitTorrent client has a really fast acceleration. This makes it very suitable for live streaming.\\
-
-The viewer can also be used in two modes:
-\begin{enumerate}
- \item Single Client Mode
- \item Client Comparison Mode
-\end{enumerate}
-
-In ``Single Client Mode'' the user can observe how a peer behaved during the streaming experiment. All the plotting options are available in this mode. A simple example of this mode can be seen in Figure \ref{gui1} . The user selects in right panel the database file for an experiment, a peer id and the plotting options. In the right panel the user receives the plotted graph for the peers.\\
-
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 6.0in, height = 3.5in]{img/gui2.png}
-\end{center}
-\caption{Viewer in Client Comparison Mode}
-\label{gui2}
-\end{figure}
-
-In ``Client Comparison Mode'' the user can compare the behavior of two peers from the experiment. An example of this mode can be seen in Figure \ref{gui2}. All plotting options are available, except the statistics options. The user selects in right panel the database file for an experiment, two peer ids and the plotting options. In the right panel the user receives plotted graphs for the selected peers.
-
-
-
+++ /dev/null
-
-
-Peer-to-Peer systems have become a very popular type of distributed systems, and one file-sharing
-protocol that had a great impact upon this raise in popularity is BitTorrent, responsible for a large
-amount of the internet traffic (approximately 27-55\% \cite{bt_traffic}). This protocol relies on small meta-files with the extension .torrent, that contain metdata about the shared files and the addresses of one or more trackers. These trackers maintain
-the up-to-date lists of available peers while BitTorrent indexes represent the lists of available .torrent files. The client
-can connect to a tracker specified in the torrent file and then receive the list of peers, or it can connect directly to peers
-using a trackerless system: DHT (Distributed Hash Table) and PEX(Peer EXchange).\\
-
-BitTorrent provides a dynamic and hard to predict environment in which the download performance is determined by
-the peers involved in transfers, by the network bandwidth, limitations and topology and also on the type of implementations (clients) used.
-This project's aim is to evaluate more deeply the factors that influence transfers' performance and to integrate our results
-in a larger project, P2P-Next\cite{p2p-next}. Involving 21 partners in 12 different countries, P2P-Next's goal is to create
-a new content delivery platform based on peer-to-peer systems that will also allow free live-streaming. \\
-
-In the current stage of our research we designed and implemented modules that will allow and simplify the file transfers analysis,
-and will provide key information in evaluating swarms behavior. Therefore, we have split our work into the following main areas: log handling, test infrastructure for performing file transfers experiments and viewing their results.
-
-
-\paragraph{Outline}
-
-The remainder of this article is organized as follows.
-Section~\ref{logging} describes the first part of our work, the logging infrastructure while
-Section~\ref{test} presents the enhancements to the existing testing system.
-Finally, Section~\ref{conclusions} gives our conclusions, results and plans for further development.
+++ /dev/null
-
-
-The previous stages of this project focused on creating a virtualized testing environment \cite{rdenv}
-and examining the performances of several BitTorrent clients, in order to choose the most appropriate
-ones for a live streaming peer-to-peer infrastructure. For each experimental session the logging information was gathered and stored but not parsed or analyzed in any way. The automated testing environment was monitored using MonALISA \cite{mon}. The monitored information included system parameters like CPU load, WAN traffic, Ethernet interfaces traffic, number of connections, and was stored in a farm corresponding to our project.\\
-
-We propose a new system for storing and analysing the logging data, that does not include MonALISA monitoring facilities, but
-a local database with a schema specific to our project. The database schema used by MonALISA to store the monitored information
-did not have the format necessary to our logging data. Our system includes several components, as depicted in figure \ref{fig:logarch}.
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 6.6in, height = 2.0in]{img/Logarch.png}
-\end{center}
-\caption{Logging System Overview}
-\label{fig:logarch}
-\end{figure}
-
-The following sections describe the details of the components we developed.
-
-\subsection{BitTorrent Messages}
-\input{logarch/bt.tex}
-\subsection{BitTorrent Clients Instrumentation}
-\input{logarch/clients.tex}
-\subsection{Parsers}
-\input{logarch/parser.tex}
-\subsection{Database}\label{database}
-\input{database.tex}
-\subsection{GUI}
-\input{gui.tex}
+++ /dev/null
-
-The BitTorrent protocol defines the following messages, as described in \cite{bt}
-\begin{itemize}
-\item BT\_HANDSHAKE $<pstrlen><pstr><reserved><info_hash><peer_id>$ - first message transmitted by the client as the initiator of a connection
-\item BT\_CHOKE $<len=0001><id=0>$
-\item BT\_UNCHOKE $<len=0001><id=1>$
-\item BT\_INTERESTED $<len=0001><id=2>$
-\item BT\_UNINTERESTED $<len=0001><id=3>$
-\item BT\_BITFIELD $<len=0001+X><id=5><bitfield>$ - The payload is a bitfield representing the pieces that have been successfully downloaded
-\item BT\_REQUEST $<len=0013><id=6><index><begin><length>$ - used to request a block
-\begin{itemize}
-\item index: integer specifying the zero-based piece index
-\item begin: integer specifying the zero-based byte offset within the piece
-\item length: integer specifying the requested length.
-\end{itemize}
-
-\item BT\_PIECE $<len=0009+X><id=7><index><begin><block>$
-\begin{itemize}
-\item index: integer specifying the zero-based piece index
-\item begin: integer specifying the zero-based byte offset within the piece
-\item block: block of data, which is a subset of the piece specified by index.
-\end{itemize}
-\item BT\_HAVE $<len=0005><id=4><piece index>$ - inform of having a certain piece
-\item BT\_CANCEL $<len=0013><id=8><index><begin><length>$ - used to cancel block requests
-\item BT\_KEEP\_ALIVE $<len=0000>$ - must be sent to maintain the connection alive if no command have been sent for a given amount of time
-\item BT\_PORT (DHT tracker) $<len=0003><id=9><listen-port>$
-\end{itemize}
-
-
-From the messages described above, the most important for future analysis are those related to choking, requesting, receiving
-and canceling pieces. They provide information about the behavior of certain peers, and might be useful also in predictions. Moreover, the performance details of the tested client can be retrived from inspecting the occurrances of this messages.
+++ /dev/null
-The log files analysis takes into consideration two open-source implementations of the BitTorrent protocol.
-The experiments prior to our research evaluated the performances of several BitTorrent clients that offered a command line interface (CLI), as described in \cite{rdeval}. Based on these results we have chosen \textbf{rasterbar libtorrent},
-which in the comparison proved to be the fastest solution, and \textbf{Tribler}.\\
-
-
-Libtorrent, developed by Rasterbar \cite{libt2} \cite{libt1}, is a C++ library that implements BitTorrent protocol and
-its extensions. It aims to be CPU and memory efficient, easy to use and suitable for many environments and platforms.
-Due to its high performance it is used by many BitTorrent applications, like FireTorrent, Deluge, qBitTorrent, SharkTorrent,
-Miro and Hrktorrent \cite{hrk}, the one we chose to work with. Hrktorrent is a lightweight torrent client written in C++ and it was appropriate for our testing infrastructure because it offered a command line interface.\\
-
-Tribler \cite{tribler}, developed at the Delft University of Technology, is a BitTorrent client written in Python, aimed to enhance the user experience by offering features like online video streaming and content search. Unlike hrktorrent, Tribler is a GUI oriented client, as well providing a command line interface. It also represents a key technology in the P2P-Next project
-because it allows watching BitTorrent-hosted peer-to-peer digital media distribution of video on demand and plays live Tribler streaming media \cite{tribler_wiki}. \\
-
-%[todo : ce versiuni de libtorrent si hrktorrent folosim?]
-
-In what concerns logging, libtorrent/hrktorrent provided extensive logging information\cite{rdeval}, proving after the comparison experiments large amounts of output that could be directly used as an example for our parsing system.
-Tribler did not offer any logging information. Therefore, we dedicated part of our research time to
-modifying Tribler source code (version 5.1.2) in order to obtain the necessary logs.
-
-\subsubsection{Changes in Tribler source code}
-
-
-When talking about logs, we differentiate between two categories: status logs and verbose logs.
-
-\textbf{Status logs} contain information about current file transfers, like download/upload speed, eta (estimated time of arrival). This information is given by the Tribler's core classes, that deal with connections, download, upload.
-The command line interface mode is implemented by the file \textit{Tools/cmdlined.py}. In its original form, its does not output all the information need for status messages, lacking timestamp, number of peers and eta. It was modified to write the necessary information in a file. Additionally, \textit{Core/APIImplementation/SingleDownload.py} file now writes status information like file path, torrent name at the start-up/end of a file download.\\
-
-\textbf{Verbose data} consists of information about BitTorrent protocol messages exchanged between peers during transfers. The sources
-dealing with protocol implementation are located in the \textit{Core/} directory of the Tribler code, and lack in documentation, comments. After identifying the files that needed to be modified several bash scripts were created in order to automate the process of changing them. These scripts are as follows:
-
-\begin{enumerate}
-\item modify\_all\_print.sh - add time and date information in all \textit{print} instructions, in all the source files.
-\item print\_timestamp.sh - add time and date information in all \textit{print} instructions, in the files listed in a file given as argument.
-\item set\_DEBUG.sh - sets DEBUG flag to a given value, in the files listed in file given as argument.
-\item set\_DEBUG\_NORMAL.sh - sets DEBUG\_NORMAL flag to a given value, in the files listed in file given as argument.
-\item modify\_Connecter.sh - redirect print instructions to stderr in
-
-\textit{Core/BitTornado/BT1/Connecter.py}
-\item make\_changes.sh - uses the above described scripts to make all the necessary changes
-\item undo\_changes.sh - uses the above described scripts to undo all the changes made, except the redirection of prints in \textit{Connecter.py}
-\end{enumerate}
-
-In addition to these scripts, two ASCII files contain the list of files to be modified, each file on a separate line.
-
-\begin{itemize}
- \item modified\_files\_DEBUG
- \item modified\_files\_DEBUG\_NORMAL
-\end{itemize}
-
-Almost all the source files contain the DEBUG flag that is set to True when print statements are enables, false otherwise.
-All the prints statements for verbose information output it to stderr, while all status information is written to a status file,
-\textit{status\_msg.log}. Unlike libtorrent, which kept separate log files for each peer, Tribler verbose logs are kept
-in one file.\\
-
-The modified Tribler code and the scripts can be found in \cite{repo}.
-
-
-
-
-
+++ /dev/null
-
-In order to fill the database's \textit{status\_messages} and \textit{verbose\_messages tables},
-the log files need to be parsed. For each BitTorrent client we implemented in Python two parsers for status and
-verbose information, which use the database communication methods discussed in section \ref{database} to save
-the parsed data in the database.\\
-
-All these scripts receive as arguments a client id corresponding to a row in \textit{clients} table, the log file
-and the database. An instance of DatabaseCommander checks the \textit{client\_id} and an instance of DatabaseWriter based on the \textit{database} argument is used to write the messages information (timestamp, message type etc).\\
-
-The verbose parser searches a given log for the following bit torrent messages taking into consideration
-the direction of the message, that indicates if it is received or send from/to a peer. The following table
-compares the type of messages output by the clients. Libtorrent log parser has support for parsing messages
-for both directions, because of the format used to display this information. Tribler prints these messages
-from several files, using different formats, and in our research of its code we could not find output in both directions
-for all the message types.
-\\
-
-\begin{tabular}{|l|c|r|}
- \hline
- \textbf{BT Message} & \textbf{Tribler} & \textbf{Libtorrent} \\
- \hline
-BT\_REQUEST & both directions & both directions\\
-\hline
-BT\_CHOKE &receive & both directions\\
-\hline
-BT\_UNCHOKE & receive & both directions\\
-\hline
-BT\_HAVE & receive & both directions\\
-\hline
-BT\_PIECE & receive & both directions\\
-\hline
-BT\_BITFIELD & receive & both directions\\
-\hline
-BT\_CANCEL & send & none\\
-\hline
-BT\_INTERESTED & receive & both directions\\
-\hline
-BT\_NOT\_INTERESTED & none & both directions\\
-\hline
-BT\_DHT\_PORT & none & both directions\\
-\hline
-\end{tabular} \label{Table1}
-
-
-
-\subsubsection {Tribler logs parsing}
-
-Both parsers analyze line by line the log files and check for certain formats. When a line corresponds
-to a message, it is split and its parts are checked and together with the client id are given as arguments to the functions from DatabaseWriter that add status/verbose messages.
-
-Example of a status line (shown here split into two lines):
-\begin{verbatim}
- 03-Nov-2009 12:18:55 aqua.mpeg DLSTATUS_DOWNLOADING 29.84%
- None up 0.00KB/s down 4414.39KB/s eta 12 peers 2
-\end{verbatim}
-
-
-
-Examples of protocol message lines:
-\begin{verbatim}
- 20-10-2009 12:56:39 Downloader: new_request 52 98304 16384 to 141.85.37.41 14398
- 27-11-2009 18:01:22 connecter: Got REQUEST( 1218 ) from 87.0.15.75
- 14-11-2009 23:11:13 connecter: Got PIECE( 141 ) from 141.85.37.41
- 14-11-2009 23:11:24 sent cancel: 130: 114688-131072
-\end{verbatim}
-
-Due to the many types of messages and their high frequency of being transmitted, the verbose log file can get to
-large sizes of hundreds of MB, even a few GB. Therefore parsing such files demands much time than parsing the status log,
-on our test files, getting to 20-30 minutes for logs of approximately 300MB on dual core 2GHz processors, 2G RAM.
-
-
-\subsubsection {Libtorrent logs parsing}
-The parsers have a similar structure to those for Tribler, differing in the way the message lines are analysed.
-
-Example of a status line :
-\begin {verbatim}
- ps: 1, dht: 8 <> dl: 119.51kb/s, ul: 3.63kb/s <> dld: 1mb, uld: 0mb, size: 698mb <> eta: 1h 39m 37s
-\end{verbatim}
-
-Samples of protocol message lines:
-\begin {verbatim}
-Jan 08 22:39:50 <== REQUEST [ piece: 6cc | s: 14000 | l: 4000 ]
-Jan 08 22:39:50 ==> PIECE [ piece: 5c6 | s: 24000 | l: 4000 ]
-\end{verbatim}
-
-In libtorrent the messages are output a lot clearer than in tribler, also there are separate log files for each peer.
-The $==>$ and $<==$ strings indicate the direction of the message: received or sent.
-
-
-
-
+++ /dev/null
-@misc{rdeval,
- title = {A BitTorrent Performance Evaluation Framework},
- author = {Razvan Deaconescu and Razvan Rughinis and Nicolae Tapus},
- year = {2009}
-}
-
-@misc{rdenv,
- title = {A Virtualized Testing Environment for BitTorrent Applications},
- author = {Razvan Deaconescu and Razvan Rughinis and Nicolae Tapus},
- year = {2009}
-}
-
-@misc{rdeval2,
- title = {A BitTorrent Performance Evaluation Framework},
- author = {Razvan Deaconescu and Razvan Rughinis and Nicolae Tapus},
- year = {2009}
-}
-@misc{libt1,
- title = {http://www.rasterbar.com/products/libtorrent/}
- }
-@misc{libt2,
- title = {http://libtorrent.rakshasa.no/}
-}
-@misc{hrk,
- title = {http://50hz.ws/hrktorrent/}
-}
-@misc{tribler,
- title = {http://www.tribler.org/trac/wiki}
-}
-@misc{tribler_wiki,
- title = {http://en.wikipedia.org/wiki/Tribler}
-}
-@misc{repo,
-title={http://koala.cs.pub.ro/git/?p=cs-p2p-next.git/.git;a=tree;f=tribler-mod;h=d8516ffa338b44dcb3d6c83b9d498481c4415eb9;hb=HEAD}
-}
-
-@misc{bt,
-title = {http://wiki.theory.org/BitTorrentSpecification}
-}
-
-@misc{daemon1,
-title = {http://www.python.org/dev/peps/pep-3143/correct-daemon-behaviour}
-}
-@misc{daemon2,
-title = {http://www.jejik.com/files/examples/daemon.py}
-}
-@misc{pickle,
-title = {http://docs.python.org/library/pickle.html}
-}
-@misc{mon,
-title = {http://monalisa.cern.ch/monalisa.htm}
-}
-@misc{bt_traffic,
-title= {http://www.ipoque.com/resources/internet-studies/}
-}
-@misc{dht,
-title = {http://lifehacker.com/5411311/bittorrents-future-dht-pex-and-magnet-links-explained}
-}
-
-@misc{p2p-next,
-title = {http://www.p2p-next.org/}
-}
+++ /dev/null
-\documentclass[12pt]{article}
-
-\usepackage{graphicx}
-\usepackage{verbatim}
-\usepackage{url}
-
-\usepackage[paper=a4paper, top=2cm, bottom=3cm, left=2.5cm, right=2.5cm]{geometry}
-
-\title{An Extensive Reporting, Storage and Analysis Infrastructure for BitTorrent}
-
-\author{Adriana Draghici, Marius Sandu-Popa, Razvan Deaconescu, Nicolae \c{T}apu\c{s}\\
- University Politehnica of Bucharest\\
- Computer Science and Engineering Department \\
- Splaiul Independen\c{t}ei nr. 313, Bucharest, Romania \\
- \emph{\{adriana.draghici, marius.sandupopa\}}@cti.pub.ro \\
- \emph{\{razvan.deaconescu, ntapus\}}@cs.pub.ro \\
-}
-
-\date{February 11, 2010}
-
-\begin{document}
-\maketitle
-
-\begin{abstract}
-\input{abstract.tex}
-\end{abstract}
-
-\section{Introduction}
-
-\input{intro.tex}
-
-\section{Bittorent Logging Architecture}\label{logging}
-\input{logarch.tex}
-
-\section{Virtualized Testing Infrastructure}\label{test}
-\input{test.tex}
-
-\section{Conclusions and Future work}\label{conclusions}
-
-\input{conclusions.tex}
-
-\bibliographystyle{abbrv}
-\bibliography{report}
-\end{document}
+++ /dev/null
-In the current phase of development, testing implied checking the correct execution of our modules on several samples and scenarios.
-The parsers were tested on log files of different sizes, varying from a few MB to hundreds of MB. When running our BitTorrent clients, we took into consideration various file sizes, from tens of MB to a few GB, different numbers of seeders and leachers. The database files generated by the parsers were not yet used for any performance evaluation, just for verification of parsers correct behavior.
-The Commander - Server applications were also tested for a functional execution, but not yet used in any experiment.
-Therefore, the result of our work consists of several fully-functional components, ready to be used in our future research.
+++ /dev/null
-Figure \ref{genarch} presents a general overview of the old BitTorrent performance testing infrastructure. The infrastructure consists of commodity hardware systems running GNU/Linux. Each system uses OpenVZ virtual machine implementation to run multiple virtual systems on the same hardware node.
-
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 4.7in, height = 4.0in]{img/general-arch.png}
-\end{center}
-\caption{Overall Architecture of BitTorrent Performance Evaluation Infrastructure}
-\label{genarch}
-\end{figure}
-
-Each virtual machine contains the basic tools for running and compiling BitTorrent clients. The used clients are instrumented which means that they output status and logging information required for subsequent analysis and result interpretation. As the infrastructure aims to be generic among different client implementations, the addition of a new type of BitTorrent client resumes only at adding the required scripts and instrumentation.\\
-
-Communication with the virtual machines is enabled through the use of DNAT and iptables. TC (traffic control) is used for controlling the virtual links between virtual systems.\\
-
-The old infrastructure also uses for each virtual machine a set of scripts to enable starting, configuration, stopping and result gathering for BitTorrent clients. A test/command system starts a series of clients automatically through the use of a scripted interface. The command system uses SSH to connect to the virtual machines (SSH is installed and communication is enabled through DNAT/iptables) and to command the BitTorrent implementations. The SSH connection uses the virtual machine local scripts to configure and start the clients.\\
-
-As we can see, the old infrastructure relies heavily on the scripted interface and static configurations. Although this works fairly well it has some major drawbacks: it is very hard to maintain and it doesn't scale well. To resolve this issues we proposed two solutions:
-\begin{itemize}
- \item use a client-server architecture for interaction with virtual machines
- \item use global configuration files
-\end{itemize}
-
-\subsection{Global Configuration files}
-
-All the configurations for the testing infrastructure and the BitTorrent experiment are placed in two files:
-\begin{itemize}
- \item swarm.xml: contains configuration information relevant to swarm experiment (BitTorrent client, download/upload speed limit, etc.)
- \item nodes.xml: contains configuration information for the virtual machines (public/private ip, public/private port, etc.)
-\end{itemize}
-
-These files are parsed by the commander at startup.
-
-\subsection{Client-Server interaction}
-
-The new testing infrastructure uses a client-server type of interaction, in which a commander daemon running on command system sends commands to a server daemon running on the virtual machines.
-
-\subsubsection{Communication protocol}\label{comm_prot}
-The Commander communicates with the Server using the following types of messages:
-
-\begin{itemize}
- \item START - sent for starting a BitTorrent client.
- \begin{itemize}
-\item first message contains the type, here START\_MSG
-\item second message contains a Python Dictionary data structure with the parameters
- needed for starting the client.
-\end{itemize}
- \item STOP - sent for stopping a running client.
-\begin{itemize}
-\item first message contains the type, here STOP\_MSG
-\item second message contains a string with the client process id.
- \end{itemize}
-
- \item ACK - contains $``ACK``$ and is sent by the server after receiving the message type
- \item ACK PID - contains $''ACK <PID>''$ sent by the server after starting a client, PID is the process id.
- \item ERROR - contains $"ERROR <error-message>``$ sent by the server when the parameters received are incorrect or the BT client can not be started.
-\end{itemize}
-
-In figure \ref{fig:test-arch} we can see the usual messages exchanged between the command system and server daemon on the virtual machines.\\
-
-We intend to extend this communication protocol with status messages sent by the Commander to check the BT client's progress.\\
-
-The messages with information stored in data structures are transmitted serialized using \textit{pickle} module \cite{pickle}, Python's most common method to serialize/deserialize object data.
-
-\begin{figure}[h]
-\begin{center}
-\includegraphics[width = 4.5in, height = 2.2in]{img/Diagram1.png}
-\end{center}
-\caption{Client-Server architecture for interaction with BitTorrent clients.}
-\label{fig:test-arch}
-\end{figure}
-\subsubsection{Server daemon}\label{sec:server}
-\input{test/server.tex}
-\subsubsection{Commander daemon}\label{sec:comm}
-\input{test/comm.tex}
+++ /dev/null
-The commander daemon uses both SSH and sockets for communication and performs several functions:
-\begin{enumerate}
- \item parses the file that describes the systems involved in the experiment: \textit{nodes.xml} and the file containing swarm information: \textit{swarm.xml}
-\item applies traffic control rules upon hardware nodes using \textit{TrafficControl.py} script. The traffic control rules are applied using SSH commands on the system.
-\item starts the Server daemon using SSH on the described nodes(OpenVZ containers). In order to connect, the commander
-uses the public IP address of the hardware node and the port.
-\item monitors the nodes using the server daemon. It interacts using the messages described in section \ref{comm_prot}. A special port is assigned to this communication.
-\end{enumerate}
\ No newline at end of file
+++ /dev/null
-
-We describe the chronological steps needed to run a swarm:
-
-\begin{enumerate}
- \item The Commander parses the file that describes the systems involved in the experiment: \textit{nodes.xml} and the file
- containing swarm information: \textit{swarm.xml}
-\item The Commander applies traffic control rules upon hardware nodes using \textit{TrafficControl.py} script
-\item The traffic control rules are applied using SSH commands on the system.
-\item The Commander starts the Server daemon using SSH on the described nodes(OpenVZ containers). In order to connect, the commander
-uses the public IP address of the hardware node and the port.
-\item The Commander monitors the nodes using the Server daemon. It interacts using the messages described in section [to do section protocol]. A special port is assigned to this communication.
-\end{enumerate}
+++ /dev/null
-
-The server daemon resides in a virtual machine on which several BT clients are installed. It is started by connecting remotely using ssh, and executes as a daemon process\cite{daemon1}.This server is written in Python, using modules for socket communication and process handling with functions similar to the ones from POSIX C. The daemon behavior is obtained by using an open source implementation that can be found in \cite{daemon2}.\\
-
-Using sockets, the server listens to connections initiated by the commander on a certain port, and
-after the exchange of messages is completed, it closes the connection. It also maintains a list of
-process ids for all the BT Client that it started. \\
-
-In order to start the BitTorrent client the Server needs to create a command string to pass to the function
-that creates a child process. This string contains information about the client and the data received from the
-Commander, and it is obtained using classes that extend \textit{BitTorrentClientRun}, like \textit{TransmissionRun}, \textit{TriblerRun}. These classes build the string by including static data about the client's configuration: path
-arguments' names.
-
-
-
-
-
\ No newline at end of file
projects / cs-p2p-next.git / commitdiff
