From 0d3775eb8a02f2c40568a51f0997230b750e27ef Mon Sep 17 00:00:00 2001 From: Adriana Draghici Date: Fri, 12 Feb 2010 00:30:58 +0200 Subject: [PATCH] paper- final --- report/database.tex | 2 +- report/gui.tex | 28 ++++++++++++++++------------ report/intro.tex | 12 +++++------- report/logarch.tex | 5 ++--- report/logarch/bt.tex | 8 ++++---- report/logarch/clients.tex | 2 +- report/test.tex | 15 +++++++-------- report/test/comm.tex | 2 +- 8 files changed, 37 insertions(+), 37 deletions(-) diff --git a/report/database.tex b/report/database.tex index 48a2058..77d3a3a 100644 --- a/report/database.tex +++ b/report/database.tex @@ -2,7 +2,7 @@ In our infrastructure the database stores all the parsed logging data. For its i \begin{figure}[h] \begin{center} -\includegraphics[width = 2.2in, height = 4in]{img/DB.png} +\includegraphics[width = 3.2in, height = 6in]{img/DB.png} \end{center} \caption{Database Schema for BitTorrent Experiments} \label{db} diff --git a/report/gui.tex b/report/gui.tex index e3b21da..97eb266 100644 --- a/report/gui.tex +++ b/report/gui.tex @@ -1,4 +1,4 @@ -Once we have collected all the logging and verbose data from the experiment, the next step will be to analyze it. Our testing infrastructure provides a graphical viewer tool for inspecting peer behavior. +Once we have collected all the logging and verbose data from the experiment, the next step will be to analyze it. Our testing infrastructure provides a graphical viewer tool for inspecting peer behavior. \\ The viewer is implemented in python with the help of two libraries: \textit{matplotlib} - for plotting graphs and \textit{TraitsUi} - for handling the widgets. It offers several important plotting options which show the user how a peer behaved and how it interacted with other peers during the experiment: \begin{itemize} @@ -7,7 +7,15 @@ The viewer is implemented in python with the help of two libraries: \textit{matp \item statistics - shows what type and amount of messages the peer exchanged with other peers \end{itemize} -The last two options are very important for our project because they provide valuable information about the performance of the BitTorrent client, with respect to live streaming, and how this performance is influenced by the messages that the BitTorrent client exchanges. The acceleration option shows how fast a BitTorrent client is able to download data at the beginning of the torrent. This is a basic requirement in live streaming, because it basically means to start playback of a torrent file with no delays. The statistics option shows what flow of messages makes a BitTorrent client have a fast acceleration. Using our viewer, we discovered in our experiments that the hrktorrent BitTorrent client has a really fast acceleration. This makes it very suitable for live streaming. We will discuss the experimental results in more detail in chapter [TODO]. +\begin{figure}[h] +\begin{center} +\includegraphics[width = 6.0in, height = 3.5in]{img/gui1.png} +\end{center} +\caption{Viewer in Single Client Mode} +\label{gui1} +\end{figure} + +The last two options are very important for our project because they provide valuable information about the performance of the BitTorrent client, with respect to live streaming, and how this performance is influenced by the messages that the BitTorrent client exchanges. The acceleration option shows how fast a BitTorrent client is able to download data at the beginning of the torrent. This is a basic requirement in live streaming, because it basically means to start playback of a torrent file with no delays. The statistics option shows what flow of messages makes a BitTorrent client have a fast acceleration. Using our viewer, we discovered in our experiments that the hrktorrent BitTorrent client has a really fast acceleration. This makes it very suitable for live streaming.\\ The viewer can also be used in two modes: \begin{enumerate} @@ -15,17 +23,8 @@ The viewer can also be used in two modes: \item Client Comparison Mode \end{enumerate} -In ``Single Client Mode'' the user can observe how a peer behaved during the streaming experiment. All the plotting options are available in this mode. A simple example of this mode can be seen in Figure \ref{gui1} . The user selects in right panel the database file for an experiment, a peer id and the plotting options. In the right panel the user receives the plotted graph for the peers. +In ``Single Client Mode'' the user can observe how a peer behaved during the streaming experiment. All the plotting options are available in this mode. A simple example of this mode can be seen in Figure \ref{gui1} . The user selects in right panel the database file for an experiment, a peer id and the plotting options. In the right panel the user receives the plotted graph for the peers.\\ -In ``Client Comparison Mode'' the user can compare the behavior of two peers from the experiment. An example of this mode can be seen in Figure \ref{gui2}. All plotting options are available, except the statistics options. The user selects in right panel the database file for an experiment, two peer ids and the plotting options. In the right panel the user receives plotted graphs for the selected peers. - -\begin{figure}[h] -\begin{center} -\includegraphics[width = 6.0in, height = 3.5in]{img/gui1.png} -\end{center} -\caption{Viewer in Single Client Mode} -\label{gui1} -\end{figure} \begin{figure}[h] \begin{center} \includegraphics[width = 6.0in, height = 3.5in]{img/gui2.png} @@ -33,3 +32,8 @@ In ``Client Comparison Mode'' the user can compare the behavior of two peers fro \caption{Viewer in Client Comparison Mode} \label{gui2} \end{figure} + +In ``Client Comparison Mode'' the user can compare the behavior of two peers from the experiment. An example of this mode can be seen in Figure \ref{gui2}. All plotting options are available, except the statistics options. The user selects in right panel the database file for an experiment, two peer ids and the plotting options. In the right panel the user receives plotted graphs for the selected peers. + + + diff --git a/report/intro.tex b/report/intro.tex index b71a5d7..b047f8e 100644 --- a/report/intro.tex +++ b/report/intro.tex @@ -1,7 +1,7 @@ -Peer-to-Peer systems have become a very popular type of distributed systems, and one -protocol that had a great impact upon this raise in popularity is BitTorrent, a file-sharing protocol, responsible for a large +Peer-to-Peer systems have become a very popular type of distributed systems, and one file-sharing +protocol that had a great impact upon this raise in popularity is BitTorrent, responsible for a large amount of the internet traffic (approximately 27-55\% \cite{bt_traffic}). This protocol relies on small meta-files with the extension .torrent, that contain metdata about the shared files and the addresses of one or more trackers. These trackers maintain the up-to-date lists of available peers while BitTorrent indexes represent the lists of available .torrent files. The client can connect to a tracker specified in the torrent file and then receive the list of peers, or it can connect directly to peers @@ -11,11 +11,10 @@ BitTorrent provides a dynamic and hard to predict environment in which the downl the peers involved in transfers, by the network bandwidth, limitations and topology and also on the type of implementations (clients) used. This project's aim is to evaluate more deeply the factors that influence transfers' performance and to integrate our results in a larger project, P2P-Next\cite{p2p-next}. Involving 21 partners in 12 different countries, P2P-Next's goal is to create -a new content delivery platform based on peer-to-peer systems that will also allow free livestreaming. \\ +a new content delivery platform based on peer-to-peer systems that will also allow free live-streaming. \\ In the current stage of our research we designed and implemented modules that will allow and simplify the file transfers analysis, -and will provide key information in evaluating swarms behavior. Therefore, we have split our work into to main areas: log handling and -test infrastructure for performing file transfers experiments. +and will provide key information in evaluating swarms behavior. Therefore, we have split our work into the following main areas: log handling, test infrastructure for performing file transfers experiments and viewing their results. \paragraph{Outline} @@ -23,5 +22,4 @@ test infrastructure for performing file transfers experiments. The remainder of this article is organized as follows. Section~\ref{logging} describes the first part of our work, the logging infrastructure while Section~\ref{test} presents the enhancements to the existing testing system. -Our results are described in Section~\ref{results}. -Finally, Section~\ref{conclusions} gives the conclusions and plans for further development. +Finally, Section~\ref{conclusions} gives our conclusions, results and plans for further development. diff --git a/report/logarch.tex b/report/logarch.tex index 93bb5e9..28afc82 100644 --- a/report/logarch.tex +++ b/report/logarch.tex @@ -2,8 +2,7 @@ The previous stages of this project focused on creating a virtualized testing environment \cite{rdenv} and examining the performances of several BitTorrent clients, in order to choose the most appropriate -ones for a live streaming peer-to-peer infrastructure. For each experimental session the status -and logging information was gathered and stored but not parsed or analysed in any way. The automated testing environment was monitored using MonALISA \cite{mon}. The monitored information included system parameters like cpu load, wan traffic, ethernet interfaces traffic, number of connections and was stored in a farm corresponding to out project.\\ +ones for a live streaming peer-to-peer infrastructure. For each experimental session the logging information was gathered and stored but not parsed or analyzed in any way. The automated testing environment was monitored using MonALISA \cite{mon}. The monitored information included system parameters like CPU load, WAN traffic, Ethernet interfaces traffic, number of connections, and was stored in a farm corresponding to our project.\\ We propose a new system for storing and analysing the logging data, that does not include MonALISA monitoring facilities, but a local database with a schema specific to our project. The database schema used by MonALISA to store the monitored information @@ -12,7 +11,7 @@ did not have the format necessary to our logging data. Our system includes sever \begin{center} \includegraphics[width = 6.6in, height = 2.0in]{img/Logarch.png} \end{center} -\caption{} +\caption{Logging System Overview} \label{fig:logarch} \end{figure} diff --git a/report/logarch/bt.tex b/report/logarch/bt.tex index 4010833..9f88db2 100644 --- a/report/logarch/bt.tex +++ b/report/logarch/bt.tex @@ -1,7 +1,7 @@ -The BitTorrent protocol defines the followingg messages, as described in \cite{bt} +The BitTorrent protocol defines the following messages, as described in \cite{bt} \begin{itemize} -\item BT\_HANDSHAKE $$ - first message transmited by the client as the initiator of a connection +\item BT\_HANDSHAKE $$ - first message transmitted by the client as the initiator of a connection \item BT\_CHOKE $$ \item BT\_UNCHOKE $$ \item BT\_INTERESTED $$ @@ -27,5 +27,5 @@ The BitTorrent protocol defines the followingg messages, as described in \cite{b \end{itemize} -From the messages decribed above, the most important for future analysis are those related to choking, requesting, receiving -and canceling pieces. They provide information about the behaviour of the certain peers, and might be useful also in predictions. Moreover, the performance details of the tested client can be retrived from inspecting the occurances of this messages. +From the messages described above, the most important for future analysis are those related to choking, requesting, receiving +and canceling pieces. They provide information about the behavior of certain peers, and might be useful also in predictions. Moreover, the performance details of the tested client can be retrived from inspecting the occurrances of this messages. diff --git a/report/logarch/clients.tex b/report/logarch/clients.tex index 4edae5b..b5b051e 100644 --- a/report/logarch/clients.tex +++ b/report/logarch/clients.tex @@ -26,7 +26,7 @@ When talking about logs, we differentiate between two categories: status logs an The command line interface mode is implemented by the file \textit{Tools/cmdlined.py}. In its original form, its does not output all the information need for status messages, lacking timestamp, number of peers and eta. It was modified to write the necessary information in a file. Additionally, \textit{Core/APIImplementation/SingleDownload.py} file now writes status information like file path, torrent name at the start-up/end of a file download.\\ \textbf{Verbose data} consists of information about BitTorrent protocol messages exchanged between peers during transfers. The sources -dealing with protocol implementation are located in the \textit{Core/} directory of the Tribler code, and lack in documentation, comments. After identifying the files that need to be modified several bash scripts were created in order to automate the process of changing them. These scripts are as follows: +dealing with protocol implementation are located in the \textit{Core/} directory of the Tribler code, and lack in documentation, comments. After identifying the files that needed to be modified several bash scripts were created in order to automate the process of changing them. These scripts are as follows: \begin{enumerate} \item modify\_all\_print.sh - add time and date information in all \textit{print} instructions, in all the source files. diff --git a/report/test.tex b/report/test.tex index 5a4859a..9c04f5e 100644 --- a/report/test.tex +++ b/report/test.tex @@ -1,21 +1,20 @@ -Figure \ref{genarch} presents a general overview of the old BitTorrent performance testing infrastructure. The infrastructure consists of commodity hard -ware systems running GNU/Linux. Each system uses OpenVZ virtual machine implementation to run multiple virtual systems on the same hardware node. +Figure \ref{genarch} presents a general overview of the old BitTorrent performance testing infrastructure. The infrastructure consists of commodity hardware systems running GNU/Linux. Each system uses OpenVZ virtual machine implementation to run multiple virtual systems on the same hardware node. \begin{figure}[h] \begin{center} -\includegraphics[width = 6.0in, height = 6in]{img/general-arch.png} +\includegraphics[width = 4.7in, height = 4.0in]{img/general-arch.png} \end{center} \caption{Overall Architecture of BitTorrent Performance Evaluation Infrastructure} \label{genarch} \end{figure} -Each virtual machine contains the basic tools for running and compiling BitTorrent clients. The used clients are instrumented which means that they output status and logging information required for subsequent analysis and result interpretation. As the infrastructure aims to be generic among different client implementations, the addition of a new type of BitTorrent client resumes only at adding the required scripts and instrumentation. +Each virtual machine contains the basic tools for running and compiling BitTorrent clients. The used clients are instrumented which means that they output status and logging information required for subsequent analysis and result interpretation. As the infrastructure aims to be generic among different client implementations, the addition of a new type of BitTorrent client resumes only at adding the required scripts and instrumentation.\\ -Communication with the virtual machines is enabled through the use of DNAT and iptables. TC (trafic control) is used for controlling the virtual links between virtual systems. +Communication with the virtual machines is enabled through the use of DNAT and iptables. TC (traffic control) is used for controlling the virtual links between virtual systems.\\ -The old infrastructure also uses for each virtual machine a set of scripts to enable starting, configuration, stopping and result gathering for BitTorrent clients. A test/command system starts a series of clients automatically through the use of a scripted interface. The command system uses SSH to connect to the virtual machines (SSH is installed and communication is enabled through DNAT/iptables) and command the BitTorrent implementations. The SSH connection uses the virtual machine local scripts to configure and start the clients. +The old infrastructure also uses for each virtual machine a set of scripts to enable starting, configuration, stopping and result gathering for BitTorrent clients. A test/command system starts a series of clients automatically through the use of a scripted interface. The command system uses SSH to connect to the virtual machines (SSH is installed and communication is enabled through DNAT/iptables) and to command the BitTorrent implementations. The SSH connection uses the virtual machine local scripts to configure and start the clients.\\ -As we can see the old infrastructure relies heavily on the scripted interface and static configurations. Although this works fairly well it has some major drawbacks: it is very hard to maintain and it doesn't scale well. To resolve this issues we proposed two solutions: +As we can see, the old infrastructure relies heavily on the scripted interface and static configurations. Although this works fairly well it has some major drawbacks: it is very hard to maintain and it doesn't scale well. To resolve this issues we proposed two solutions: \begin{itemize} \item use a client-server architecture for interaction with virtual machines \item use global configuration files @@ -35,7 +34,7 @@ These files are parsed by the commander at startup. The new testing infrastructure uses a client-server type of interaction, in which a commander daemon running on command system sends commands to a server daemon running on the virtual machines. -\subsubsection{Communication protocol} +\subsubsection{Communication protocol}\label{comm_prot} The Commander communicates with the Server using the following types of messages: \begin{itemize} diff --git a/report/test/comm.tex b/report/test/comm.tex index f9a1e69..54457df 100644 --- a/report/test/comm.tex +++ b/report/test/comm.tex @@ -4,5 +4,5 @@ The commander daemon uses both SSH and sockets for communication and performs se \item applies traffic control rules upon hardware nodes using \textit{TrafficControl.py} script. The traffic control rules are applied using SSH commands on the system. \item starts the Server daemon using SSH on the described nodes(OpenVZ containers). In order to connect, the commander uses the public IP address of the hardware node and the port. -\item monitors the nodes using the server daemon. It interacts using the messages described in section [to do section protocol]. A special port is assigned to this communication. +\item monitors the nodes using the server daemon. It interacts using the messages described in section \ref{comm_prot}. A special port is assigned to this communication. \end{enumerate} \ No newline at end of file -- 2.20.1