- The Generic Multiparty Transport Protocol (P2TP)
+ The Generic Multiparty Transport Protocol (swift)
Status of this Memo
Abstract
- This text is intended to explain inner workings of the P2TP
+ This text is intended to explain inner workings of the swift
(peer-to-peer transport protocol), which is currently work in
- progress. P2TP was devised to simplify and unify the peer-to-peer/
+ progress. swift was devised to simplify and unify the peer-to-peer/
peer-assisted/ multi-source download domain with the long-term
practical goal of embedding it into mass software and hardware.
1. Requirements notation
2. Introduction
3. Design goals
- 4. P2TP subsystems and design choices
+ 4. swift subsystems and design choices
4.1. The atomic datagram principle
4.2. Handshake and multiplexing
4.3. Data integrity and on-demand Merkle hashes
download itself. Thus, the niche for multiparty transfers expands.
Still, current, relevant technologies are tightly coupled to a
single usecase or even infrastructure of a particular corporation.
- Hence, the focus of P2TP is to find the simplest solution involving
- the minimum set of primitives, still being sufficient to implement
- all the targeted usecases (see Table 1) and suitable for use in
- general-purpose software and hardware (i.e. a web browser or a
- set-top box).
+ The mission of the project is to create a generic content-centric
+ multiparty transport protocol to allow seamless, effortless data
+ dissemination on the Net.
| mirror-based peer-assisted peer-to-peer
------+----------------------------------------------------
3. Design goals
+ The technical focus of the swift protocol is to find the simplest
+ solution involving the minimum set of primitives, still being
+ sufficient to implement all the targeted usecases (see Table 1),
+ suitable for use in general-purpose software and hardware (i.e. a
+ web browser or a set-top box).
The five design goals for the protocol are:
1. Embeddable kernel-ready protocol.
control makes sense in the general case.
-4. P2TP subsystems and design choices
+4. swift subsystems and design choices
- To large extent, P2TP design is defined by the cornerstone decision
+ To large extent, swift design is defined by the cornerstone decision
to get rid of TCP and not to reinvent any TCP-like transports on
top of UDP or otherwise. The requirements (1), (4), (5) make TCP a
bad choice due to its high per-connection footprint, complex and
4. TCP congestion control is less useful as custom congestion
control is often needed.
In general, TCP is built and optimized for a different usecase than
- we have with swarmed downloads. Thus, the choice is to make a
- UDP-based transport, possibly reserving HTTP tonneling as an
- universal fallback. Further, instead of reimplementing TCP we
- create a datagram-centered protocol, completely dropping the
- sequential data stream abstraction. Ripping of unnecessary features
- of TCP makes it easier both to implement the protocol and to check
- it for vulnerabilities; numerous TCP vulnerabilities were caused by
- complexity of the protocol's state machine.
- Pursuing the maxim of making the things as simple as possible but
- not simpler we drop all the transmission's metadata except for the
- content's root hash (compare to e.g. .torrent files in BitTorrent).
+ we have with swarmed downloads. The abstraction of a Òdata pipeÓ
+ orderly delivering some stream of bytes from one peer to another
+ turned out to be irrelevant. In even more general terms, TCP
+ supports the abstraction of pairwise _conversations_, while we need
+ a content-centric protocol built around the abstraction of a cloud
+ of participants disseminating the same _data_ in any way and order
+ that is convenient to them.
+
+ Thus, the choice is to make a UDP-based transport, possibly
+ reserving HTTP tonneling as an universal fallback. Further, instead
+ of reimplementing TCP we create a datagram-based protocol,
+ completely dropping the sequential data stream abstraction. Ripping
+ of unnecessary features of TCP makes it easier both to implement
+ the protocol and to check it for vulnerabilities; numerous TCP
+ vulnerabilities were caused by complexity of the protocol's state
+ machine.
+ Pursuing the maxim of making things as simple as possible but not
+ simpler, we fit the protocol into the constraints of the transport
+ layer by dropping all the transmission's technical metadata except
+ for the content's root hash (compare to e.g. .torrent files in
+ BitTorrent). Elimination of technical metadata was achieved through
+ the use of Merkle hash trees, exclusively single-file transfers
+ and other techniques. As a result, a transfer is identified and
+ bootstrapped by its root hash only.
To avoid the usual layering of positive/negative acknowledgement
mechanisms we design a scale-invariant acknowledgement system (Sec
4.4). The system allows for aggregation and variable level of
datagrams, so each datagram SHOULD be processed separately and a
loss of one datagram MUST NOT disrupt the flow. Thus, a datagram
carries zero or more messages, and neither messages nor message
- interdependencies must span over multiple datagrams. In particular,
- Merkle hashes necessary for verifying data integrity are put into
- the same datagram as the data (Sec. 4.3). As a general rule, if
- some additional data is still needed to process a message within a
+ interdependencies should span over multiple datagrams. In
+ particular, any data piece is verified using uncle hash chains; all
+ hashes necessary for verifying data integrity are put into the same
+ datagram as the data (Sec. 4.3). As a general rule, if some
+ additional data is still missing to process a message within a
datagram, the message SHOULD be dropped.
Each datagram starts with four bytes corresponding to the receiving
- channel number (Sec. 4.2). Each message within a datagram has fixed
+ channel number (Sec. 4.2). The rest of a datagram is a
+ concatenation of messages. Each message within a datagram has fixed
length, depending on the type of the message. The first byte of a
message denotes its type. Integers are serialized in the network
- (big-endian) byte order. No variable-length messages, free-form
- text or JSON object allowed. E.g. an acknowledgement message (Sec
- 4.4) having message type of 2 and payload of a four-byte integer 1
+ (big-endian) byte order. Variable-length messages, free-form text
+ or JSON/bencoded objects are not allowed.
+ Consider an example of an acknowledgement message (Sec 4.4). It has
+ message type of 2 and a payload of a four-byte integer (say, 1); it
might be written in hex as: "02 00000001". Later in the document, a
hex-like two char per byte notation is used to represent message
formats.
For the sake of simplicity, one transfer always deals with one file
only. Retrieval of large collections of files is done by retrieving
a directory list file and then recursively retrieving files, which
- might also be directory lists. To distinguish different transfers
- to/from the same peer the protocol introduces an additional layer
- of multiplexing, the channels. "Channels" loosely correspond to
- TCP connections; "content" of a single "channel" is a single file.
- Channel is established with a handshake. To start a handshake, the
- initiating peer needs to know (1) IP address (2) UDP port and
- (3) SHA1 root hash of the content (Sec. 4.3). The handshake is make
- by a HANDSHAKE message, whose only payload is a channel number.
- HANDSHAKE message type is 0. The initiating handshake must be
- followed with the transfer's root hash.
+ might also turn to be directory lists. To distinguish different
+ transfers between the same pair of peers, the protocol introduces
+ an additional layer of multiplexing, the channels. "Channels"
+ loosely correspond to TCP connections; "content" of a single
+ "channel" is a single file. A channel is established with a
+ handshake. To start a handshake, the initiating peer needs to know
+ (1) the requested peer's IP address
+ (2) peer's UDP port and
+ (3) root hash of the content (see Sec. 4.5).
+ The handshake is made by a HANDSHAKE message, whose only payload is
+ a channel number. HANDSHAKE message type is 0. The initiating
+ handshake must be followed by the transfer's root hash.
Initiator sends an initiating datagram to a peer:
- 00000000 00 00000011 04 FFFFFFFF
+ 00000000 00 00000011 04 7FFFFFFF
1234123412341234123412341234123412341234
(to unknown channel, handshake from channel 0x11, initiating a
transfer for a file with a root hash 123...1234)
+
Peer's response datagram:
- 00000011 00 00000022
- (peer to the initiator: use channel number 0x22 for this transfer)
+ 00000011 00 00000022 02 00000003
+ (peer to the initiator: use channel number 0x22 for this transfer;
+ I also have first 4 kilobytes of the file, see Sec. 4.3)
+
At this point, the initiator knows that the peer really responds;
for that purpose channel ids MUST be random enough to prevent easy
guessing. So, the third datagram of a handshake MAY already contain
some heavy payload. To minimize the number of initialization
roundtrips, the first two datagrams MAY also contain some minor
payload, e.g. a couple of ACK messages roughly indicating the
- current progress of a peer.
+ current progress of a peer or a HINT (see Sec. 4.7).
00000022
(this is a simple zero-payload keepalive datagram; at this point
both peers have the proof they really talk to each other; three-way
4.3. Generic acknowledgements
- The generci acknowledgements came out of the need to simplify the
+ Generic acknowledgements came out of the need to simplify the
data addressing/requesting/acknowledging mechanics, which tends
to become overly complex and multilayered with the conventional
approach. Take BitTorrent+TCP tandem for example:
5. Still, one layer lower, TCP also operates with bytes and byte
offsets which are totally different from the torrent's bytes and
offsets as TCP considers cumulative byte offsets for all content
- sent by a connection, both data, metadata and commands.
+ sent by a connection, be it data, metadata or commands.
6. Finally, one more layer lower IP transfers independent datagrams
(typically around a kilobyte), which TCP then reassembles into
continuous streams.
- Obviously, the addressing schemes need mappings; from piece number
- and block to file(s) and offset(s) to TCP sequence numbers to the
- actual packets and the other way around. Lots of complexity is
- introduced by mismatch of bounds: packet bounds are different from
- file, block or hash/piece bounds. The picture is typical for a code
- which was historically layered.
+ Obviously, such addressing schemes need lots of mappings; from
+ piece number and block to file(s) and offset(s) to TCP sequence
+ numbers to the actual packets and the other way around. Lots of
+ complexity is introduced by mismatch of bounds: packet bounds are
+ different from file, block or hash/piece bounds. The picture is
+ typical for a codebase which was historically layered.
To simplify this aspect, we employ a generic content addressing
scheme based on binary intervals (shortcutted "bins"). The base
interval is 1KB "packet", the top interval is the complete file.
- Till Sec. 4.4.1 any file is considered to be 2^k bytes long.
+ Till Sec. 4.4.1 any file is considered to be 2**k bytes long.
The binary tree of intervals is simple, well-understood, correlates
well with machine representation of integers and the structure of
Merkle hashes (Sec. 4.4). A novel addition to the classical scheme
order of interval's right bound, then in the order of length, both
ascending:
- 15
- 7 14
- 3 6 10 13
- 1 2 4 5 8 9 11 12
+ 7
+ 3 11
+ 1 5 9 13
+ 0 2 4 6 8 10 12 14
- Zero number stands for empty interval. The important feature is
- that for any file, numbers of filled intervals (i.e. intervals not
- extending past the end of the file) represent a solid range. For
- example, in a 7KB file intervals 1-11 are filled (1KB is the basic
- unit). In general, this numbering system allows to work with
- simpler datastructures, use arrays instead of binary trees in most
- cases. As a minor convenience, it also allows to use one integer
- instead of two to denote an interval.
+ The number 0xFFFFFFFF (32-bit) or 0xFFFFFFFFFFFFFFFF (64-bit)
+ stands for an empty interval; 0x7FFFFFFF (32-bit) stands for
+ "everything". In general, this numbering system allows to work with
+ simpler data structures, e.g. to use arrays instead of binary trees
+ in many cases. As a minor convenience, it also allows to use one
+ integer instead of two to denote an interval.
Back to the acknowledgement message. An ACK message (type 2) states
that the sending peer obtained the specified bin and successfully
checked its integrity:
- 02 00000007
+ 02 00000003
(got/checked first four kilobytes of a file/stream)
For keeping the state information, an implementation MAY use the
4.4. Data integrity and on-demand Merkle hashes
- The integrity checking scheme is unified both for the usecases of
+ The integrity checking scheme is unified for two usecases of
download and streaming. Also, it works down to the level of a
single datagram by employing Merkle hash trees [MERKLE]. Peers
- receive chains of uncle hashes as required, just in time to check
- the incoming data. As the file metadata is restricted to a single
- hash, hashes also play the role of proving file size to newcomer
- peers. Those functionalities heavily depend on the concept of peak
- hashes, discussed in Sec. 4.4.1. Any specifics related to the cases
- of file download and streaming is discussed in Sec. 4.4.2, 4.4.3
+ receive chains of uncle hashes just in time to check the incoming
+ data. As metadata is restricted to just a single root hash,
+ newcomer peers derive the size of a file from hashes. That
+ functionalities heavily depend on the concept of peak hashes,
+ discussed in Sec. 4.4.1. Any specifics related to the cases of file
+ download and streaming is discussed in Sec. 4.4.2, 4.4.3
respectively.
Here, we discuss the common part of the workflow. As a general
rule, the sender SHOULD prepend data with hashes which are
necessary for verifying that data, no more, no less. While some
- optimistic optimizations are possible, the receiver SHOULD drop
- data if it is impossible to verify it. Before sending a kilobyte
+ optimistic optimizations are definitely possible, the receiver
+ SHOULD drop data if it is impossible to verify it. Before sending a
packet of data to the reciever, the sender inspects the receiver's
previous acknowledgements to derive which hashes the receiver
- already has. I.e. if the receiver had acknowledged bin 8, it must
- already have uncle hashes 9, 13 and 7 for the example case of
- 7KB-long file. That is because those hashes are necessary to check
- the packet against the root hash. Then, hashes 10, 14 and 15 must
- be also known as they are calculated in the process of checking the
- uncle hash chain. Hence, to send the packet 11 (i.e. the last, 7th
- kilobyte of data), the sender needs to prepend no hashes, as hash
- 12 covers an empty range and thus set to zeros, and the hash 13 is
- already known to the receiver. In less lucky cases, the sender MUST
- put into the datagram the chain of uncle hashes necessary for
- verification of the packet, always before the data message itself,
- i.e.:
-
- 04 00000007 F01234567890ABCDEF1234567890ABCDEF123456
- 04 0000000A 01234567890ABCDEF1234567890ABCDEF1234567
- (uncle hashes for the packet 11, the trivial hash for 12 omitted)
+ already has for sure.
+ Suppose, the receiver had acknowledged bin 1 (first two kilobytes
+ of the file), then it must already have uncle hashes 5, 11 and so
+ on. That is because those hashes are necessary to check packets of
+ bin 1 against the root hash. Then, hashes 3, 7 and so on must be
+ also known as they are calculated in the process of checking the
+ uncle hash chain. Hence, to send bin 12 (i.e. the 7th kilobyte of
+ data), the sender needs to prepend hashes for bins 14 and 9, which
+ let the data be checked against hash 11 which is already known to
+ the receiver.
+ The sender MUST put into the datagram the chain of uncle hashes
+ necessary for verification of the packet, always before the data
+ message itself, i.e.:
+
+ 04 00000009 F01234567890ABCDEF1234567890ABCDEF123456
+ 04 0000000E 01234567890ABCDEF1234567890ABCDEF1234567
+ (uncle hashes for the packet 12)
+
The sender MAY optimistically skip hashes which were sent out in
- previous (still unacknowledged) datagrams.
- This way, the receiver incrementally builds the Merkle tree, as it
- is necessary for data validation.
+ previous (still unacknowledged) datagrams. It is an optimization
+ tradeoff between redundant hash transmission and possibility of
+ collateral data loss in the case some necessary hashes were lost in
+ the network so some delivered data cannot be verified and thus
+ has to be dropped.
+ In either way, the receiver builds the Merkle tree on-demand,
+ incrementally, starting from the root hash, and uses it for data
+ validation.
4.4.1. Peak hashes
- Download/streaming unification, also proving of file size both
- depend on the use of peak hashes. Formally, peak hashes are hashes
- defined over filled bins, whose parent hashes are defined over
- incomplete (not filled) bins. Practically, we use peak hashes to
- cover the data range with logarithmical number of hashes, so each
- hash is defined over a "round" 2^k interval.
+ The concept of peak hashes enables two cornerstone features of
+ download/streaming unification, and file size proving. Formally,
+ peak hashes are hashes defined over filled bins, whose parent
+ hashes are defined over incomplete (not filled) bins. Filled bin is
+ a bin which does not extend past the end of the file, or, more
+ precisely, contains no empty packets. Practically, we use peak
+ hashes to cover the data range with logarithmical number of hashes,
+ so each hash is defined over a "round" aligned 2^k interval.
+ As an example, suppose a file is 7162 bytes long. That fits into
+ 7 packets, the tail packet being 1018 bytes long. The binary
+ representation for 7 is 111. Here we might note that in general,
+ every "1" in binary representation of the file's packet length
+ corresponds to a peak hash. Namely, for this particular file we'll
+ have three peaks, bin numbers 3, 9, 12.
+ Thus, once a newcomer joins a swarm, the first peer sending him
+ data prepends it with peak hashes; the newcomer checks them against
+ the root hash (see Sec 4.2.2).
- 04 00000007 1234567890ABCDEF1234567890ABCDEF12345678
- 04 0000000A 234567890ABCDEF1234567890ABCDEF123456789
- 04 0000000B 34567890ABCDEF1234567890ABCDEF1234567890
+ 04 00000003 1234567890ABCDEF1234567890ABCDEF12345678
+ 04 00000009 234567890ABCDEF1234567890ABCDEF123456789
+ 04 0000000C 34567890ABCDEF1234567890ABCDEF1234567890
(this sequence of peak hashes proves that a file is 7KB long)
4.4.2. Hash trees for files
- In the case of static file download, as it was mentioned, a
- transfer is bootstrapped with the root hash. The root hash covers
- the entire 2^30KB data range, i.e. a terabyte. Every hash in the
- tree is defined in the usual way, as a SHA-1 hash of a
- concatenation of two lower-level SHA-1 hashes, corresponding to
- left and right data half-ranges resp. For example,
- hash_2 = SHA1 (hash_1+hash_2)
+ In the case of static file download a transfer is bootstrapped with
+ a root hash. The root hash corresponds to the root bin covering
+ the entire 2**63 KB data range. Every hash in the tree is defined
+ in the usual way, as a SHA-1 hash of a concatenation of two
+ lower-level SHA-1 hashes, corresponding to left and right data
+ half-ranges resp. For example,
+ hash_1 = SHA1 (hash_0+hash_2)
where + stands for concatenation and hash_i stands for Merkle hash
of the bin number i. Obviously, that does not hold for the
base-layer hashes, which are normal SHA-1 hashes over 1KB data
- ranges ("packets"), except probably for the last packet in the
- file, which might have less than 1KB of data. Hashes over empty
- intervals are set to zeros.
+ ranges ("packets"), except probably for the tail packet, which
+ might have less than 1KB of data. The normal recursive formula does
+ not apply to empty bins, i.e. bins that have no data absolutely;
+ their hashes are just zeros.
Lemma. Peak hashes could be checked against the root hash.
Proof. (a) Any peak hash is always the left sibling. Otherwise, be
Thus, once a peer gets peak hashes and checks them against the
root hash, it learns the file size and it also gets practical
anchors for building uncle chains during the transmission (as the
- root hash is too high in the sky). A newcomer peer signals it
- already has peak hashes by acknowledging an empty bin:
- 02 00000000
+ root hash is too high in the sky). A newcomer peer MAY signal it
+ already has peak hashes by acknowledging any bin, even empty one:
+ 02 FFFFFFFF
Otherwise, the first of the senders bootstraps him with all the
peak hashes.
Peer exchange messages are common for many peer-to-peer protocols.
By exchanging peer IP addresses in gossip fashion, the central
coordinating entities (trackers) might be releived of unnecessary
- work. Following the example of BitTorrent, P2TP features two types
+ work. Following the example of BitTorrent, swift features two types
of PEX messages: "peer connected" (type 5) and "peer disconnected"
(type 6). Peers are represented as IPv4 address-port pairs:
05 7F000000 1F40
(connected to 127.0.0.1:8000)
To unify peer exchange and NAT hole punching functionality, the
- sending pattern of PEX messages is restricted. As P2TP handshake is
- able to do simple NAT hole punching [SNHP] transparently, PEX
+ sending pattern of PEX messages is restricted. As swift handshake
+ is able to do simple NAT hole punching [SNHP] transparently, PEX
messages must be emitted in the way to facilitate that. Namely,
once peer A introduces peer B to peer C by sending a PEX message to
C, it SHOULD also send a message to B introducing C. The messages
300-600ms. The peers are supposed to initiate handshakes to each
other thus forming a simple NAT hole punching pattern where the
introducing peer effectively acts as a STUN server. Still, peers
- MAY ignore PEX messages if uninterested in obtaining new peer or
+ MAY ignore PEX messages if uninterested in obtaining new peers or
because of security considerations (rate limiting) or any other
- reason.
+ reason.
4.6. Congestion control
- Custom weaker-than-TCP to eliminate seeding counter-incentives
- lazy seeding
- further simplify by making tit-for-tat rarest-first unnecessary
-
+ swift employs pluggable congestion control. In general, it is
+ expected that servers would use TCP-like congestion control schemes
+ such as classic AIMD or CUBIC [CUBIC]. End-user peers are expected
+ to use weaker-than-TCP (least than best effort) congestion control,
+ such as [LEDBAT] to minimize seeding counter-incentives.
+
+
+4.7. Data requests (HINTs)
+
+ While bulk download protocols normally do explicit requests for
+ certain ranges of data (e.g. BitTorrent's REQUEST message), live
+ streaming protocols quite often do without to save round trips.
+ Explicit requests are often needed for security purposes; consider
+ that BitTorrent can only verify hashes of complete pieces that
+ might consist of multiple blocks requested from many peers.
+ As swift has no such implications, it is supposed to work both
+ ways. Namely, a peer SHOULD send out requested pieces, while it
+ also may send some other data in case it runs out of requests or
+ on some other reason. To emphasize that, request messages are named
+ HINTs; their only purpose is to coordinate peers and to avoid
+ unnecessary data retransmission. A peer is supposed to process
+ HINTs sequentially. HINT message type is 3.
+ 03 00000009
+ (a peer requests fifth and sixth packets)
+
5. Security Considerations
projects / swift-upb.git / commitdiff