A free C++ BitTorrent/HTTP/FTP Download Client

An instance of a BitTorrent client running on a computer on the Internet, which transfers data to and from other clients, is called a peer. Speaking in the strict meaning of the word, a peer does not have the complete file; only parts of it. However, in a more general sense, the word “peer” is often used to refer any BitTorrent client connected in a swarm (see below). In which case, it's synonymous with “client”, disregarding whether or not that client has the complete torrent contents.

Sometimes peers are also called leeches, which in a neutral sense mean “downloaders”. But note that some other times, leech also has a negative connotation, meaning a peer with bad impact on the swarm; namely, someone who has a very poor share ratio (s/he downloads much more than s/he uploads back). This occurs whenever one of the following things happens:

• The leech has an asymmetric Internet connection, thus downloading much faster than s/he uploads but s/he doesn’t keep seeding after finishing the download;
• The leech has a hacked, modified client in order to intentionally avoid uploading;
• The leech drastically limits his/her client’s upload bandwidth.

A torrent swarm is a term used to describe the totality of clients sharing that torrent at any point, or to put it in other words, the totality of peers actively participating in a torrent by downloading or just seeding or LT-Seeding (for BitComet clients only) that torrent.

A seed is a peer that has a complete copy of the torrent’s contents and keeps uploading it (i.e. the torrent is still present and active in the Task List).

How to seed?

Usually one can do that, in any BitTorrent client, by leaving the task running after completing the download. Generally speaking and especially in the initial phases of the swarm, the more seeders there are, the better the chances of getting a higher download speed. However, after a while, depending on how many aggregate copies of the torrent there are among the peers (health percentage), that number might not be that relevant, anymore.

Nevertheless, if every client who completed a download would remove the task as soon as finished, that torrent would pretty soon be dead. Therefore, seeding is a paramount activity on the BitTorrent Network; it's what makes it work and what gives you the possibility to find all those files you seek.

Therefore, in order to share with others what you have downloaded, just leave the task running in your client until it reaches at least a 1.5 - 2.0 share ratio. Also in BitComet, another technology called Long-Term Seeding will make sure that any present task even if stopped (i.e. inactive) will still upload towards other BitComet peers through the means of LT-Seeding protocol, which is proprietary to BitComet, provided there is available unused upload bandwidth, that LT-Seeding is enabled globally on you client as well as on that particular task (which is the case by default) and that the torrent isn't private.
Bottom line, as long as the task is present in the Task List of BitComet in a state or another it's a good chance it's still uploading.

On the other hand, if you mean to share with others files which you have on your local storage media but which are not described by tasks already present in your Task List, then you should look into making and uploading torrents into the BitTorrent Network. For that you can find a pretty detailed guide here: Making torrents with BitComet.

In those particular moments when someone uploads one torrent the first time on the Internet, seed replication can be delayed due to the fact that the initial seeder will send the same torrent piece to several different peers, while other pieces have not yet been uploaded at all. To prevent that, some clients have a “super-seeding” mode, in which they will send out only pieces that haven’t been sent out yet or are very rare and will wait for confirmation that the piece has been uploaded again in the swarm before informing the same peer of a new piece, theoretically making the initial propagation of the first aggregate copy inside the swarm faster and also providing a means for those paying for bandwidth by the byte, to minimize the amount of data uploaded.¹⁾

However, this mode should only be used for starting a new torrent, or for one which must be re-seeded because no other seeds are available. That is because, the normal, “rarest first”, BitTorrent’s model of operation is far more efficient and provides better performance under normal circumstances, compared to the former. Due to the not-very-inspired name, many novice users assume “super-seeding” to be a “faster” seeding mode and thus needlessly enable it, bearing a negative impact on the swarms where they participate. Therefore, to avoid any confusion, in BitComet this mode is called “initial seeding”.<html><br></html> BitComet implemented support for Initial Seeding since version v.1.16.

One of the most often encountered confusions among the newcomers on the BitTorrent realm, is the one created by the term “torrent”. That is because this term is commonly used to refer two, different but related things: the .torrent file and the content it describes. In order to avoid this confusion, many times the word “torrent” gets a period as prefix (.torrent) when it refers to the torrent file itself as opposed to when used to refer the content files the .torrent file describes.

First, let’s talk about the .torrent file.

In order to share one or several files on the BitTorrent Network, someone, who will become the initial seeder (uploader), will have to first create a small file called a “.torrent” file (e.g. examplefile.torrent). This file contains Metadata (such as hash value, file size and name, piece size and hash, tracker IP etc.) about the shared file(s) and about the tracker. By convention the torrent files use the .torrent file extension. The peers which want to download the torrent’s contents must first obtain a .torrent file for it, and connect to the, inside specified, tracker which tells them which are the other peers from whom to download the pieces of the torrent.

A value of special interest is the <html><span id=“infohash” style=color:darkred;font-weight:bold;font-style:italic>info-hash</span></html>, which is a hash value calculated for the info key of the .torrent file and which uniquely identifies that torrent on the Internet. Most index sites display that value for each torrent they index, as does BitComet too for any torrent in its Task List, on its Summary tab. This way (by info-hash) a torrent can be quickly tracked down on multiple index sites or other Internet locations without regard to it's name, thus easily avoiding fake torrents which try to impersonate a valid one by using the same filename.

The other common use of the word “torrent”

This is when “torrent” is referring to the file-set described by a .torrent file, as a whole (a.k.a. the content of the torrent). It is common place to hear expressions like: “The torrent’s size is 4.3GB” or “The torrent contains 10 files” or “I’ve downloaded a torrent”. Especially in this last case confusion occurs, since one usually downloads both .torrent files and “torrent” contents (which are in fact the real BitTorrent downloads, since .torrent files are usually downloaded through HTTP protocol from webpages, prior to initiating BitTorrent transfers). That is why it is a good practice to use the “.torrent” format of the word when referring to the actual torrent file itself, in order to avoid this kind of confusion.

Transfers in the BitTorrent Network, use pieces as trading units. The torrent making application, upon creation of the .torrent file, virtually concatenates together all the files of the torrent and then splits the resulted data chunk in pieces of equal size (usually between 64KB and 4MB). For each piece a SHA-1 hash-value is calculated, which is then put in the .torrent file. When another peer receives a piece later, it calculates the hash value for it, using the same algorithm, and compares that to the value recorded in the .torrent file, in order to check the piece’s data integrity.

Many clients will perform another integrity check upon the complete download of all pieces for a task. This consists of the same procedure as the one described above, except this time the client will calculate the hash value for each downloaded piece for that task and compare it against the expected value, which is already written inside the .torrent file. Any piece which doesn' pass the hash check will be discarded, as it's considered corrupted (that could happen either during transport or even at source).

This same procedure can be initiated manually by the user, usually by right-clicking the task and choosing the option to hash-check from the context menu.
In BitComet this option is called “Manual Hash Check”.

When a peer wants to join the swarm of a certain torrent it will need a bootstrap method. That is, it needs some way to learn about the other peers who share that torrent in order to begin negotiating connections and transfers with them. According to the original design of the BitTorrent protocol, that bootstrap mechanism is provided by the tracker (later extensions also allow the use of other additional methods such as DHT).

A BitTorrent tracker is a HTTP/HTTPS server that indirectly mediates the communication between peers (which are using the BitTorrent protocol) and that keeps track of which seeds and peers are in the swarm. In order to initiate downloads, clients need to communicate with the tracker, and to get a list of peers currently connected to it, who are participating in that torrent’s swarm. The tracker, actually, is not directly involved in any data transfer and does not have a copy of the file(s) for the torrents it tracks. Once the list of peers is obtained, peer communication can continue without a tracker. However, clients report statistics information to the tracker periodically and in exchange receive updated information about new peers to which they can connect or about peers which have left the swarm.

Basically, the tracker is a service which responds to HTTP GET requests. The requests include metrics from clients that help the tracker keep overall statistics about the torrent. The response includes a peer list that helps the client participate in the torrent. The base URL consists of the “announce URL” as defined in the metadata (.torrent) file. The parameters are then added to this URL, using standard CGI methods.

A tracker server should not be mistaken with and should be differentiated from a BitTorrent index server. A BitTorrent index is a server which contains a list of .torrent files, usually including descriptions and other information and which will serve the actual .torrent files to anyone who will choose to download something from the list.
Trackers merely coordinate communication between peers attempting to download the payload of the torrents.

Many BitTorrent websites act as both tracker and index. Sites such as these publicize the tracker's URL and allow users to publish (upload) torrents to the index with the tracker's URL embedded in them, providing all the features necessary to initiate a download.

The DHT (Distributed Hash Table) Network is used to find IP addresses of peers present in a swarm, in addition to or instead of those provided by a tracker. DHT allows to search for peers using queries based on info hash and requires no interaction whatsoever with the tracker(s) of that torrent. By default it is enabled in BitComet and many people use it unknowingly.

There are two flavors of DHT used in the BitTorrent Network; one used by Azureus and an alternative but incompatible version introduced by the Mainline BitTorrent client. There is, however, a plugin available for Azureus Vuze which allows it to access the Mainline DHT Network, which is the network used by BitComet, μTorrent and other clients which have implemented DHT.

“Distributed hash tables (DHTs) are a class of decentralized distributed systems that provide a lookup service similar to a hash table: (key, value) pairs are stored in the DHT, and any participating node can efficiently retrieve the value associated with a given key. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption. This allows DHTs to scale to extremely large numbers of nodes and to handle continual node arrivals, departures, and failures.”²⁾

In the original BitTorrent design the tracker is a fixed bootstrap point localized in a single location on the network. That represents a critical spot of the network or so to speak, a vulnerable “Achilles’ heel” for BitTorrent swarms; if the tracker goes down then the peers have no further means of knowing about each other. As a result the torrent will soon die. Once DHT is in place, however, that problem is mostly out off the window. The difference between the two is that, unlike a tracker, DHT doesn’t rely on a single machine to bootstrap other peers but instead it regards all the peers in the DHT Network as potential bootstrap nodes, thus providing a very fault tolerant mechanism. Usually the bootstrapping mechanism of a peer into the DHT Network, involves any of several redundant methods such as:

• using a list of known or well known DHT nodes (e.g. router.bitcomet.com, router.bittorrent.com etc.);
• using a list of cached nodes saved at every exit of the BitTorrent client;
• using a list of bootstrap nodes included in a .torrent file, into the “nodes” key;
• downloading a regular torrent in a swarm with at least a peer which supports DHT and the “exchange UDP port number” extension. (This makes use of the BitTorrent Extension protocol and the PORT message of the exchange UDP port number extension and not all BitTorrent clients support that extension yet. BitComet added support for it, since v.1.17) ³⁾

After it's got one node, your client can use the DHT Network to find other nodes (until it updates/seeds its routing table) and afterwards it can use them to retrieve peers for its running tasks, using the info-hash values of the torrents for queries. As opposed to this, in the conventional tracker approach, your client needs to communicate with the tracker to learn about each additional peer added. See also: Using DHT tracker.

In BitTorrent file sharing networks, Peer Exchange is used to help maintain a swarm of peers that are collaborating to share a given torrent. In the original BitTorrent design, the peers depend on a tracker to find and maintain the swarm. Peer Exchange allows the members of the swarm to exchange information about other peers in the swarm, directly, without polling the tracker. This is faster and reduces the load on the tracker.

Peer Exchange does not entirely eliminate the need for a tracker since when a peer wants to join the swarm of a given torrent for the first time, it must still make contact with the tracker to find the swarm. Additionally, in some scenarios it is possible for the swarm to split and occasional polling of the tracker(s) will merge it back together.⁴⁾

How is PEX different from DHT?

Well, Peer Exchange takes place after two peers have already connected – they exchange with each other the lists of other peers they have. This is possible only if you have already bootstrapped the swarm somehow, either by connecting to a tracker or to the DHT Network. Also this only works to get you more peers for the current swarm, from the ones already connected to you; it cannot help with cross-swarm situations.

BitComet supports peer exchange using a proprietary protocol in all its older versions.

Starting with v.1.19, BitComet also supports the Mainline/uTorrent/libtorrent implementation of PEX, which is based on the Extension Protocol, therefore it can trade peers with all the clients which support this type of PEX, which pretty much has become the de facto PEX standard, at present time, on the BitTorrent Network.<html><br></html> It still retains support for the older proprietary BitComet PEX protocol, for reasons of compatibility with older versions of the client.

Therefore if you use a version of BitComet older than v.1.19, please note that its PEX implementation is not compatible with other clients' flavors of PEX, therefore, through it you will only be able to obtain other BitComet peers.

According to the original BitTorrent Network model, .torrent files are downloaded from torrent web sites (usually index sites). Upon downloading the file the BitTorrent client calculates a 20-byte SHA-1 hash of the info key from the .torrent file which it uses in the query made to the tracker (or to the DHT Network) to uniquely identify the torrent and find out IP addresses of other peers sharing that torrent, to which it will subsequently connect and download the contents referred in the .torrent file.

Magnet Links take that a step further, since they contain embedded as a parameter, not the link to a .torrent file but instead, the info-hash value already calculated for that specific torrent file. Therefore, by clicking on a Magnet Link your client gets the info-hash of the torrent passed to it, which it further uses to query the DHT Network and find other peers which share that torrent, just as in the case above. The only major difference is that your client will first download the .torrent file from one of the peers. Once it’s obtained the .torrent file, it will be able to begin negotiating standard BitTorrent connections with the rest of peers which it got from the DHT Network (and possibly through PEX).

As you can see, the .torrent file is still needed by the client at some point in the bootstrapping process (since it contains crucial information needed to perform the downloading operations) but you don’t have to rely on a tracker anymore and you don’t even need to download the .torrent file from a website anymore in order to initiate a download; it is sufficient that it be present on the DHT Network.

However, note that Magnet Links cannot eliminate the need to have the .torrent files present and available for download on a website, at some point. Nevertheless, they can reduce the load on torrent index websites and also may offer a better chance of keeping a torrent alive, as once the .torrent file is on the DHT Network it theoretically doesn’t need to be available for download on a website anymore; all you need is just a Magnet Link. And in case the original site hosting the torrent goes down or doesn’t provide it anymore, links are more likely to have been propagated on the Internet than are .torrent files to be hosted for download on alternate sites.

On a larger scope, the advantage of Magnet Links is their versatility given by the open nature and platform independence; the same magnet link can be used for downloading a resource using very different client applications designed for different networks and protocols (provided that the resource is available on those networks) on almost any operating system. The same goes for multi-network clients which can search and retrieve the same resource from multiple networks simultaneously, all of that with a single magnet link (due to the fact that magnet links act like an umbrella for all types of content hashes).⁵⁾

Also, because Magnet Links are plain-text, it is possible to simply copy-and-paste the links into: emails, instant messages, blogs or other social networking media, allowing for a very fast distribution; a property not exactly available for .torrent files, for instance.

“Magnet links consist of a series of one or more parameters, the order of which is not significant, formatted in the same way as the query string at the end of many HTTP URLs. The most common parameter is “xt”, meaning “exact topic”, which is generally a URN formed from the content hash of a particular file” ⁶⁾ (e.g. the info-hash value of a .torrent file).

<html><span style=font-size:85%>Example:</span><br></html> <html><span style=font-size:85%;color:darkgreen>magnet:?xt=urn:btih:b0a8dbd866c5f6d2b619f17e1988f46bdace72ba&dn=Absolute+OpenBSD+-+UNIX+For+The+Practical+Paranoid</span></html>

This is an example of a Magnet Link containing the “xt” parameter with a hex encoded SHA-1 hash value of the referenced resource and the “dn” (display name) parameter containing a user-friendly name of the URN.

The first implementations for Magnet Links required that the BitTorrent hash values contained in Magnet Links be Base32 encoded.⁷⁾ Later that was changed to hex encoding, which is the format recommended at this point for magnet links by the official BitTorrent specifications.⁸⁾ However, links Base32 encoded may still be seen around and BitTorrent clients, normally, should support both formats.

BitComet implemented support for Magnet Links, starting with version v.1.17.

References

Wikipedia - BitTorrent Protocol
Wikipedia - Terminology of BitTorrent
Wiki.Theory - BitTorrent Specifications
Wikipedia - Super-seeding
www.torrentfreak.com - BitTorrent's future, DHT, PEX and magnet links explained
www.filehsarefreak.com - Magnet Links explained

Main Index