RetroShare

What's new in this version:

DIFFERENTIAL FILE LIST SYSTEM:
- The old system for sharing list of files had originally been designed for a totally different purpose, and was very inefficient for this particular job since it required sending the full list of files every time something changed. We have designed a new system where directories are synced independently using a passive/active system: now friend lists are both passively synchronized in the background at a rather slow speed while being interactively updated while browsing. Random IDs are used when publishing directories to neighbor nodes so that they cannot be used to figure out what a friend is not sharing with you. The whole hashing system has also been improved.
- The file sharing manager has been re-designed with less popup windows and more contextual handles. Most data (directories, virtual names, friend groups,…) can be changed in place by double clicking on it. Changes only take place when the “apply and close” button is hit.

Backward compatibility is not fully provided, mainly in order to avoid keeping complicated pieces of code, but an acceptable middle ground has been found. In summary:
- users will need to re-define their shared directories, which is a good opportunity to test the new share manager ???? and does not take long;
- hashes are recovered from the previous version. So if you share the same directories, the hashing step should be very fast. The previous hash cache “file_cache.bin” is still loaded when found, and renamed to “file_cache.bin.bak”, and then converted into the new one “file_sharing/hash_cache.bin”. If needed, you can still rename this file to force Retroshare to import it again.
- All these lists are of course stored encrypted on your disk including friends’ lists, which wasn’t the case before. There is one limitation however: the total number of shared files and directories should stay below 4,194,303 for a maximum number of friends of 1023, because of the storage encoding that has been designed for maximal efficiency (the new system removed all pointers so that Qt can access the hierarchy content both securely and rapidely).

END-TO-END FT ENCRYPTION:
- There has been a lot of discussion about both the possibility and the reason to encrypt anonymous file transfer tunnels, since anonymity guaranties that whatever the data is, it cannot be attributed to anyone in particular. There are however a number of situations where such a feature is wanted, which can be summarized as: files that are shared in private channels, private forums, or even privately sent using distant chat should not be possible to intercept by a node in the network who’s passively listening for the traffic.

Safely encrypting file transfer along anonymous tunnels is of course a chicken-and-egg problem: you would require a pre-shared secret information from which the encryption key is derived, with someone you cannot identify (by design, since the tunnels are made to be strictly anonymous), so it sounds impossible to avoid a man-in-the-middle decryption (if you’re a troll, stop reading here and think for a while). We found a middle ground strategy (Partly similar to what GnuNet does, without using asymmetric authentication keys). Let H(f) represent a hash of the file to share:
- encrypted tunnels are opened through tunnels requests based on H(H(f))
- data packets are encrypted and authenticated using Chacha20+HMAC(sha256) with a random key for each packet. That key is derived by combining a 96-bits random initialisation vector with the actual hash of the file H(f). The construction is exactly the AEAD described in the RFC7539.
- files can optionally be non search-able, which keeps their hash secret
- Consequently, files shared anonymously using private channels, forums, or securely sent using private chat are guarantied to reach destination using end-to-end encryption that is both anonymous and MITM-resistant, the secret pre-shared information being the hash of the file.
- Note that we use our own implementation of Chacha20, which passes all the tests from RFC7539 (Chacha20 is not complicated really). When OpenSSL offers Chacha20 in its mainstream version, we’ll switch to it, without backward compatibility issues.

For backward compatibility reasons, users have now the possibility to change the encryption along file transfer tunnels with two modes (accessible from options->Files):
- accepted: downloaded files will open both encrypted and clear tunnels. Uploaded files accept both modes.
- enforced: downloaded files only open encrypted tunnels.
- Obviously, only “enforced” mode totally protects the data, and unless needed, it should be used preferably.

REVISED REPUTATION SYSTEM:
- The reputation system has been improved so that (1) users that do not give any opinion can rely on their friends to rate post authors; (2) if set, your own opinion is always a priority; (3) anti-spam prevents trolls to create new identities at every post; (4) new comers can easily bootstrap the voting problem.

Reputation now has 5 levels which are figured out by Retroshare combining 2 factors: your own opinion (with absolute priority) and opinions of your friend nodes. The resulting reputation score can be:
- negative: you personally flagged that identity as bad.
- remotely negative: your own opinion is neutral, but your friends provide more negative votes than positive votes
- neutral: no information is available whatsoever for this identity
- remotely positive: your opinion is neural but your friends globally vote positively.
- positive: your own opinion is positive
- The calculation of reputation has been split into positive/negative votes, and users now have the possibility to decide what is the required balance of positive/negative votes to make an identity flagged as remotely bad or remotely positive (See Preferences->People).
- The reputation system still governs the distribution of data, but the set of rules have been improved so that distribution channels with anti-spam flag ON only distribute posts from authors with a non negative reputation. This applies differently to signed identities and identities signed by a known or friend node.
- Forums with anti-spam activated show a special “Distribution” column. The column only displays a yellow (rep. red) flag for those authors who locally have a limited ability to distribute their posts (resp. who are locally banned)
- This means in practice that a newcomer to a forum with anti-spam will only spread his posts to his direct neighbors. The posts will only travel further if the nodes around give a positive opinion to their author. This allows new users of a forum to get known and up-voted, while preventing nasty users to create new identities for each (troll) post, since their data will not go anywhere beyond their direct friends, which in turn can easily decide to fight accordingly.

IMPROVED GUI:

The UI has been cleaned up, with new icons, new design. The settings are now totally interactive, meaning that any change you make immediately takes effect. We tried to improve the first time user experience in many ways. Here’s a non exhaustive list:
- New profile creation has been simplified and all complicated options are hidden (while still here, but at least the average user won’t need to figure out what they mean). The look of it has also been improved. The profile (PGP) passphrase is now asked only once and cached for a few seconds which avoids re-asking the PGP passphrase when signing the SSL certificate;
- all explicit mention of PGP keys have been removed, which are now named “Profile” keys, while keeping the ability to see fingerprints and ascii armored key, for advanced users;
- a Home page has been added as a place to start, make friends and grab your own certificate. This should help newcomers to create their network;
- it is now impossible to supply a badly formed certificate when making friends. Some users in the past tried to supply a PGP key instead of a RS certificate, which would obviously not work. This is now automatically detected when inserting the text in the certificate box.
- Finally, a few new options will probably help a lot: GXS now auto-cleans unused groups, and offers the possibility to choose sync/storage time. It’s important to note that some of these values may require old nodes connected to your node to keep sending data. So unless you stay on the default values for GXS sync, it’s better to have your friend nodes upgraded too.

COMPATIBILITY WITH OPENSSL-1.1.0:
- This step has required some effort! OpenSSL-1.1.0 hides a few structures which where previously open. The changes have been tested successfully on Debian Stretch.
- However: at the time of writing this post, Debian Stretch continues to distribute a version of sqlcipher (3.2.0) that crashes when combined with openssl-1.1.0. We cannot do much about this beyond waiting for the patch to reach mainstream. This should apparently not take long, so be patient.