[tor-dev] [GSoC] CONIKS for Tor Messenger - status report #5

[Huy Vu Quoc]

Hi everyone,

These are things that I have been doing in the last two weeks:

- I and my mentors have had a discussions for next steps for the
project. We decided to reorganize our project’s repo [1]. All repos
has been merged into one [1]. This helps us maintain and keep tracking
the development easier.

- A lot of issues and PRs have been resolved/merged. The merkletree
library is almost done for now.

- Thanks for Ismail and Arlo’s suggestions, the client library for
crypto verifications (including the VRF verification, Ed25519
signature verification, auth path verification and so on) is
implemented in Go and compiled with `c-shared` build mode [2]. This
library then will be wrapped with js-ctypes to use in the add-on. I
have successfully loaded the library into the add-on in Linux
machines. There’s still some problem since Tor Messenger for OS X is
compiled from a Linux machine, while the library is compiled from
macOS, then when loading the library into Tor Messenger for OS X, Tor
Messenger may crash. We will probably dig deeper on this issue.
Windows’ DLL version of the library also should be taken care.

Next steps are mostly to focus on the client-side, along with
finishing the server reference implementation. Here’s thing I expect
in the next weeks:
- Merge the server-registration and registration-bots PR.
- Merge the coniks-preference PR.
- Merge the PR of client library.
- Submit PR for client registration and lookup.

I also expect to have a working version of project before August 15,
including the functions which I described in my proposal.

Best,
Huy

—
[1] https://github.com/coniks-sys/coniks-go
[2] https://github.com/coniks-sys/coniks-go/tree/client-cgo
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] [GSOC 16] Ahmia status update #5

[Ismael R]

Hi everyone,

I'm working on ahmia.fi, the hidden service search engine and you're
reading status update #5.

During the last two weeks I worked on improving search.
It required working on a new index format and updating the crawler to
fill this new index.

Search now has these new features:
- Token search is done in addition to pure full-text search. For
instance, "foxes" and "fox" are different words but are indexed as the
same token. Same for "running" and "run", "tor's and tor".
- Associated words are also stored. For instance, "I’m not happy I’m
working" have the same words than "I’m happy I’m not working", but the
meaning is different.
- Meaningful words have way more impact than frequent words. 
- The text used with a link (called anchor text) is now stored with the
page which is the target of the link. Let's say a site A is linking a
site B like that: B is a great search engine. Searching
"great search engine" is going to return B instead of A.
These anchors could also replace a title in the results list if the
site has no title (.txt files are like that).
- A word present in the title, description, anchors has way more
strength than a word present in the content of a page.
- An authority score is computed for each page thanks to the pagerank
algorithm. Important pages (like homepages) tends to appear first in
the result list. It also solves problems we have with duplicated
content.
- The result list now returns pages instead of sites/domains.

Before updating ahmia.fi, I need to finish porting the rest of the
django site (API and statistics) to the new structure. I hope to finish
this next week so you can try it.

After this, here are the features I plan to implement in the remaining
time:
- Add a language to the search engine (site:msydqstlz2kzerdg.onion to
specify a domain, -term to exclude a term, filetype:txt to specify a
filetype, lang:fr to specify a language, etc).
- Use If-Modified-Since HTTP header to avoid crawling unmodified pages.
- Improve documentation
- Add unit tests

Also, I have some ideas I won't have the time to implement during GSoC.
Hopefully, I will do it later because I plan to continue working with
the Tor community :)
A small list:
- Improve search for languages other than english by making one index
per language
- Make the crawler better understand query parameters. Some are useful
(?page=3), some are useless (?printable=1).
- Better tokenize the url to extract words from it.
- Add highlights to each result (especially for those without meta-
description) to show the context in which query terms appears.
- Make a script to make sure every indexed url is still online.
- Make a true RESTful API.

I see you in two weeks for the next status report and maybe sooner if I
push something online.

Cheers!
Ismael 
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] prop224: Ditching key blinding for shorter onion addresses

[bancfc]

On 2016-07-29 17:26, George Kadianakis wrote:

Hello people,

this is an experimental mail meant to address legitimate usability 
concerns
with the size of onion addresses after proposal 224 gets implemented. 
It's

meant for discussion and it's far from a full blown proposal.

Anyway, after prop224 gets implemented, we will go from 16-character 
onion

addresses to 52-character onion addresses. See here for more details:

https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt#n395

This happens because we want the onion address to be a real public key, 
and not
the truncated hash of a public key as it is now. We want that so that 
we can do
fun cryptography with that public key. Specifically, we want to do key 
blinding

as specified here:

https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt#n1692

As I understand it the key blinding scheme is trying to achieve the
following properties:
a) Every HS has a permanent identity onion address
b) Clients use an ephemeral address to fetch descriptors from HSDir
c) Knowing the ephemeral address never reveals the permanent onion 
address

c) Descriptors are encrypted and can only be read by clients that know
the identity onion key
d) Descriptors are signed and verifiable by clients who know the
identity onion key
e) Descriptors are also verifiable in a weaker manner by HSDirs who
know the ephemeral address

In this email I'm going to sketch a scheme that has all above
properties except from (e).

The suggested scheme is basically the current HSDir protocol, but with 
clients
using ephemeral addresses for fetching HS descriptors. Also, we 
truncate onion

address hashes to something larger than 80bits.

Here is a sketch of the scheme:

--

Hidden service Alice has a long-term public identity key: A
Hidden service Alice has a long-term private identity key: a

The onion address of Alice, as in the current scheme, is a truncated 
H(A).

So let's say: onion_address = H(A) truncated to 128 bits.

The full public key A is contained in Alice's descriptor as it's
currently the case.

When Alice wants to publish a descriptor she computes an ephemeral 
address
based on the current time period 't': ephemeral_address = H(t || 
onion_address)


Legitimate clients who want to fetch the descriptor also do the same, 
since

they know both 't' and 'onion_address'.

Descriptors are encrypted using a key derived from the onion_address. 
Hence,

only clients that know the onion_address can decrypt it.

Descriptors are signed using the long-term private key of the hidden 
service,

and can be verified by clients who manage to decrypt the descriptor.

---

Assuming the above is correct and makes sense (need more brain), it 
should

maintain all the security properties above except from (e).

So basically in this scheme, HSDirs won't be able to verify the 
signatures of

received descriptors.

The obvious question here is, is this a problem?

IIUC, having the HSDirs verify those signatures does not offer any 
additional
security, except from making sure that the descriptor signature was 
actually

created using a legitimate ed25519 key. Other than that, I don't see it
offering much.

So, what does this additional HSDir verification offer? It seems like a 
weak
way to ensure that no garbage is uploaded on the HSDir hash ring. 
However, any
reasonable attacker will put their garbage in a descriptor and sign it 
with a

random ed25519 key, and it will trivially pass the HSDir validation.

So do we actually care about this property enough to introduce huge 
onion

addresses to the system?

Please discuss and poke holes at the above system.

Cheers!
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev



Speaking out of turn here:

Why not integrate kernelcorn's OnioNS project and keep all the current 
security properties?


OnioNS addresses are much more user friendly than even the shorter 
.onion addresses.

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] prop224: Ditching key blinding for shorter onion addresses

[Nick Mathewson]

On Fri, Jul 29, 2016 at 11:26 AM, George Kadianakis
 wrote:
> So basically in this scheme, HSDirs won't be able to verify the signatures of
> received descriptors.
>
> The obvious question here is, is this a problem?

I'm not sure I fully understand, so here's a couple of quick questions
before I look more deeply.  (I'm assuming that descriptors are indexed
by their ephemeral address here.  If that's wrong and they're indexed
by something other than than ephemeral address, my analysis is wrong.)


1) In your scheme, how does descriptor replacement work?  In the
current scheme, if my introduction points change, I can upload a new
descriptor.  In this scheme, it seems that either _anyone_ can upload
a new descriptor with the same ephemeral address (which is insecure),
or descriptors cannot be replaced (which is problematic).

2) Even if descriptors can't be replaced, there's still a problem:
What stops an attacker from racing the hidden service to try to upload
their own competing descriptor with the same ephemeral address?
-- 
Nick
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] prop224: Ditching key blinding for shorter onion addresses

[George Kadianakis]

Hello people,

this is an experimental mail meant to address legitimate usability concerns
with the size of onion addresses after proposal 224 gets implemented. It's
meant for discussion and it's far from a full blown proposal.

Anyway, after prop224 gets implemented, we will go from 16-character onion
addresses to 52-character onion addresses. See here for more details:
  
https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt#n395

This happens because we want the onion address to be a real public key, and not
the truncated hash of a public key as it is now. We want that so that we can do
fun cryptography with that public key. Specifically, we want to do key blinding
as specified here:
  
https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt#n1692

As I understand it the key blinding scheme is trying to achieve the following 
properties:
a) Every HS has a permanent identity onion address
b) Clients use an ephemeral address to fetch descriptors from HSDir
c) Knowing the ephemeral address never reveals the permanent onion address
c) Descriptors are encrypted and can only be read by clients that know the 
identity onion key
d) Descriptors are signed and verifiable by clients who know the identity onion 
key
e) Descriptors are also verifiable in a weaker manner by HSDirs who know the 
ephemeral address

In this email I'm going to sketch a scheme that has all above properties except 
from (e).

The suggested scheme is basically the current HSDir protocol, but with clients
using ephemeral addresses for fetching HS descriptors. Also, we truncate onion
address hashes to something larger than 80bits.

Here is a sketch of the scheme:

--

Hidden service Alice has a long-term public identity key: A
Hidden service Alice has a long-term private identity key: a

The onion address of Alice, as in the current scheme, is a truncated H(A).
So let's say: onion_address = H(A) truncated to 128 bits.

The full public key A is contained in Alice's descriptor as it's currently the 
case.

When Alice wants to publish a descriptor she computes an ephemeral address
based on the current time period 't': ephemeral_address = H(t || onion_address)

Legitimate clients who want to fetch the descriptor also do the same, since
they know both 't' and 'onion_address'.

Descriptors are encrypted using a key derived from the onion_address. Hence,
only clients that know the onion_address can decrypt it.

Descriptors are signed using the long-term private key of the hidden service,
and can be verified by clients who manage to decrypt the descriptor.

---

Assuming the above is correct and makes sense (need more brain), it should
maintain all the security properties above except from (e).

So basically in this scheme, HSDirs won't be able to verify the signatures of
received descriptors.

The obvious question here is, is this a problem?

IIUC, having the HSDirs verify those signatures does not offer any additional
security, except from making sure that the descriptor signature was actually
created using a legitimate ed25519 key. Other than that, I don't see it
offering much.

So, what does this additional HSDir verification offer? It seems like a weak
way to ensure that no garbage is uploaded on the HSDir hash ring. However, any
reasonable attacker will put their garbage in a descriptor and sign it with a
random ed25519 key, and it will trivially pass the HSDir validation.

So do we actually care about this property enough to introduce huge onion
addresses to the system?

Please discuss and poke holes at the above system.

Cheers!
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Implementation plan for prop271 "Another algorithm for guard selection"

[George Kadianakis]

Hello people,

our new guard algorithm proposal (prop271) is getting finalized and we are 
planning to
move to implementation soon. You can find the proposal here:
  
https://gitweb.torproject.org/torspec.git/tree/proposals/271-another-guard-selection.txt

You can also find the STRIKE team's implementation of proposal 259 here:
https://github.com/twstrike/tor_for_patching
Ideally we should borrow as much code as possible from their implementation.

To move forward with engineering, here follows an initial thirteen step
implementation plan for proposal 271.

Where it made sense, I tried to provide links to current functions that can be
used, or to code that the STRIKE team wrote and can be reused or studied.

Let me know what I should make clearer or where I should go further in depth.

---

Design decisions:

An initial design decision here is whether we should make a new file for this
feature or use the old entrynodes.c file? A related question here is whether
both algos should co-exist, or should the new one replace the old? I leave both
decisions to the person who writes the code.

---

Implementation plan:

1) Create main guard selection state data structure (instead of using global 
variables)
** Ideally there should be no global variables, so that in the future we can 
have multiple parallel guardsets (e.g. prop247)
** Thoughtworks: guard_selection_t

2) Sample SAMPLED_GUARDS from GUARDS
** Fill in state for each sampled guard (added_on_date, etc.)
** Save/load SAMPLED_GUARDS to/from state
** Unittests for sampling/saving/loading
** Thoughtworks function: fill_sampled_guards_from_entrynodes()
** Thoughtworks unittest: test_fill_in_sampled_set()

3) Derive FILTERED_GUARDS / USABLE_FILTERED_GUARDS from SAMPLED_GUARDS
** The filtering function can be called at any point (e.g. ON_CONSENSUS)
** Write unittests ensuring correctness of filtering
** See populate_live_entry_guards()
** Thoughtworks function: filter_set() / filter_sampled()

4) Maintain set of PRIMARY_GUARDS
** Functions to extract set of (live) primary guards out of FILTERED_GUARDS
** Unittest
** Thoughtworks function: retry_primary_guards() / next_primary_guard()
** Thoughtworks unittests: test_next_primary_guard()

5) Selecting guards for circuits
** Meant to replace choose_random_entry_impl()
** See section SELECTING in prop271
** Add new circuit states to or_circuit_t
** Implement the guard selection logic
** Unittests on circuit state machine
** Unittests on guard selection logic

6) Update guard state when a circuit fails/succeeds
** Section ON_FAIL and ON_SUCCESS
** See entry_guard_register_connect_status()
** Unittest

7) Maintain CONFIRMED_GUARDS
** Add guard to CONFIRMED_GUARDS when circuit succeeds
** Fill in state info for each confirmed guard (confirmed_on_date, etc.)
** Migration from old state format to new
** Thoughtworks unittest: test_used_guards_parse_state()

8) Retry schedule for guards (section RETRYING)
** See entry_is_time_to_retry()
** Unittest

9) Update list of waiting circuits (section UPDATE_WAITING)
** Implementation of guard "higher priority" ordering (see CONFIRMED)
** Unittest

10) Update state whenever we get a new consensus (section ON_CONSENSUS)
** Update SAMPLED_GUARDS elements (section SAMPLED)
** Remove obsolete/expired guards
** See entry_guards_compute_status() / remove_obsolete_entry_guards()
** Thoughtworks function: entry_guards_update_profiles()

11) Bridge support
** Thoughtworks function: fill_in_from_bidges()

12) Separate state instances when EntryNodes/ExcludeNodes/etc are used
** See prop271 MEANINGFUL_RESTRICTION_FRAC etc.

13) Glue things together!
** Add clean and meaningful logging so that we can *heavily* field test the 
feature in our machines
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] [GSOC16] Fingerprint Central - Status report n°5

[Pierre Laperdrix]

Hi everyone,

Here is my fifth status report for my GSOC project.
Repository: https://github.com/plaperdr/fp-central
Beta: https://fpcentral.irisa.fr

1 - Support of TBB versions for statistics through tags
I have added the selection of TBB versions for the custom statistics
page. In order to do that, I designed and developed the tag system that
I planned to develop much later for the project. For those who do not
know what tags are, it is a way to classify fingerprints and put them
into different categories. By analyzing collected fingerprints, the
system can assign one or more tags to them.
After several iterations, I think I found a good balance between
expressiveness and simplicity. You can find the documentation on how to
add your own tag to the system here:
https://github.com/plaperdr/fp-central/blob/master/fingerprint/tags/README.md
I have added a command to refresh and recompute all tags in case a new
one is added to the system or if the logic of one tag has been changed.
Right now, the tag system is used to tag each fingerprint with the
corresponding TBB version but it can be used later on for many different
things like identifying the OS or the type of the browser.

2 - Better and more complete custom statistics page.
The custom statistics page is now much more complete than before.
In addition to the already-existing pie chart, I have added an HTML
table which gives the complete list of data from the chosen attributes.
You can sort the data and download it in several different formats. It
is a handy feature if you want to put the fingerprints into a different
software or if you want to copy/paste them for a thorough bug report.
I've pushed the latest updates today to the beta server so that you can
try them right now and send me feedback on them.

My goal for the next two weeks is to iron things out in the beta and fix
bugs. I still need to make sure that only TBB fingerprints are stored in
the database. Feature-wise, it seems that I'm getting near the end on
the list of features I wrote in my proposal. The only real big one that
is left is to have a page to see if your complete TBB fingerprint is
"acceptable" (right now, only several attributes are checked). Finally,
for the final stretch in the second half of august, I'm planning on
completing the documentation and the french localization while making
sure that deployment and updates of the site are as easy as possible.

Have a great week-end,
Pierre



signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev