Re: [tor-dev] Much-revised draft, RFC: removing current obsolete clients from the network

[grarpamp]

The list of server/client -versions seems long and unuseful...

0.2.4.23
0.2.4.24
0.2.4.25
0.2.4.26
0.2.4.27
0.2.5.8-rc
0.2.5.9-rc
0.2.5.10
0.2.5.11
0.2.5.12
0.2.6.5-rc
0.2.6.6
0.2.6.7
0.2.6.8
0.2.6.9
0.2.6.10
0.2.7.1-alpha
0.2.7.2-alpha
0.2.7.3-rc
0.2.7.4-rc
0.2.7.5
0.2.7.6
0.2.8.1-alpha

I'd cut it to a fixed falloff pattern more like...

0.2.4.27
0.2.5.11
0.2.5.12
0.2.6.8
0.2.6.9
0.2.6.10
0.2.7.3-rc
0.2.7.4-rc
0.2.7.5
0.2.7.6
0.2.8.1-alpha

... to be made shorter by dropping any with security issues,
refill with future releases. Point: stay current.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] A few ideas about improved design/modularity in Tor

[Nick Mathewson]

(Hi!  Here's a document I've been poking at , on and off, for a while.
There's a lot more to say here, but I think it's ready to share with
tor-dev for comment. Thanks!)





++
+++ 0. PRELIMINARIES.
++


== WHY?


If you've ever looked at even slightly serious software engineering
methodology, you've seen some discussion of modularity: the idea that
programs should be divided into a number of smaller modules; that
complexity should be hidden within modules and not in the interfaces
between them; that which modules can call which other modules should be
limited; and other little rules like that.

We'd like modules isolated from one another for these reasons:

* Make them simpler to test
* Minimize what is allowed to touch what
* Make tor easier to understand
* Make tor easier to hack on
* Let us write parts of tor in other languages.
* Make sandboxing more powerful


This is actually several problems all in one!

  - OO/Isolation/Modularity design that doesn't actually split into multiple
processes, but makes the callgraph much much simpler.

  - Multiprocess sandbox framework with RPC implementing the above abstraction

  - Finding the right stuff to carve out into separate sandboxes

  - Finding the right stuff to carve into separate modules.



 Steps/phases:

Over the past 6-8 months we've gone a lot of work to simplify Tor's
callgraph in preparation for this work.  We've got our callgraph
simplified to a pretty extreme degree, for example.  Here are some
of our next steps, some of which can be done in parallel.



1. Figure out which modules should be able to call each others.  Isolate
   them based on their ability to do so.  Refactor until they actually work
   that way.
   - In some cases we will need to split
   - hide big piles of modules behind single entry points.
   - separate src/common and src/or into separate parts.
   - Add tools to enforce this separation
   - Add tools to separate Tor functions safely and not too error-prone.

   (See sections 1 and 3 below.)

2. Add backend abstractions as needed to minimize module coupling. These
   should be abstractions that are friendly to in- and multi-process
   implementations.  We will need at least:

   - publish/subscribe{,/acknowledge}.

 (See
https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern.
 The 'acknowledge' abstraction
 allows the publisher to wait for every subscriber to acknowledge
 receipt.  More on this in section 4 below.)

   - handles

 (We need frequently need handles to objects where the handles need
 to be persistent while the objects themselves can go away. We use
 several patterns for this now. We should just have a handle
 pattern for this instead. This would need separate implementations for
 in-process and cross-process access. See section 2 below for more
 information.)

   - Need a way to mark functions as "internal to module".

 We're going to have a larger number of multi-file modules.  Right now,
 our only restriction for making functions less visible is "static", and
 the fact that we don't allow src/common to call src/or.  Both of these
 should be cleaned up.


3. Write abstractions for splitting code into multiple processes that
   talk over IPC.

 (See section 5 below)


=  Metrics

We can measure our success in a number of ways:

  * How large are the strongly connected components among modules
calling another? Right now the largest SCC is 52 C files.

  * How many layer/module violations can we find?  This needs to
be an automated process.

  * How many exported functions from each module can we remove?

  * How much can we remove from the main linux sandbox code?  How
many functions can we isolate to purpose-built modules?


++
+++ 0. THE MODULES
++


=  Modules in src/or

LOWEST LEVEL

  These should only get called from higher-level components.  They shouldn't
  call anything of a higher level.

  Types and data structures.
- part of configuration.
- parts of statefile
- routerset
- policy (some)
- torcert

  Various backends and utilities.
- confparse
- geoip
- fp_pair?
- reasons
- replaycache
- buffers

  Nodelist data structures, excluding routerinfo.
- nodelist
- microdesc
- networkstatus
- routerparse.c (parts)

SLIGHTLY HIGHER

  These should only get called from higher-level components, and only
  call lower-level components.  When they currently call a higher-level
  piece, they should use a callback or a publish/subscribe API or
  something.

  Nothing in here should really be 

[tor-dev] Using Let's Encrypt for meek bridges

[David Fifield]

I'm looking for ideas of good ways to handle TLS certificates and their
renewal for meek bridges. I want to use Let's Encrypt for this process,
and I hope that someone who knows Let's Encrypt well can contribute some
ideas.

All three of the meek bridges use HTTPS to receive connections from the
CDN, so they need TLS certificates. For example, when you use
meek-azure, your traffic is forwarded to the bridge at
https://meek.bamsoftware.com/. How it works now is I do the usual domain
validation procedure with a CA, receive an email to show that I control
the domain, install the cert and key, and then run the server like this:
meek-server --cert /etc/meek/cert.pem --key /etc/meek/key.pem

When I used Let's Encrypt in the past, using the --webroot option, it
wanted to write a file to the URL path "/.well-known/acme-challenge/{token}".
That won't work for meek-server as it exists today, because meek-server
never serves files from the filesystem. But it could. Perhaps we could
add an option like --acme-webroot that would allow serving files from a
single whitelisted directory.

I notice that there are other ways of proving domain ownership (HTTP,
TLS SNI, DNS). Maybe we could use one of those?
https://ietf-wg-acme.github.io/acme/#identifier-validation-challenges

I also note that there are third-party plugins:
https://github.com/letsencrypt/letsencrypt/wiki/Plugins
Maybe there could be a plugin for meek-server (ideally not requiring new
complicated code in meek-server itself).

Currently you have to restart meek-server in order to make it notice a
changed certificate and key file. It would be better if that were not
necessary--maybe we could periodically stat the files, and re-load them
if they have changed?

This is going to be an issue for Snowflake as well, because we will want
to use WebSocket over TLS for the server component.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Much-revised draft, RFC: removing current obsolete clients from the network

[Spencer]

Hi,



Nick Mathewson:
I should try to clarify!



Awesome!



questions don't seem to apply to proposal 266



They are about the central control of a [somewhat] distributed network, 
specifically, the execution of clients on behalf of the operator.


So, #264 & #266.



I've tried to split the first version of the
proposal into 2.



I understand the proposals as:



prop#264 is for how things _should_ work ;
prop#266 is what we do in the absence of
client-side support in existing Tor versions.

anybody who doesn't know how to die via prop264
will be killable in whatever way we choose for prop266.



And would recommend the titles [though obviously not as relevant as the 
contents]:


'How to ensure client death'

'How to kill clients that wont die'



I'm not aware of anything published.



Bummer ):



reasons:

  1) A non-updated Tor is insecure.
  2) the bulk of [some older] deployed versions appear
 to be defunct botnets
  3) [Depreciated] features



Word.



impact is so large it requires this level of action



Where can this impact be studied?

Given there is no research, there must be a way to visualize the impact.



Windows XP clients still running today, making the
internet less secure.



Business clients pay money to keep MS supporting XP systems, though that 
doesn't weaken the internet as a whole.




every current Tor MAY eventually prove so broken it
needs to go away



Word.

It feels like a decision that the operator should make but I kind of see 
the issue with abandoned clients.


The poison consensus seems fun.

Thanks for taking the time to write, it means a lot (:

Wordlife,
Spencer



___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] [GSoC '16] Exitmap project - Final Proposal

[Mridul Malpotra]

Really sorry for replying to my own mail!

The initial mail threads for reference:
https://lists.torproject.org/pipermail/tor-dev/2016-March/010565.html
https://lists.torproject.org/pipermail/tor-dev/2016-March/010566.html
https://lists.torproject.org/pipermail/tor-dev/2016-March/010574.html


On 25 March 2016 at 22:27, Mridul Malpotra 
wrote:
>
> Hey everyone!
>
> I have completed my proposal on the project Exitmap improvements. Sorry
for submitting it late.
>
> Link for suggestions:
https://docs.google.com/document/d/1AqjH3AsBbfesJg36SZg1frPQsdpAnJmzH8k1-G6dz70/edit?usp=sharing
>
> Some points that have changed from the previous discussion:
>
> 1. Telnet/SNMP sniffing attacks would not be appropriate for Exitmap as
it is an active attack scanner compared to say HoneyConnector etc. hence it
has not been included.
>
> 2. I have added most of the points that I had suggested above with tweaks
as advised by Dr. Winter.
>
>
> I will be submitting the proposal in an hour.
>
> Thank you for your time.
> Sincerely,
>
> --
> Mridul Malpotra
> Undergrad @ IIIT-Delhi
> mridulmalpotra.github.io
> =
> PGP keyID: 0xb716e33ab6d0a653




--
Mridul Malpotra
Undergrad @ IIIT-Delhi
=
PGP keyID: 0xb716e33ab6d0a653
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] GSoC proposal: Panopticlick

[Akito Ono]

Hi Georg,

>Thanks. What does "check fingerprint's uniqueness" entail? Is it just a
>"Unique? - Yes/No"-thing? If not, what are your ideas in this regard?

I think it is not a "Yes/No"-thing.
My idea is that the uniqueness of fingerprint is evaluated in each element
relatively
in recorded data and total uniqueness of a fingerprint is calculated by
those ratios.
I added my ideas to my proposal.

Best regards,
Akito
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?

[Rob Jansen]

> On Feb 11, 2016, at 2:51 PM, Rob Jansen  wrote:
> 
>> These statistics are being collected for years now, and it might take
>> another year or so for relays to upgrade to stop collecting them.  So
>> what's another month.
>> 
> 
> Agreed.

Hi Karsten,

Could you please summarize your current plans regarding stopping or replacing 
the collection method for sensitive statistics, based on discussions in 
Valencia? I'm particularly interested in the plans for the client IP and the 
exit stats that are categorized by exit port.

Thanks,
Rob


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Notes from the prop259 proposal reading group

[Roger Dingledine]

On Fri, Mar 25, 2016 at 01:51:53PM +0200, George Kadianakis wrote:
>   In the future we should make it so that all Guards are both Stable and Fast,
>   so that this stupid check does not need to happen [TODO: I should open a
>   ticket for this if it doesn't already exist].

https://trac.torproject.org/projects/tor/ticket/18624 -- it even comes
with a patch. :)

>   In the future (almost) all guards will be directory guards so this should
>   become less of an issue (see #12538).

On first thought, I would advocate for a similar approach here - if
almost all guards are foo, and it's complicated to handle non-foo guards,
and simple to ask guards to do foo, then let's do the simple thing.

--Roger

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] [GSoC 16] Proposal to work on the Ahmia.fi search engine

[Ismael R]

Hi everyone,

I'm a french student and I would like to share my GSoC application with
all of you.

I plan to work on the Ahmia hidden service search engine. Because of my
experience with python, django and web in general, I feel like I can
make a substantial contribution to this project during summer.

I'm also very interested about the Tor ecosystem, and privacy in
general.

You can read my proposal on Google doc [1] or download a pdf [2].
You are welcome to comment it, I'm open to suggestions.

Best regards,
Ismael R.

[1] : https://goo.gl/AT37hA
[2] : http://dl.free.fr/f6EMq8vOu
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Review of GSoC proposal - Ahmia.fi

[Srikrishna]

Hi people,

Sorry for the late approach. Here is my proposal for the project,
Improving Ahmia.fi, search engine for hidden services. Any suggestions
are most welcome. Thanks to Juha ( juha.nu...@ahmia.fi ) for helping me
out with the proposal. I'll be available on irc as 'skrish13' or you can
comment on the doc itself or reply here :)

Link for my proposal ->
https://docs.google.com/document/d/1GOQMi7qkR7Ppuh4sF5EoSv4Io5D2Dwpz8OyzNoicMHQ

-- 

Thanks and Regards
Srikrishna







signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Notes from the prop259 proposal reading group

[George Kadianakis]

Hello,

we had a meeting about proposal 259 "New Guard Selection Behaviour". You can
see the logs here: 
http://meetbot.debian.net/tor-dev/2016/tor-dev.2016-03-23-15.01.log.txt

Some notes:

- The latest version of the proposal can be found here:
 https://lists.torproject.org/pipermail/tor-dev/2016-March/010625.html
  There is also some discussion with Tim that has been happening in that thread
  since yesterday.

  It's likely that the proposal will be slightly changed (read: improved) as
  implementation and testing proceeds.

  The implementation of the proposal is currently happening here:
https://github.com/twstrike/tor_for_patching/tree/prop259

- We discussed whether the guard algorithm should care about circuit
  restrictions like requiring that the guard of a circuit needs to be Stable
  (needs_uptime) or Fast (needs_capacity).

  We decided that for now the algorithm should be able to handle these
  restrictions, by skipping to the next guard that satisfies the conditions in
  case the top guard does not. This is also how the current guard algorithm 
works.

  In the future we should make it so that all Guards are both Stable and Fast,
  so that this stupid check does not need to happen [TODO: I should open a
  ticket for this if it doesn't already exist].

- We discussed how ReachableAddresses should work with regards to guardlists.

  A suggestion by Tim came from here: 
https://lists.torproject.org/pipermail/tor-dev/2016-March/010630.html
  who says:
   "I suggest that we compose the set of UTOPIC guards based on addresses that
are reachable and preferred (or, if there are no guards with preferred
addresses, those guards that are reachable). I suggest that we use the same
mechanism with DYSTOPIC guards, but add a port restriction to 80 & 443 to 
all
the other restrictions. (This may result in the empty set.)"

  I think this suggstion makes sense for now.

- We also talked about directory guards and how they should work but we didn't
  come to a conclusion.

  My current intuition is that if a directory circuit appears that requires a
  directory guard, we treat it as a constraint the same way we treat
  needs_capacity and needs_uptime. So if our top guard cannot be a directory
  guard (is not a V2Dir), we skip it and go down our list till we find a guard
  that does.

  In the future (almost) all guards will be directory guards so this should
  become less of an issue (see #12538).
  
- Finally, a topic that came up after the meeting in: 
https://lists.torproject.org/pipermail/tor-dev/2016-March/010635.html
  had to do with how we treat bad guards in SAMPLED_UTOPIC_GUARDS and
  SAMPLED_DYSTOPIC_GUARDS.

  So if suddenly 90% of the guards in SAMPLED_UTOPIC_GUARDS drop out of the
  consensus, what do we do? We mark them as bad, and then what? Do we remove
  them from the sampled list? Or do we keep them in there in case they come
  back? (This latter behavior is what tor currently does).

  But then if we keep them, don't we also need to add some more guards in there
  to make up for the bad ones? How exactly should this work? Some thinking
  needs to be done here.

- We also discussed the need for good debug logging in the prop259 tor branch,
  so that we can test it ourselves. Ideally, the logging should be nice and
  intuitive, so that we can monitor the logs every now and then and check if
  the guard picking works properly.

  For example, if the algorithm keeps on switching guards all the time, there
  is a problem. If the algo is using the 5th guard while the 1st guard is up,
  it's a problem. Good logging should be able to reveal all these problems
  without having to spend 20 minutes investigating every new log message.

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Proposal 259: New Guard Selection Behaviour

[George Kadianakis]

Tim Wilson-Brown - teor  writes:

> [ text/plain ]
>
>> On 25 Mar 2016, at 00:31, George Kadianakis  wrote:
>> 
>> Tim Wilson-Brown - teor > 
>> writes:
>> 
>>> [ text/plain ]
>>> 
 On 24 Mar 2016, at 22:55, George Kadianakis > wrote:
 

 

>> I think Reinaldo et al. were also thinking of incorporating the
>> ReachableAddresses logic in there, so that DYSTOPIC_GUARDS changes based on 
>> the
>> reachability settings of the client. I'm not sure exactly how that would 
>> work,
>> especially when the user can change ReachableAddresses at any moment. I think
>> we should go for the simplest thing possible here, and improve our heuristics
>> in the future based on testing.
>
> I suggest that we compose the set of UTOPIC guards based on addresses that 
> are reachable and preferred (or, if there are no guards with preferred 
> addresses, those guards that are reachable). I suggest that we use the same 
> mechanism with DYSTOPIC guards, but add a port restriction to 80 & 443 to all 
> the other restrictions. (This may result in the empty set.)
>

Alright, this seems like a good process here. We should do it like that.

What happens if a utopic guard suddenly is not included in the
ReachableAddresses anymore? Maybe we mark it as 'bad' (the same way we mark
relays that leave the consensus).

>>
>> 
>>
>> I think the current proposal tries to balance this, by enabling this 
>> heuristic
>> only after Alice exhausts her utopic guardlist. Also, keep in mind that the
>> utopic guardlist might contain 80/443 guards as well. So if Alice is lucky, 
>> she
>> got an 80/443 guard in her utopic guard list, and she will still bootstrap
>> before the dystopic heuristic triggers.
>> 
>> There are various ways to make this heuristic more "intelligent", but I would
>> like to maintain simplicity in our design (both simple to understand and to
>> implement). For example, we could imagine that we always put some 80/443 
>> guards
>> as our primary guards, or in the utopic guardlist. Or, that we reduce the 2%
>> requirement so that we go trigger the dystopic heuristic faster.
>
> Or that tor can get a hint about which ports it can access based on which 
> ports it used to bootstrap.
> (See below for details.)
>

Yes, could be.

How would that work though?
And what happens if the network changes? How does the hint work then though?

>> Currently, I'm hoping that we will understand the value of this heuristic
>> better when we implement it, and test it on real networks...
>> 
>> Any suggestions?
>
> There's a whole lot of my thoughts below.
>
> Why such a large list of guards?
>
> Apart from the fingerprinting issue (which I think gets worse with a larger 
> list, at least if it's tried in order), I wonder why we bother trying such a 
> large UTOPIC guardlist.
> Surely after you've tried 10 guards, the chances that the 11th is going to 
> connect is vanishingly small.
> (Unless it's on a different port or netback, I guess.)
> And if our packets are reaching the guard, and being dropped on the way back, 
> we have to consider the load this places on the network.
>

Indeed, I also feel that 80 guards is a lot of guards to try before switching 
to dystopic mode.

I would be up for reducing it. I wonder what's the right number here.

My fear with having a small number of sampled guards in a guardlist is that if
all of them go down at the same time, then that guardlist is useless.

Also, this reminds me that the proposal does not precisely specify what happens
when guards in SAMPLED_UTOPIC_GUARDS become bad (they drop out of the
consensus).  Do we keep them on the list but marked as bad? What happens if
lots of them become bad? When do we add new guards? Currently the proposal only
says:

  It will be filled in by the algorithm if it's empty, or if it contains
  less than SAMPLE_SET_THRESHOLD guards after winnowing out older
  guards. It should be filled by using NEXT_BY_BANDWIDTH with UTOPIC_GUARDS
  as an argument.

I think we should be more specific here.

> Client Bootstrap
>
> The proposal ignores client bootstrap.
>
> There are a limited number of hard-coded authorities and fallback directories 
> available during client bootstrap.
> The client doesn't select guards until it has bootstrapped from one of the 9 
> authorities or 20-200 fallback directories.
>

What do you think should be mentioned here?

> Bootstrap / Launch Time
>
> The proposal calculates bootstrap and launch time incorrectly.
>
> The proposal assumes that Tor attempts to connect to each guard, waits for 
> failure before trying another. But this isn't how Tor actually works - it 
> sometimes tries multiple connections simultaneously. So summing the times for 
> individual connection attempts to each guard doesn't provide an accurate 
> picture of the actual connection time.
>
> When