Re: [tor-dev] Latency measurement from exits
On Thursday 10 March 2016 15:42:48 Zack Weinberg wrote: > Before I go reinventing wheels, has anyone got code to measure network > latency (either RTT or one-way packet travel time will do) from a > chosen Tor exit to an arbitrary destination? Latency from client to > exit would also be useful. Sounds like you can use NavigaTor[0] to achieve that (with small adjustments). At least some parts can be recycled. Best, Robert [0] https://naviga-tor.github.io/ ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Contact Stem example authors.
On Monday 06 April 2015 12:35:27 Lluís wrote: After completing the Stem tutorial I tried to run one of the examples it points to, the RTT Prober. I succeeded. Wow. You are the first one who did (that I know of). However, after patching, recompiling and running the tor version they recommend, I had to change the python sources. In which way? I would share my results, but I really don't know how to contact the authors. Seems I did a bad job in placing my contact details (I think my email address is only listed in the source's header). However, I do read this list and am interested in your results. Best, Robert signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Prototype of a next-generation Tor control interface
A short summary of the discussion on IRC: It is generally considered a good idea to better encode data in Tor's control protocol. In this case Protocol Buffers probably is more suitable for data serialization than JSON. Though Flask seems to be a nice prototyping framework, security concerns were raised regarding the use of HTTP as transport protocol, where ZeroMQ or raw-TCP may be more suitable. Best, Robert signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Status report - Stream-RTT
On Saturday 10 August 2013 02:37:48 Damian Johnson wrote: Yup. It's unfortunate that tor decided to include an 'Exit' flag with such an unintuitive meaning. You're not the first person to be confused by it. Is this meaning at least documented somewhere and I have just read over it? Here's the relevant part of the spec... https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt#l1738 Patch submitted in ticket 9932[0]. Best, Robert [0] https://trac.torproject.org/projects/tor/ticket/9932 signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Status report - Stream-RTT
tl;dr When building a circuit, measuring the RTT a single time could provide better latency and anonymity while not affecting throughput. Multiple measurements could be used for running real-time applications like VoIP or optimizing throughput. Despite the fact that the Tor network is currently in an unusual state so to say, I have been spending the last weeks looking into stream-RTT data of circuits. I gathered the data shortly before and at the beginning of the huge botnet usage. This is what I have found out: As assumed stream-RTT measurements of a single circuit are not at a fixed value but distributed since they are subject to multiple influences. After comparing stream-RTT distributions of multiple circuits, I found lots of different shapes and I realized that no single distribution fits them all. The Time-To-First-Byte (TTFB) for fetching a small website over HTTP is used to approximate the latency of a certain circuit. I used different methods to check the correlation between the RTT of a circuit and its TTFB - all indicating a very high correlation. Hence, stream-RTTs of a circuit make a good estimator for its TTFB and therefor its latency. In terms of latency, using a single stream-RTT measurement (First-RTT) performs better than the currently used method CBT. So far I haven't done any testing/calculations on the other metrics: bandwidth and anonymity. I would assume the former to be unaffected by First-RTT. Latter could probably be slightly increased, if the percentage of discarded circuits would be reduced from 20% with CBT to 10% or 15% with First-RTT - while still achieving a minor improvement in latency. Nevertheless I would not recommend using First-RTT as method for providing low latency circuits to applications, because it only gives a small hint about the quality of a circuit and cannot make sure that some latency properties hold for a certain circuit. Nevertheless First-RTT works pretty well comparing to the minimum effort it takes. Additionally I played around a lot with methods to provide a better estimator for latency properties of a certain circuit. But they all need far more than a single measurement and are therefor out of scope for the common case. Besides they cannot protect against suddenly changing circuit conditions. But they could be used to fulfill a application specific maximum RTT for real-time applications like VoIP. With the use of similar techniques it should be possible to detect circuits that include a node that's within its bandwidth limit. This could be used for providing high bandwidth circuits for applications like BitTorrent. Best, Robert signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Status report - Stream-RTT
On Saturday 10 August 2013 23:52:44 Damian Johnson wrote: If I understand this correctly you're thinking that multiple calls to extend_circuit() cause parallel EXTENDCIRCUIT requests, and the first response would be used for both callers. Is that right? Yes. If so then I would be very interested if you actually see that behaviour. Stem provides thread safe controller communication. See the msg() method of the BaseController - though the Controller's methods are called in parallel the actual socket requests are done in serial to prevent that exact issue that you describe. That looks fine to me. I obviously drew the wrong conclusion from the issues I have encountered. My fault, sorry. Best, Robert signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Status report - Stream-RTT
On Saturday 10 August 2013 02:37:48 Damian Johnson wrote: Hi Robert. Here's the relevant part of the spec... https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt#l1738 Thanks. I will try to make that part more clear and open a ticket. If requests are sent to Tor to create more then a single circuit at once, the mapping between circuit events and create-request is unknown because the circuit ID is not known until the LAUNCHED-event has been received. This is clearly an issue on Tor's side but one could argue that Stem should stop me from using it that way. Not sure that I follow. The extend_circuit() returns the circuit id (it's provided by the EXTENDCIRCUIT call). Are you saying that tor's EXTENDCIRCUIT response is wrong when done in parallel? As far as I understand it it's not necessarily wrong but it might be the case that a response that does not belong to the call is received first: Assume a single program making two extend_circuit() calls within a short time. If the first EXTENDED response is delayed for some reason, both calls receive the EXTENDED response belonging to the second call - both calls use the same circuit ID. Another case, again a single program making two extend_circuit() calls within a short time: if the second call has been made before the first EXTENDED response is received, the second call will use the EXTENDED response from the the first call when it arrives - both calls use the same circuit ID. Therefore the await_build parameter should be True by default IMHO. Anyway it should be made clear that the await_build parameter doesn't work when extend_circuit() is used by two separate programs/threads that run concurrently. The user has to do the locking of (at least) the LAUNCHED event herself then. Besides I could not find any filtering of Tor-internal circuit events. If a Tor- internal circuit EXTENDED event occurs during an extend_circuit() call, the wrong circuit ID will be used. I hope, this is not too confusing. Best, Robert signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Status report - Stream-RTT
On Saturday 27 July 2013 04:41:45 Damian Johnson wrote:
Hi ra, glad to see that you're using stem!
Sure, stem works really great!
If you have any questions,
suggestions, feature requests,
These parts were a bit tricky to figure out for me:
-) When I wanted to check if a certain node is an exit node it took me some
time to figure out that looking for an exit flag is not sufficient because some
nodes are in fact exit nodes but don't have an exit flag. One has to look at
the nodes exit policy which is unaccessible by default because of
microdescriptors. Maybe returning some meaningful message when one uses
get_server_descriptor() and microdescriptors are enabled would help..?
-) It is not safe to use extend_circuit in parallel for creating new circuits.
I think this is not mentioned anywhere.
-) Router status V2/V3 also took me some time but this has already been fixed.
or would like a code review then let me
know.
That would be awesome!
As of just four weeks ago the Controller started providing v3
responses
I missed that obviously. Fixed in [0].
circ.build_flags.count('IS_INTERNAL') == 0
This would more commonly be done as...
'IS_INTERNAL' not in circ.build_flags
Fixed in [0].
try:
controller.reset_conf(__DisablePredictedCircuits)
controller.reset_conf(__LeaveStreamsUnattached)
controller.close()
except NameError:
pass
What raises a NameError?
This was a leftover where it has been possible that controller doesn't exist
at that time. Fixed in [0].
# close circuit, but ignore if it does not exist anymore
try:
self._controller.get_circuit(self._cid)
self._controller.close_circuit(self._cid)
except (ValueError, InvalidArguments):
pass
What is the purpose of the get_circuit() call? If it's not superfluous
It doesn't do any harm but is definitely superfluous. Fixed in [0].
try:
controller = Controller.from_port()
except SocketError:
sys.stderr.write(ERROR: Couldn't connect to Tor.\n)
sys.exit(1)
controller.authenticate()
This is certainly a fine way of doing it, but you might want to also
look at connection.connect_port()...
https://stem.torproject.org/api/connection.html#stem.connection.connect_por
t
It is intended to be a quick and easy method of getting a Controller
for command-line applications. For instance, it will present a
password prompt if tor is configured to use password authentication.
Just realized I should have included it in a tutorial somewhere...
I didn't know that. Since the script now depends on stem version 1.0.1
anyway, I integrated it.
Thank you for your feedback so far!
Your code looks great! If you wouldn't mind I'd love to reference it
on stem's examples page...
Sure, go ahead.
Shall I reference 'https://bitbucket.org/ra_/tor-rtt/' or do you
anticipate your project having a more permanent home? (this might be a
question for Mike as much as you)
I would not mind but I don't have any plans for that. Mike only asked me to
make the code accessible online.
Best,
Robert
[0] https://bitbucket.org/ra_/tor-
rtt/commits/666e0b173871ba3f699c8bc07bfb156f653adf7a
signature.asc
Description: This is a digitally signed message part.
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Status report - Stream-RTT
Hi all! During the last weeks I have been very busy working on my GSoC project which is about reducing the RTT of preemptively built circuits. There is now a single script called rttprober[0] that depends on a patched[1] Tor client running a certain configuration[2]. The goal is to measure RTTs of Tor circuits. It takes a few parameters as input: an authenticated Stem Tor controller for communication with the Tor client, the number of circuits to probe, the number of probes to be taken for each circuit and the number of circuits that should be probed concurrently. It outputs a tar file containing lzo-compressed serialized data with detailed node information, all circuit- and stream-events involved and the circuit build time for further analysis. Since the RTT-measurements are run in parallel with very short locks it is important not to overload Tor nodes. Therefore a single node is not probed more than once at a time. A first analysis of some measurements taken supports the original assumption that a Frechét distribution fits both the circuit build times[3] and round trip times[4]. I will continue gathering and analyzing measurement data and will hopefully be able to draw some conclusions from that. Best, Robert [0] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/rttprober.py?at=master [1] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/patches?at=master [2] https://bitbucket.org/ra_/tor- rtt/src/1127f6936086664981fc55b4dbc82b1570714140/torrc?at=master [3] http://postimg.org/image/je8k5yydt/ [4] http://postimg.org/image/ktk90vxm7/ signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Reduce RTT for preemptively built circuits (GSoC)
Hello everyone! During this year's Google Summer of Code I[0] will be working on reducing the Round-Trip-Time (RTT) for preemptively built circuits.[1] My mentors are Mike and Aaron. A brief summary of the project: RTTs of circuits can be measured by violating the exit policy of the exit node and the resulting error can be timed in a measuring client. It is assumed that the RTTs are Fréchet-distributed which could be used to reject a preemptively built circuit if its RTT is below a certain threshold value. A basic algorithm will be implemented to gather the required data for further statistical analysis which should help answering open questions like: • Are the RTTs Fréchet-distributed? • Does this strategy make new attacks feasible? • How many probes per circuit are needed to do reasonable estimations? • How much additional load is added to the network? • What is an appropriate cut-off percentile? • Does the strategy work in terms of anonymity and performance? • Does the RTT vary for destination ports? (This might be the case for destination ports that occur rarely in exit policies.) • Does this strategy also work if guard nodes are congested? Best, Robert [0] ra_ on OFTC [1] http://www.google-melange.com/gsoc/project/google/gsoc2013/ra_/19001 signature.asc Description: This is a digitally signed message part. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
