Re: [tor-dev] Stem ORPort protocol support

[teor]

> On 8 Feb 2018, at 07:23, Damian Johnson  wrote:
> 
> Hi all. Over the last few months Tim and I have been collaborating on
> Python support for the ORPort protocol. With it you can download
> descriptors without a DirPort, and possibly fancier things in the
> future like full circuit construction.

Stem's ORPort support is great news!

We have been looking at alternate Tor implementations that are
easier to hack for testing:

https://trac.torproject.org/projects/tor/ticket/17270

> Tim put together a wonderful proof of concept called Endosome...
> 
>  https://github.com/teor2345/endosome
> 
> ... and I just finished integrating it into Stem...
> 
>  https://gitweb.torproject.org/stem.git/tree/stem/client/__init__.py

I have added both Endosome and Stem to the Tor implementations wiki:

https://trac.torproject.org/projects/tor/wiki/doc/ListOfTorImplementations

Looking forward to trying out the ORPort features when the descriptor
download API is stable. It will be great to be able to check Fallback
Directory Mirror ORPorts:

https://trac.torproject.org/projects/tor/ticket/19129

T

--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n







signature.asc
Description: Message signed with OpenPGP
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Stem ORPort protocol support

[meejah]

Damian Johnson  writes:

> Thanks meejah! Took a peek but they both look pretty old and it's
> unclear to me how complete either got. If there's something in
> particular you think is worthwhile integrating I'm all ears.

I haven't looked closely enough to know the answer to that ;) but as I
understand it had (2 years ago) enough to bootstrap and make circuits.

>> For any of these efforts, writing a "Tor protocol library" that
>> *doesn't* do any I/O would be the most useful; then other Python tools
>> can benefit from the protocol support without being tied to "threads" or
>> to a particular async framework.

> Actually, this uses a similar pattern as the rest of Stem in that cell
> packing/unpacking is independent of the threaded socket. Just as you
> could use stem.response for controller message parsing in txtorcon,
> you can use stem.client.cell for cell packing/unpacking with a twisted
> application too.

Interesting, okay! Last time I looked, I had to re-assemble the whole
"response" to feed it into stem.response -- so I ended up needing to
keep most of the protocol state-machine anyway (i.e. to figure out which
bytes constituted "a response"). I should probably look again if this
has changed :)

The way Hyper/h2 works is it just gets fed bytes and "interesting
events" come out (essentially), as I understand it. So, the
corresponding thing here would be: set up some Stem 'protocol' object
with an "interesting event happend" callback. Then, feed bytes into some
single API on this object and it calls the "event happened" callback
every time enough bytes have been fed in for a complete parsed response
object to be generated. (This is just the one side of the protocol; the
other side would be similar)

-- 
meejah
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Stem ORPort protocol support

[Damian Johnson]

> So, there is this already -- not sure how "complete" it is though (and
> looks like hasn't seen commits for 2+ years) but might have useful code:

Thanks meejah! Took a peek but they both look pretty old and it's
unclear to me how complete either got. If there's something in
particular you think is worthwhile integrating I'm all ears.

> For any of these efforts, writing a "Tor protocol library" that
> *doesn't* do any I/O would be the most useful; then other Python tools
> can benefit from the protocol support without being tied to "threads" or
> to a particular async framework.

Actually, this uses a similar pattern as the rest of Stem in that cell
packing/unpacking is independent of the threaded socket. Just as you
could use stem.response for controller message parsing in txtorcon,
you can use stem.client.cell for cell packing/unpacking with a twisted
application too. That said, thanks for the reminder to keep this in
mind as we go.

Cheers! -Damian
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] Stem ORPort protocol support

[meejah]

Damian Johnson  writes:

> they'd care to go (potentially all the way up to a Python Tor client,
> similar to Orchid).

So, there is this already -- not sure how "complete" it is though (and
looks like hasn't seen commits for 2+ years) but might have useful code:

   https://github.com/pycepa/pycepa

There is also this one, that I've barely looked at (and is older than
the above) based around Scapy:

   https://github.com/cea-sec/TorPylle

For any of these efforts, writing a "Tor protocol library" that
*doesn't* do any I/O would be the most useful; then other Python tools
can benefit from the protocol support without being tied to "threads" or
to a particular async framework.

One of the best examples of this style of library is the hyper/h2
implementation of HTTP2 (which powers Twisted's HTTP2 support and also
supports threaded HTTP2 servers and clients) by separating the
"protocol" implementation into its own library (that does no I/O and
doesn't start any threads "for" you). This style is usually referred to
as "sans-io" (at least in the Python community). You can read more about
these libraries: https://python-hyper.org

It would be really cool to have a Python implementation of the Tor
protocol -- and double-extra-useful if it's a "pure" protocol library
without any messy I/O constructs involved :)


Cheers,
meejah
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] Stem ORPort protocol support

[Damian Johnson]

Hi all. Over the last few months Tim and I have been collaborating on
Python support for the ORPort protocol. With it you can download
descriptors without a DirPort, and possibly fancier things in the
future like full circuit construction.

Tim put together a wonderful proof of concept called Endosome...

  https://github.com/teor2345/endosome

... and I just finished integrating it into Stem...

  https://gitweb.torproject.org/stem.git/tree/stem/client/__init__.py

With stem.client you can now download descriptors...

  import stem.client

  with stem.client.Relay.connect('127.0.0.1', 12345, [3]) as relay:
circ = relay.create_circuit()
circ.send('RELAY_BEGIN_DIR', stream_id = 1)
desc = circ.send('RELAY_DATA', 'GET /tor/server/authority
HTTP/1.0\r\n\r\n', stream_id = 1).data
circ.close()

print(desc)

When run this looks like...

  % python demo.py
  HTTP/1.0 200 OK
  Date: Wed, 07 Feb 2018 18:42:41 GMT
  Content-Type: text/plain
  Content-Encoding: identity
  Expires: Fri, 09 Feb 2018 18:42:41 GMT

  router Unnamed 97.113.177.53 12345 0 23456
  identity-ed25519
  -BEGIN ED25519 CERT-
  AQQABm/qAazUltT1iUUbIMw8VNNhGb50FDHKJz6S94FLQNxL0LObAQAgBAAapbO9
  iLFD0l9SEiEMFQWIT2VnbLyCZKvbrxTs5ULC1l1hQPoui6Y/lEd3yjrQhIs/vl6R
  1S6FbwSFDmiXOzq47mFrse4C71ht3TpLOD0F3wiyjWtsqU1k7iPmmpejUgs=
  -END ED25519 CERT-
  master-key-ed25519 GqWzvYixQ9JfUhIhDBUFiE

I'd like to emphasize this is still very alpha. The API isn't set in
stone and there's no doubt quite a few rough edges. However, I wanted
the list to be aware just in case anyone would care to build on it. I
plan to draw a line at 'download descriptors through ORPorts' but I'd
be delighted to help others if there's more ambitious directions
they'd care to go (potentially all the way up to a Python Tor client,
similar to Orchid).

Now that we've reached this milestone I'm taking a break to focus on
Stem support for v3 Onion Services for a bit. However, when I come
back the next things on my dance card are...

  a. Support ORPort downloads in the stem.descriptor.remote module.
  b. More integ tests so Stem can be used as a tool for testing tor's ORPort.
  c. Give more thought to the API we'd like to vend.
  d. Brainstorm a GSoC project idea that expands these capabilities.

Cheers! -Damian
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] [release] Onionoo 5.0-1.10.1

[iwakeh]

Hi there!

A fix for the most recent Oninonoo version is available at:
 https://dist.torproject.org/onionoo/5.0-1.10.1/

The changes [1] concern also the resolution of graph data:
three months weights and bandwidth data are now provided in 24 hours detail.

Deployment of the new version 5.0-1.10.1 finished successfully for all
onionoo.torproject.org instances.  The changes for bandwidth data might
take another day to be visible.  So, we need to have a little patience.

Please direct comments and questions to the metrics-team mailing list [2].

Cheers,
iwakeh

[1] 
https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md?id=a7418b7c4d1b84ae1c1c062df847b013cb002e82
[2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team














signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[tor-dev] [release] Onionoo 5.0-1.10.0

[iwakeh]

Hi there!

the new Oninonoo version is available at:
 https://dist.torproject.org/onionoo/5.0-1.10.0/

All changes [1] this time are supposed to make writing of bandwidth,
clients, uptime, and weights documents independent of system time,
which is a step toward making Onionoo instances serve the same data
where possible.  Other steps remain and will be addressed in future [2].

Deployment of version 5.0-1.10.0 will start at 13:00 UTC for all
onionoo.torproject.org instances.  It might cause a a few hours without
updates as we need to make additional backups.  For those interested,
I will post a deployment finished message to metrics-team mailing list [3]
approximately during UTC evening.

Please direct comments and questions to the metrics-team mailing list [3].

Cheers,
iwakeh

[1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md
[2] https://trac.torproject.org/projects/tor/ticket/25002
[3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team












signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev