Re: [tor-dev] IRC meeting to discuss sponsor F progress on Wed July 3, 16:00 to 17:00 UTC in #tor-dev

2013-06-27 Thread Karsten Loesing
On 6/20/13 7:44 PM, Karsten Loesing wrote:
 Hi all,
 
 I'd like to schedule an IRC meeting to discuss what progress we made on
 sponsor F deliverables in June.  Suggested time and place are:
 
   Wed July 3, 16:00 to 17:00 UTC in #tor-dev
 
 That time in other timezones is:
 
   9:00 in San Francisco
   12:00 in Boston
   18:00 in Berlin
   19:00 in Athens
   21:30 in New Delhi

Confirming the suggested date and time above.  Nobody told me that the
alternative date works any better than the suggested date.  So, it's:

  Wed July 3, 16:00 to 17:00 UTC in #tor-dev

 People who should attend are: Roger, Nick, Andrea, George, David,
 Sathya, Moritz, Linus, Andrew, Tom, and anyone else who wants to attend.

The above is still true, though I hear Andrea and Linus might not be
able to make it.

Here's what we're going to talk about:

https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorF/Year3

See in particular items 10 to 23 in phase 2, due October 31, 2013.
We're going to discuss progress on these deliverables in June and
outline next steps for July.

Thanks,
Karsten

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Client simulation

2013-06-27 Thread Karsten Loesing
On 6/26/13 5:59 PM, Norman Danner wrote:
 Continuing this discussion of client behavior simulation...
 
 I'm in the process of rewriting the data collection code..
 
 One thing I need to do is make a reasonable guess as to whether a given
 connection is from a client.  Is there a straightforward way to do that
 programmatically?  As a first pass, I'd even take isn't a known
 relay/authority/etc.
 
 I've been poking through the source code, and I assume I'll find
 something appropriate eventually.  But I wouldn't mind a shortcut...

This code looks related:

/* only report it to the geoip module if it's not a known router */
if (!router_get_by_id_digest(chan-identity_digest)) {
  if (channel_get_addr_if_possible(chan, remote_addr)) {
geoip_note_client_seen(GEOIP_CLIENT_CONNECT, remote_addr,
   now);

https://gitweb.torproject.org/tor.git/blob/HEAD:/src/or/channel.c#l2379

Best,
Karsten

___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


[tor-dev] Damian's Status Report - June 2013

2013-06-27 Thread Damian Johnson
Hi all. Without much ado here's my status report for June...

* Stem: Migrated to the Mock module

Our homemade mocking framework has served us well, but over time it
taught me one very important lesson: writing a mocking framework is
hard. On the surface it seems pretty simple: apply and revert a set of
monkey patches. But how do you monkey patch class methods? What about
alias imports like the os module? And god forbid you want to mock
python's open() function.

I'm finally taking a lesson from one of my coworkers and using a
library for this. Python has several options but the most common is
PyPI's mock module, which became part of the standard library in
Python 3.3...

  http://www.voidspace.org.uk/python/mock/
  https://gitweb.torproject.org/stem.git/commitdiff/101cf0b?hp=65846e8



* Arm: Pruning the torTools utility

Arm's torTools module was a wrapper around TorCtl that provided
caching, thread safety, and a better API. Now that we're using stem
it's obsolete, and my aim is to completely eliminate it from the
codebase. This is easier said than done however. This month I pruned
vast swaths of the module, reducing it from 2768 lines to 1020...

  before: 
https://gitweb.torproject.org/arm.git/blob/refs/heads/release:/src/util/torTools.py
  after: https://gitweb.torproject.org/arm.git/blob/HEAD:/src/util/torTools.py

Doing this required some expansions on stem's part. Added
functionality includes...

  * get_pid() and get_user() methods in the Controller
  * system functions for getting a process' start time and FreeBSD jail path
  * the system module's call() wasn't respecting exit codes



* Stem: Remote descriptor fetching

Throughout the month Karsten and I have been bouncing ideas back and
fourth concerning a stem API for remote descriptor fetching...

https://trac.torproject.org/projects/tor/wiki/doc/stem#RemoteDescriptorFetching
https://lists.torproject.org/pipermail/tor-dev/2013-June/004957.html

I have some s3krit ideas for improving this that will address both our
use cases even better (spoiler: future style instances). We're coming
up on a four day weekend so hopefully I'll be able to start
implementing this soon!



* Stem: Expanded FAQ with answers to common Stack Overflow questions

I took a pass over the tor related questions on Stack Overflow
(http://stackoverflow.com/questions/tagged/tor), answering fifteen
that concerned controller scripting. The vast majority of those
unfortunately were some variation of 'how do I programmatically change
my IP?' which I answered with a Stem FAQ entry...

  https://stem.torproject.org/faq.html#how-do-i-request-a-new-identity-from-tor

The only question I thought was especially interesting went along the
lines of 'How do I check the IPs of the exits I'm presently using?'. I
answered this with a FAQ entry too...

  
http://stackoverflow.com/questions/9777192/how-do-i-get-the-tor-exit-node-ip-address-over-the-control-port
  https://stem.torproject.org/faq.html#how-do-i-get-information-about-my-exits



... and a handful of smaller tasks...

  * Investigated the descriptor version provided via tor's 'GETINFO
ns/*' options. Contrary to to the spec it turns out these have been v3
all along, and stem now parses them as such. (#7953)
  * Our automated Jenkins test runs caught their first instance of tor
regression. This concerned LOADCONF's behavior after merging a branch
for ticket #6752 (#9122).
  * Handful of GSoC tasks (welcome/sorry emails after acceptance was
announced, and org admin prep for the program's start)
  * Added msg_type argument to ControlMessage.from_str() (request by
meejah, #8605)
  * Investigated cache consistency issue (thanks to Ravi, #7713)
  * Fixes for ONLINE target (patch by Jeremy, #8692)
  * Minor revisions to how consensus bandwidth-weights are validated (#6872)
  * Addressed an arm issue with certain TERMs (#9064)
  * Answered some questions from Sreenatha that came up while
migrating Tor Weather to Stem
(https://lists.torproject.org/pipermail/tor-dev/2013-June/005035.html)

Cheers! -Damian
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Gitian builds in VirtualBox with Vagrant

2013-06-27 Thread David Fifield
On Sat, Jun 22, 2013 at 05:17:44PM -0700, Arlo Breault wrote:
 If anyone has VirtualBox and Vagrant installed,
 
 https://www.virtualbox.org/
 http://www.vagrantup.com/
 
 and wants to try the new TBB deterministic builds,
 this worked for me,
 
  git clone -b vagrant https://github.com/arlolra/tor-browser-bundle.git
  cd tor-browser-bundle/vagrant
  make
 
 Mike, there's a few patches in there you may want,
 https://github.com/arlolra/tor-browser-bundle/compare/vagrant

This is rather exciting. Do you think that this method can be adapted to
build the pluggable transports bundles? This is currently done
semi-manually, with a makefile that builds the pluggable transports,
unzips the bundle, copies in some files, and re-zips the bundle.

https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Makefile

We also have instructions for setting up a build VM from scratch--which
it sounds like is what Vagrant does--but I guess we would instead use
whatever is the build machine for the vanilla bundle.

https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Makefile

It would solve a lot of problems for us to have the PT bundles built at
the same place and time as the vanilla bundles.

David Fifield
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Torsocks development status

2013-06-27 Thread David Goulet
Ian Goldberg:
 On Wed, Jun 26, 2013 at 03:55:58PM -0400, David Goulet wrote:
 Hi everyone,

 For those who don't know, I've been working on a new version of Torsocks
 in the last three weeks or so.

 https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html

 I just wanted to give a quick status report on the state of the development.

 The DNS resolution is working for domain name (PTR) and IPv4 address.
 Currently, Tor does not support IPv6 resolution but the torsocks code
 support it.

 Hidden service onion address resolution is also working using a dead IP
 range acting as cookie that is sent back to the user and mapped to the
 .onion address on the hijacked connect().

 I've changed quite a bit the configuration file (torsocks.conf) to fit
 the style of tor (torrc). At this point, the tor address and port can be
 configured as well as the dead IP range mention above. More is coming
 but pretty simple for now.

 Logging is working, connection registry and thread safety as well. There
 is also a compat layer for mutexes and once I start porting the project
 to other *nix system (BSD, OS X, ...) probably more subsystem will be
 added to that compat layer.

 So, in a nutshell, some libc calls still need to be implemented, *moar*
 tests and other OS supports. I'm confident to have a beta version to
 present to the community in a couple of weeks (if nothing goes wrong).

 Feel free to browse the code, comment on it, contribute!, etc...

 https://github.com/dgoulet/torsocks/tree/rewrite
 
 Are non-blocking sockets, select/poll/etc. (especially at connect()
 time), and optimistic data on the to-do list?

Yes! Good point I should have put the todo list. So yes, non block socket 
support.

For optimistic data, it is kind of tricky. I can use it for DNS resolution
without a problem because torsocks control the complete flow of data from
opening a SOCKS5 connection to closing it after the DNS response is received
however for actual real data (sendmsg, send, ...) a connect is needed before so
it would means that a connect() call will return yes OK socket connected but
where in fact it is not really true. So, when the first data are sent, there is
a possibility that the Tor connections failed or even we block for an unknown
amount of time during the send*/write() call.

Now the question is, is this the kind of behavior that would be acceptable
meaning basically lying to the caller at connect() and possibly blocking I/O
calls and returning something like ECONNRESET or ENOTCONN if the Tor socks5
connection fails.

This is *real* tricky especially with non blocking socket, if torsocks needs to
do some possible blocking call for the SOCKS5 replies during an I/O call from
the caller that is not suppose to block. Furthermore, having pending data that
*might* come at any time on the connection from the SOCKS5 negotiation, the
caller could put the file descriptor in poll() mode, wake up and try to receive
the data but where in fact it's the socks5 reply... it's possible to handle that
but it seems here a VERY intrusive behavior. Does optimistic data worth it here
vis-a-vis the complexity of handling that it and high intrusiveness ?

Cheers!
David

 
 Thanks,
 
- Ian





signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Torsocks development status

2013-06-27 Thread Ian Goldberg
On Thu, Jun 27, 2013 at 03:11:23PM -0400, David Goulet wrote:
 Ian Goldberg:
  On Wed, Jun 26, 2013 at 03:55:58PM -0400, David Goulet wrote:
  Hi everyone,
 
  For those who don't know, I've been working on a new version of Torsocks
  in the last three weeks or so.
 
  https://lists.torproject.org/pipermail/tor-dev/2013-June/004959.html
 
  I just wanted to give a quick status report on the state of the 
  development.
 
  The DNS resolution is working for domain name (PTR) and IPv4 address.
  Currently, Tor does not support IPv6 resolution but the torsocks code
  support it.
 
  Hidden service onion address resolution is also working using a dead IP
  range acting as cookie that is sent back to the user and mapped to the
  .onion address on the hijacked connect().
 
  I've changed quite a bit the configuration file (torsocks.conf) to fit
  the style of tor (torrc). At this point, the tor address and port can be
  configured as well as the dead IP range mention above. More is coming
  but pretty simple for now.
 
  Logging is working, connection registry and thread safety as well. There
  is also a compat layer for mutexes and once I start porting the project
  to other *nix system (BSD, OS X, ...) probably more subsystem will be
  added to that compat layer.
 
  So, in a nutshell, some libc calls still need to be implemented, *moar*
  tests and other OS supports. I'm confident to have a beta version to
  present to the community in a couple of weeks (if nothing goes wrong).
 
  Feel free to browse the code, comment on it, contribute!, etc...
 
  https://github.com/dgoulet/torsocks/tree/rewrite
  
  Are non-blocking sockets, select/poll/etc. (especially at connect()
  time), and optimistic data on the to-do list?
 
 Yes! Good point I should have put the todo list. So yes, non block socket 
 support.
 
 For optimistic data, it is kind of tricky. I can use it for DNS resolution
 without a problem because torsocks control the complete flow of data from
 opening a SOCKS5 connection to closing it after the DNS response is received
 however for actual real data (sendmsg, send, ...) a connect is needed before 
 so
 it would means that a connect() call will return yes OK socket connected but
 where in fact it is not really true. So, when the first data are sent, there 
 is
 a possibility that the Tor connections failed or even we block for an unknown
 amount of time during the send*/write() call.
 
 Now the question is, is this the kind of behavior that would be acceptable
 meaning basically lying to the caller at connect() and possibly blocking I/O
 calls and returning something like ECONNRESET or ENOTCONN if the Tor socks5
 connection fails.
 
 This is *real* tricky especially with non blocking socket, if torsocks needs 
 to
 do some possible blocking call for the SOCKS5 replies during an I/O call from
 the caller that is not suppose to block. Furthermore, having pending data that
 *might* come at any time on the connection from the SOCKS5 negotiation, the
 caller could put the file descriptor in poll() mode, wake up and try to 
 receive
 the data but where in fact it's the socks5 reply... it's possible to handle 
 that
 but it seems here a VERY intrusive behavior. Does optimistic data worth it 
 here
 vis-a-vis the complexity of handling that it and high intrusiveness ?
 
 Cheers!
 David

It *is* kind of tricky.  (See #3711.)  But I don't think it's that much
trickier than properly handling non-blocking sockets in the first place.
For example:

- Application calls connect()
- Torsocks intercepts, calls connect()
- Now you have to do a fancy dance where the application is going to
  select() to wait for the connection to complete, but where torsocks
  has to get the connection to complete, *and* send the connect request,
  *and* wait for the connect reply.  (In fact, with optimistic data, you
  *don't* have to do that last step.)  So you have to play around a bit
  with the parameters to the select() call, etc.  The torsocks version
  of select, poll, etc., have to recognize when select is called, and
  *any* socket is not fully end-to-end connected, to add ready for
  write / ready for read events for those sockets as appropriate.
  If libc_select returns ready for those sockets, handle them inside
  torsocks before returning to the caller.  (But no blocking!)  In the
  case you say above, where the application is polling for read, but
  it's really just the socks5 reply that's come in, the poll() in
  torsocks will need to read the reply first and mark the socket as
  fully connected.  If there's more data (or if any other socket is
  ready), then great, return to the caller.  If not, poll() again.

The only thing optimistic data changes is that (a) you don't wait for
the Tor SOCKS5 connected response, (b) you have to be ready to eat that
response when it comes, and (c) if the response is an error, ECONNRESET
(or something) the socket.

Is it worth it?  There's *significant* (like up to 33%) improvement in

[tor-dev] Status Report - Evil Genius

2013-06-27 Thread Johannes Fürmann
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi *,

The first two weeks of working on my GSoC project[0], which has been
dubbed EvilGenius, are nearly over. and I just wanted to share a bit
of the experience I made during that time.

EvilGenius is a tool that - much like Descartes' idea of an evil
genius, it presents a complete illusion of an external world,
including other minds, to Descartes' senses, where in fact there is no
such external world in existence. (Source: Wikipedia), it distorts
the guests' view of the real internet, in short, it creates a virtual
environment that simulates censorship in order to verify OONI's
nettest modules.

So, after getting started with vagrant, the virtualization abstraction
layer used for my project, i started simulating a virtual netwotrk and
configuring the hosts to act as if they were connected to each other
and the rest of the internet through a router that is part of the
simulated environment.

the precise32 box is used as base image for all nodes, and I created
scripts that configure the network and packed it up in a nice vagrantfile.

Then I began deploying ooniprobe and oonibackend and executed the
first successful tests.

Next week, I will begin working on censoring environments.

Have a nice weekend everyone!

Johannes
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRzKDWAAoJEAgie5o3Q/JpUhAQAIt2WEpDsNyLVDuSdMBM0ETf
q0sOxdx20Q6o77J93FJ4ZBJRzpBX4mlsE4t22Dl/TPgnHu3b+DG+dvVhfCBrQOqw
wGUuA6/X/khCikAOo+f3T9fNtaHKw54phW1Tw9Yj3Gea7SBN5ZikfXKjIIHOT+lu
n0xqWVzxAxqGOETy0byDznsdgc2OfaR2NkdxzCsLlZTLtE75h/EQ3U96ZCej8lw/
xmZZUD+q+4ZCE6bBKTebufQeh0eI/rUPgo0UX1rNLgUapfCs9oLVIF+8nwBKBU0j
E2myEOUbXMCxtTURbHikRQKzGKlsXMCEqqSatik37Rc3VaeVdkxNiiyXFVRwzTr0
IwRTibzJ8hD+aqh9d9VDD6s7dIdRio42p+XzD8jswFltEyYcbBGeAFFiybdNeeE2
EFb7n7iHz7Gr1csezyma1fdqvvMHZaaeCMTqeRENQtBaazOnqDwrGJn83r/FC7QF
m8i+3eVDm8suM7CzYvuYi8YSIhF7yMVerIBOcBSZ2A929SVKtQFT4by1V2Jc4pz+
0ZJV4teS8xct0wjhpDLANBCXgnV07OO6Z6EgLhcWpJBWoZDONDaxikH7clmTlzdg
w/9ZktL3Ac3PjhSdMjOI/TKqc68+CHQ7ln1mG+YtSimMjf5Abpx1wPgLLILmzFNS
lpZ2vev2pW0DsCnvs946
=abNV
-END PGP SIGNATURE-
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Torsocks development status

2013-06-27 Thread Matthew Finkel
On Thu, Jun 27, 2013 at 03:11:23PM -0400, David Goulet wrote:
 Ian Goldberg:
  Are non-blocking sockets, select/poll/etc. (especially at connect()
  time), and optimistic data on the to-do list?
 
 Yes! Good point I should have put the todo list. So yes, non block socket 
 support.
 
 For optimistic data, it is kind of tricky.

It definitely is tricky. You just need to find the best way to have
torsocks return the least untrue response that's allowed by the OS. I'm
not going to reiterate what Ian said, but I'll just make some points
about what I did.

 I can use it for DNS resolution
 without a problem because torsocks control the complete flow of data from
 opening a SOCKS5 connection to closing it after the DNS response is received
 however for actual real data (sendmsg, send, ...) a connect is needed before 
 so
 it would means that a connect() call will return yes OK socket connected but
 where in fact it is not really true. So, when the first data are sent, there 
 is
 a possibility that the Tor connections failed or even we block for an unknown
 amount of time during the send*/write() call.

Yup, this is exactly the case (in addition to SOCKS4/A also).

 
 Now the question is, is this the kind of behavior that would be acceptable
 meaning basically lying to the caller at connect() and possibly blocking I/O
 calls and returning something like ECONNRESET or ENOTCONN if the Tor socks5
 connection fails.
 

The main problem I foresee with this is when torsocks wraps a program that
does not fully implement error handling or does not implement it
correctly. And, to be honest, I don't think you can let potentially
faulty programs influence the features of *your* program (too much).

For what it's worth, I returned ENOTCONN and EBADF, but I think ENOTCONN
is the most descriptive, I'm just not sure most programs check for it
after a send()/write().

 This is *real* tricky especially with non blocking socket, if torsocks needs 
 to
 do some possible blocking call for the SOCKS5 replies during an I/O call from
 the caller that is not suppose to block. Furthermore, having pending data that
 *might* come at any time on the connection from the SOCKS5 negotiation, the
 caller could put the file descriptor in poll() mode, wake up and try to 
 receive
 the data but where in fact it's the socks5 reply... it's possible to handle 
 that
 but it seems here a VERY intrusive behavior. Does optimistic data worth it 
 here
 vis-a-vis the complexity of handling that it and high intrusiveness ?

Well, you actually have more guarantees than you may think (unless I
misunderstand you). You know torsocks will send the SOCKS5/4{,A) request
and you know that before torsocks returns anything to the client
application, torsocks *must* receive a response from Tor regarding the
success or failure of establishing the proxy connection. As such, if you
receive optdata from the client app and pass it to Tor which then will
pass it to the endpoint (if possible), you know Tor *must* return a
SOCKS reply *before* you receive any client data, so you simply read
that off the buffer and then handle the connection in an appropriate
manner. Simple. :)

Regarding poll(), torsocks really needs to wrap the multiplexing I/O
syscalls ({p,}poll, {p,}select, epoll, kqueue, etc) or else you will
run into some major problems (select() and poll() being much more
important than the others). This is intrusive, but it's only a single
write request (for all values of write).

Personally, I think the most important feature of the optdata
implementation is that you make it configurable. 

 
 Cheers!
 David
 
  
  Thanks,
  
 - Ian
 

- Matt
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Torsocks development status

2013-06-27 Thread David Goulet


Matthew Finkel:
 On Thu, Jun 27, 2013 at 03:11:23PM -0400, David Goulet wrote:
 Ian Goldberg:
 Are non-blocking sockets, select/poll/etc. (especially at connect()
 time), and optimistic data on the to-do list?

 Yes! Good point I should have put the todo list. So yes, non block socket 
 support.

 For optimistic data, it is kind of tricky.
 
 It definitely is tricky. You just need to find the best way to have
 torsocks return the least untrue response that's allowed by the OS. I'm
 not going to reiterate what Ian said, but I'll just make some points
 about what I did.
 
 I can use it for DNS resolution
 without a problem because torsocks control the complete flow of data from
 opening a SOCKS5 connection to closing it after the DNS response is received
 however for actual real data (sendmsg, send, ...) a connect is needed before 
 so
 it would means that a connect() call will return yes OK socket connected 
 but
 where in fact it is not really true. So, when the first data are sent, there 
 is
 a possibility that the Tor connections failed or even we block for an unknown
 amount of time during the send*/write() call.
 
 Yup, this is exactly the case (in addition to SOCKS4/A also).

Actually, I did not see any reasons why this rewrite should support SOCKS4. Is
there ?

 

 Now the question is, is this the kind of behavior that would be acceptable
 meaning basically lying to the caller at connect() and possibly blocking I/O
 calls and returning something like ECONNRESET or ENOTCONN if the Tor socks5
 connection fails.

 
 The main problem I foresee with this is when torsocks wraps a program that
 does not fully implement error handling or does not implement it
 correctly. And, to be honest, I don't think you can let potentially
 faulty programs influence the features of *your* program (too much).
 
 For what it's worth, I returned ENOTCONN and EBADF, but I think ENOTCONN
 is the most descriptive, I'm just not sure most programs check for it
 after a send()/write().
 
 This is *real* tricky especially with non blocking socket, if torsocks needs 
 to
 do some possible blocking call for the SOCKS5 replies during an I/O call from
 the caller that is not suppose to block. Furthermore, having pending data 
 that
 *might* come at any time on the connection from the SOCKS5 negotiation, the
 caller could put the file descriptor in poll() mode, wake up and try to 
 receive
 the data but where in fact it's the socks5 reply... it's possible to handle 
 that
 but it seems here a VERY intrusive behavior. Does optimistic data worth it 
 here
 vis-a-vis the complexity of handling that it and high intrusiveness ?
 
 Well, you actually have more guarantees than you may think (unless I
 misunderstand you). You know torsocks will send the SOCKS5/4{,A) request
 and you know that before torsocks returns anything to the client
 application, torsocks *must* receive a response from Tor regarding the
 success or failure of establishing the proxy connection. As such, if you
 receive optdata from the client app and pass it to Tor which then will
 pass it to the endpoint (if possible), you know Tor *must* return a
 SOCKS reply *before* you receive any client data, so you simply read
 that off the buffer and then handle the connection in an appropriate
 manner. Simple. :)

Yup agree. Actually, the question here was more about if supporting optdata
worth that non trivial effort but 33% is quite a big factor to consider for
performance :).

 
 Regarding poll(), torsocks really needs to wrap the multiplexing I/O
 syscalls ({p,}poll, {p,}select, epoll, kqueue, etc) or else you will
 run into some major problems (select() and poll() being much more
 important than the others). This is intrusive, but it's only a single
 write request (for all values of write).

Can you detail why it's very important? You did some hacking in the old code
base and I would like to know your experience on that. What possible major
problems? What happens if it's not hijacked?

 
 Personally, I think the most important feature of the optdata
 implementation is that you make it configurable. 

By configurable you mean disabled or not ? What else is there to configure?

Thanks!
David

 

 Cheers!
 David


 Thanks,

- Ian

 
 - Matt
 ___
 tor-dev mailing list
 tor-dev@lists.torproject.org
 https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev



signature.asc
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


Re: [tor-dev] Torsocks development status

2013-06-27 Thread Matthew Finkel
On Thu, Jun 27, 2013 at 05:39:08PM -0400, David Goulet wrote:
 
 
 Matthew Finkel:
  On Thu, Jun 27, 2013 at 03:11:23PM -0400, David Goulet wrote:
  Ian Goldberg:
  Are non-blocking sockets, select/poll/etc. (especially at connect()
  time), and optimistic data on the to-do list?
 
  Yes! Good point I should have put the todo list. So yes, non block socket 
  support.
 
  For optimistic data, it is kind of tricky.
  
  It definitely is tricky. You just need to find the best way to have
  torsocks return the least untrue response that's allowed by the OS. I'm
  not going to reiterate what Ian said, but I'll just make some points
  about what I did.
  
  I can use it for DNS resolution
  without a problem because torsocks control the complete flow of data from
  opening a SOCKS5 connection to closing it after the DNS response is 
  received
  however for actual real data (sendmsg, send, ...) a connect is needed 
  before so
  it would means that a connect() call will return yes OK socket connected 
  but
  where in fact it is not really true. So, when the first data are sent, 
  there is
  a possibility that the Tor connections failed or even we block for an 
  unknown
  amount of time during the send*/write() call.
  
  Yup, this is exactly the case (in addition to SOCKS4/A also).
 
 Actually, I did not see any reasons why this rewrite should support SOCKS4. Is
 there ?

As far as I know, latency is probably the reason SOCKS4 is still useful,
but you can leave it on the TODO list as patches welcome if you
don't think it's too important.

  
  Regarding poll(), torsocks really needs to wrap the multiplexing I/O
  syscalls ({p,}poll, {p,}select, epoll, kqueue, etc) or else you will
  run into some major problems (select() and poll() being much more
  important than the others). This is intrusive, but it's only a single
  write request (for all values of write).
 
 Can you detail why it's very important? You did some hacking in the old code
 base and I would like to know your experience on that. What possible major
 problems? What happens if it's not hijacked?

Hrm. So my initial response was Everything breaks :) but then I
thought about this and that's actually not true, at all. The real benefit
to hijacking them is to progress the SOCKS handshake with a single
select()/poll() from the client app rather than multiple calls. So, in
retrospect, I'm not sure how important this is. I now realize I was
actually thinking about another bug.

 
  
  Personally, I think the most important feature of the optdata
  implementation is that you make it configurable. 
 
 By configurable you mean disabled or not ? What else is there to configure?

Yeah, sorry, I only meant the ability to enable/disable it, unless you
can think of other nifty features to add.

 
 Thanks!
 David
 
  
 
  Cheers!
  David
 
 
  Thanks,
 
 - Ian
 
  
  - Matt
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev


[tor-dev] Status Report - Steganography browser addon a GSOC 2013 project

2013-06-27 Thread Hareesan
Hi all,
In the first 4 weeks of my GSOC project steganography browser add-on,
I have implemented the basic UI parts in first two weeks, and later I
spent more time on web content downloading part. Up to now, I have
finished downloading image and get them as bytes. Now when you right
click on an image and click on test button, you will be getting the
byte format of image which can be used with steganography algorithms
to write messages. Since I'm mainly concentrating on features rather
than algorithms now,
my plan for next 2 week is to extend the file downloading for video
and sound files.

You can find the source code under git repository [1].

[1] https://github.com/rharishan/Steganography-Browser/

--
Hareesan
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev