Re: [tor-dev] Tor project automation work
On Mon, 09 Dec 2013, Georg Koppen wrote: Hi Nicolas, some remarks are below. Hi Georg, Thanks for your remarks. Nicolas Vigier: In order to help me doing that, I'm very interested to receive from developers of any tor components : - a description or ticket number of bugs that you wish could have been detected earlier with automated tests https://trac.torproject.org/projects/tor/ticket/8143 comes to mind here. Ok. We should soon be able to push tor browser to Mozilla's Try servers, which will run all the unit tests, so we should be able to detect bugs like #8143. https://wiki.mozilla.org/ReleaseEngineering/TryServer - any wish or specific needs that you may have You write: Tor Browser includes some patches to make the build reproducible. We could have a test that check the reproducibility of the build by building the browser twice. While this is indeed a good idea, it won't be enough as we had bugs in the past that were only visible when builds on different machines got compared. So, what I'd like to have (in addition to running browser builds twice? on different machines?) are tests that cover specific bugs we avoided (see: the Remaining Build Reproducibility Issues in Mike's blog post covering the technical details of the Gitian build) or tracked down. See: https://trac.torproject.org/projects/tor/ticket/10159 for an example for the latter. (There, one could write a test that automates the creation of the browser.manifest which would eventually (i.e. if run a couple of times) show whether this bug still exists or whether not).) Ok, we can have a test rebuilding several times the files that are most likely to become non deterministic. However it would be better if we can find some way to trigger those non-deterministic builds with only two builds. Maybe we can try something like this : - a library that we put in LD_PRELOAD as a wrapper on readdir to return directory entries in random order instead of inodes number, so it's more likely to be different on each build. - a python library that we put in PYTHONPATH, to override dict.iterkeys to return keys in random order. If I understand ticket #10159 correctly, it was caused by iterkeys returning keys in the same order most of the time, but not all the times. - anything else that you think might be useful for me to know You write: We can produce some packages for Tor Browser, to make testing of the browser easier. In which regard is it easier to test the browser if you have packages? Hmm yes, it does not really help in that regard, I've removed this. What can help is packages nightly builds, as it makes it easy for someone to update their tor browser every day to act as a beta tester. And how should that look like outside of the Tor Browser Bundle? I fear we create extra bugs if we move the browser outside the environment it will be used in (i.e. the TBB) just for testing purposes. Even if we won't create extra bugs this way we might miss some. To be clear, running the tests you call Usablility tests and Reproducible build test outside the bundle, seems fine to me, while my concerns apply to the fingerprinting and privacy tests. What it would look like is similar to what they use in tails : https://tails.boum.org/contribute/release_process/iceweasel/ But you are right that we can miss some bugs if we only test it outside TBB, so we should also run the tests in TBB. The test helpers are a good idea. You might want to rename Cookies tool to something like Identifier tool matching the Tor Browser specification more closely (especially as you actually mean Identifier tool as Later versions could be extended to also use other techniques for storing informations in the browser indicates). Ok, updated. An other thing that can be done is keeping an archive of all builds for 3 months (or more depending on space available), so that git bisect can be done more quickly. Would that be helpful ? pgpmP9AIkHR6Z.pgp Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Tor project automation work
Nicolas Vigier: Ok, we can have a test rebuilding several times the files that are most likely to become non deterministic. However it would be better if we can find some way to trigger those non-deterministic builds with only two builds. Maybe we can try something like this : - a library that we put in LD_PRELOAD as a wrapper on readdir to return directory entries in random order instead of inodes number, so it's more likely to be different on each build. - a python library that we put in PYTHONPATH, to override dict.iterkeys to return keys in random order. If I understand ticket #10159 correctly, it was caused by iterkeys returning keys in the same order most of the time, but not all the times. I would suggest using reverse order instead of random order. Random order has the probability of being just like the original order. For readdir(3), have a look at: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719845#39 -- Lunar lu...@torproject.org signature.asc Description: Digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] InjectSOCKS: 2nd try
Hello David, thank you for your reaction. I've read some information about Torsocks now and yes, it seems to be similar. Unfortunately the information on Torsocks's homepage is rather short. So I can't tell you if the internal technology is similar. It tells that it explicitly rejects UDP traffic. It would be possible to block UDP traffic in InjectSOCKS as well. By the way, why does Tor not support UDP via SOCKS? The Torsocks documents also say that it blocks local traffic as well. While implementing InjectsSOCKS I saw that some Windows software needs local traffic in order to work for internal inter process communication (maybe Internet Explorer - not sure anymore). So I've explicitly prevented using the SOCKS server for local traffic (wouldn't make sense) and don't reject it - otherwise this software wouldn't work anymore. Do you think you can put your code into a git repository (github, gitourious, ...). That would be *very* helpful to review/contribute and track changes. Unfortunately I'm not familiar with git. However, the source code is rather small. The most important parts are in InjectSOCKS_DLL.c The sources are already commented a little bit, but I think that I'll add Visual C++ XML or doxygen style documentation. This way it's easier to understand and review/analyze the code. For the moment I'd like to suggest sending me any change suggestions or questions via e-mail. If we detect that this is not manageable, we can still switch to a different solution. By the way, I've released V1.1 of InjectSOCKS now. This adds the BSD license text and a dialog for creating a Windows shortcut file to call InjectSOCKS correctly. The dialog pops up when you start InjectSOCKS without any parameter. This helps users not familiar with the command line. Known bug: There is a small possible memory leak in the function CreateLink of that dialog; it will be fixed in the next release :-) Regards, ghostmaker ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] fteproxy v0.2.1
Hi dcf1/asn, Yesterday I tagged release v0.2.1 of fteproxy. In this release I focused on breaking away from heavyweight dependencies: OpenFST and boost. Cross-platform PTTBB binaries are available on the fteproxy website: https://fteproxy.org/download. asn - Please have a look at the code and confirm that you're happy. I had to make a few changes to the Python modules written in C++, as I can no longer rely upon boost::python. I've also added additional information to the function-level descriptions, which should help explain the significance of the language theoretical algorithms. It is also worth noting that the pseudocode is available in Appendix A of the full version of our paper [0]. dcf1 - I finally have the gitian build process producing vanilla binaries. However, I don't see anything specific to pluggable transports in [1] or [2]. Is there a fork of this code that has logic for building the pluggable transports? Or is that still an open task? -Kevin [0] http://eprint.iacr.org/2012/494 [1] https://gitweb.torproject.org/builders/tor-browser-bundle.git [2] https://gitweb.torproject.org/builders/gitian-builder.git ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] fteproxy v0.2.1
On Tue, Dec 10, 2013 at 07:38:21PM -0800, Kevin P Dyer wrote: dcf1 - I finally have the gitian build process producing vanilla binaries. However, I don't see anything specific to pluggable transports in [1] or [2]. Is there a fork of this code that has logic for building the pluggable transports? Or is that still an open task? It's in a different repo. https://trac.torproject.org/projects/tor/ticket/9444 https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/shortlog/refs/heads/pt https://lists.torproject.org/pipermail/tor-dev/2013-December/005913.html David ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
