[tor-dev] [GSoC '16] Exitmap Improvements Project - Status Report #3
Hi everyone! For the past week, I have shifted my focus to the main sub-project of continuous scanning in Exitmap. What we'll be trying to achieve is having the Exitmap utility running in the background and running the various modules. This report recounts one week worth of work since my 2nd report was on 26th June. Work done: --- 1. Wire-framed the structure of how continuous scanning will work. Major components that will be focused on are (a) Periodically updating the network consensus to scan for new or previously invisible exit nodes for malicious behaviour, (b) A mechanism that keeps a close eye on some exit nodes while does not care much for nascent exit relays or well established guard nodes, and (c) Add as much randomization as possible for these scans to make it more difficult for a malicious exit relay to distinguish regular traffic from scanner activity. 2. IRC meeting with my primary mentor Philipp Winter (phw) to discuss some of the features and how will go about implementing them. Some points arrived at from the discussion: a. Need to create an asynchronous element for periodically fetching consensus data and to integrate the element with the existing Exitmap functionality. b. Find a way to update the Tor micro-descriptors and use new consensus for the modules. I still need to find a way to programmatically force the Tor client to update its consensus at the moment specified. Damian Johnson (atagar) advised me to use the FetchDirInfoEarly parameters that helps. If anyone has any more information about this, it would help a lot. I will be scouring through the control-spec for the same. c. Create an algorithm to give more weightage to some exit nodes from consensus depending on what the criteria is. We need to come up with a selection mechanism that selects nodes visible in the say 0-24 hours time frame but is not a node that has just come or has been around the Tor network for some time. Some research still needed on this aspect. Have a great weekend everyone! -- Mridul (mtyamantau) = PGP keyID: 0xb716e33ab6d0a653 ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSoC 2016] Exitmap Improvements Project - Status Report #1
Hey everyone! Here a few points from my initial 2 weeks at working on Exitmap improvements project. 1. Gone through the Exitmap code base and understood the working. Primarily identified the Tor Browser actions that can be replaced for better automation. 2. Currently there is a simple HTTP header that is being used for a simple GET request. As suggested in the proposal, using Selenium seems like the right choice. I have read about the RC and WebDriver and have decided to go forward with the latter as it allows for programmable actions as supported by the Firefox browser and also has Python support, allowing for better integration with the existing scanner. 2a. Studied the basics of the Selenium Webdriver and learnt simple browser automation techniques on Firefox - interaction with the webpage and other pool of actions. 2b. Ported basic actions to Tor Browser. Facing some issues with integration which I hope to rectify in a day or two and push. Next two weeks will involve: 1. Integrating randomized actions on Tor Browser with Exitmap. Creating a pool of possible ways to interact with the web service to make it difficult for the malicious Exit node to suspect scanner activity. 2. Add test cases to added interactions and observe changes in scanner output from the initial simple usage of urllib2 actions and assert the changes. I apologize for the delay in submitting my first report. Cheers! Mridul (mtyamantau) = PGP keyID: 0xb716e33ab6d0a653 ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [GSoC '16] Exitmap project - Final Proposal
Really sorry for replying to my own mail! The initial mail threads for reference: https://lists.torproject.org/pipermail/tor-dev/2016-March/010565.html https://lists.torproject.org/pipermail/tor-dev/2016-March/010566.html https://lists.torproject.org/pipermail/tor-dev/2016-March/010574.html On 25 March 2016 at 22:27, Mridul Malpotra <mridul.malpo...@gmail.com> wrote: > > Hey everyone! > > I have completed my proposal on the project Exitmap improvements. Sorry for submitting it late. > > Link for suggestions: https://docs.google.com/document/d/1AqjH3AsBbfesJg36SZg1frPQsdpAnJmzH8k1-G6dz70/edit?usp=sharing > > Some points that have changed from the previous discussion: > > 1. Telnet/SNMP sniffing attacks would not be appropriate for Exitmap as it is an active attack scanner compared to say HoneyConnector etc. hence it has not been included. > > 2. I have added most of the points that I had suggested above with tweaks as advised by Dr. Winter. > > > I will be submitting the proposal in an hour. > > Thank you for your time. > Sincerely, > > -- > Mridul Malpotra > Undergrad @ IIIT-Delhi > mridulmalpotra.github.io > ===== > PGP keyID: 0xb716e33ab6d0a653 -- Mridul Malpotra Undergrad @ IIIT-Delhi = PGP keyID: 0xb716e33ab6d0a653 ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSoC '16] Exitmap project - Introduction and request for comments
Hi everyone! I'm Mridul. I wish to apply for the Exitmap improvements project mentored by Dr. Philipp Winter for the Google Summer of Code 2016. My current IRC handle is mtyamantau. Contents 1. Introduction - About myself and experience with Tor 2. Exitmap - Current progress and questions 3. GSoC - Rough proposal structure and questions 1. Introduction - About myself and experience with Tor -- I'm Mridul Malpotra, currently in my senior year pursuing bachelors in Computer Science from IIIT Delhi, India. My interests primarily lie in computer networks and network security, specifically anonymous networks like Tor and I2P. Through my now 1-year long undergraduate thesis work under Dr. Sambuddho Chakravarty, I have had exposure to the Tor network, relevant literature and some related projects, which helped me better understand and appreciate the current research and development going on. My work involved manually setting up testbeds through testing Tor networks on our institute intranet as well as on PlanetLab (for those wondering, I had recommended Chutney and Shadow). The current private testing Tor network is running on a PlanetLab slice (iiitd_mridul2) with ~170 nodes globally and 3 directory authorities. I used the control protocol through the Stem library to help in multiple circuit creation and stream attachments for measuring performance of a software over Tor. I have also had experience with open source software, by contributing to the Non intrusive load-monitoring toolkit (NILMTK) which is based on Python and Pandas. While working there, I helped contribute code for additional features, fixed a few bugs and also worked with a few of Python's package management and documentation systems. Relevant links: github.com/nilmtk/nilmtk/commits?author=mridulmalpotra 2. Exitmap - Current progress and questions --- I recently read about Exitmap in the 'Differential Treatment of Anonymous Users' paper by Khattak et. al. The use case for fast automated scanning through Exitmap to evaluate ~1000 exit nodes was really interesting. On top of that, it fitted my use case of testing a particular software's performance over Tor. Familiarizing myself with the source code, I think I understand the basic layout for how the scanner works and appreciate the modularity of task executions. I followed the project's progress on github and have read the 'Spoiled Onions' paper by Winter et. al. In the coming 2 days, I plan to tinker around more with the code, discuss concerns, issues and/or suggestions if any, and get myself properly familiarized with the codebase. I also have certain ideas regarding what modules could be added and improvements made, some of which I have mentioned in the next section. I will also be reading the tech report on Exitmap and would be grateful if you can recommend any other resource(s) that I should be referring to. Lastly, I had a few queries related to the project and/or paper and apologize for the naivety in the questions if any. a. How was the bifurcation between stand-alone and same-process modules decided? Are there any advantages to allow for multiple forked processes for specific modules? b. For testing active attacks, can there be modules developed keeping other cleartext protocols like SNMP and Telnet in mind? Alternatively, is there a way to determine what protocols are being used over Tor and their popularity? c. How is Exitmap being crowdsourced currently? I'm interested to know how data is being collected from volunteers running the scanner. 3. GSoC - Rough proposal structure and questions Here I am listing the possible objectives that my project will be focusing on. I request your feedback and comments on the chosen topics and their descriptions. 1. Achieve autonomous scanning in Exitmap with periodic scans that, based on a certain algorithm, fetches relay descriptors and automates various subtasks for consistent data collection and verification. The main challenges that I expect will be intelligently recognizing which tasks to automate and when, and making the entire background process execution efficient in resource consumption. 2. Emulating multiple user interaction in individual modules and in Exitmap overall to provide indistinguishability to Exitmap from regular users. I will try to explore libraries for this purpose like Splinter with Selenium or BeautifulSoup with Requests that help dynamically interaction with the web resource. The main challenges that I expect will be to scale this automated testing alongside the running asynchronous jobs and making the entire scans look like genuine user interactions. Any suggestions on better ways to do this will be helpful. 3. Making the codebase more robust by adding unit test cases. I plan on using either
