Re: Google and Tor.
Thus spake Orionjur Tor-admin (tor-ad...@orionjurinform.com): This should be fixed in Torbutton 1.2.6. When you plan to release it? Well the current plan is to add support for FF4 and fix a smattering of bugs, including this one, in the 1.2.6 release. However, I am also trying to help fix bugs in 0.2.2.x, and help improve the Google Chrome APIs to allow for a Chrome Tor mode (https://trac.torproject.org/projects/tor/ticket/1770), amongst a few other things that I feel are rather important in the near term. In fact, I've been so busy lately that I haven't even fixed the issue in git or my copy of Torbutton, so rest assured that I feel the pain as much as you do. But it still may be a while. -- Mike Perry Mad Computer Scientist fscked.org evil labs pgpLcjr3A4Mhl.pgp Description: PGP signature
Re: Google and Tor.
Thus spake Robert Ransom (rransom.8...@gmail.com): On Wed, 25 Aug 2010 20:04:01 -0700 Mike Perry mikepe...@fscked.org wrote: I also question Google's threat model on this feature. Sure, they want to stop people from programmatically re-selling Google results without an API key in general, but there is A) no way people will be reselling Tor-level latency results, B) no way they can really expect determined competitors not to do competitive analysis of results using private IP ranges large enough to avoid DoS detection, C) no way that the total computational cost of the queries coming from Tor can justify denying so many users easy access to their site. If Tor exit nodes were allowed to bypass Google's CAPTCHA, someone could put up a low-bandwidth Tor exit node and then send their own automated queries directly to Google from their Tor exit's IP. Good point. However I wasn't advocating whitelisting Tor exits, I was advocating more intelligent treatment of all high user-count IP addresses, and better mechanisms of rate limiting in general. It's my understanding that a lot of NATed users also run into these captchas during search. To reduce scraping by suspect IPs, their servers could perform all sorts of browser tests to ensure that there is a full working DOM supported by javascript, which can be computationally costly to deploy by scrapers. They can also serve javascript code that performs semi-large integer factorization in the background and post the factors back with queries to rate limit scrapers computationally, or at least tip the cost ratios more in favor of just paying for an API key. Perhaps more effective, they could use various metrics to indirectly estimate the number of humans behind an IP. There are plenty of Google services and applications they provide that aren't really usable by bots. The rate of use of these non-search services per IP should provide a strong indicator of human activity behind that IP. Again, the impression I got was that if they had done the analysis on the captcha solve rate vs the query rate per IP, the cost/benefit analysis of the DoS mechanisms they apply, or the cost vs effectiveness vs user impact of alternatives, they certainly weren't willing to discuss any of this with us. They also seemed disinclined to meet to explore any realistic alternatives we could jointly develop in both Torbutton and the DoS side to help reduce the captchas and 403s experienced by our users. -- Mike Perry Mad Computer Scientist fscked.org evil labs pgpNrwJkzXL5G.pgp Description: PGP signature
Re: Google and Tor.
Mike Perry wrote: This should be fixed in Torbutton 1.2.6. Hello Mike, When you plan to release it? *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Google and Tor.
On 25/08/10 15:38, Gregory Maxwell wrote: On Wed, Aug 25, 2010 at 6:28 AM, Matthewpump...@cotse.net wrote: On numerous occasions when using Google with Tor (yes, I know there are other options like Scroogle) it claims I might be sending automated queries and gives me a CAPTCHA. Sometimes this allows me to search; other times I am caught in a loop and am constantly send back to the CAPTCHA screen. I am wondering why Google does not deal with this. I can understand that if dozens of people are using the same IP then some sites think zombies are being used. But if the IP is a Tor node then this is not the case. Google could surely exclude these Tor IPs. So my question is: why don't they? What are the politics behind their decision not to acknowledge Tor exit nodes as bona fide? Really? This isn't obvious? Would I have asked if it was obvious? People are running automated datamining queries _via tor_ in order to gain control of more IPs and avoid being blocked. What is a datamining query exactly? Is this what I would call typing some text into the search box and pressing enter? And how does entering a datamining query allow one to gain control of more IPs? And being blocked - from what? Totally confused. Even if they weren't, they'd certainly start if Google exempted tor exits. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Google and Tor.
On Wed, Aug 25, 2010 at 11:31 AM, Matthew pump...@cotse.net wrote: People are running automated datamining queries _via tor_ in order to gain control of more IPs and avoid being blocked. What is a datamining query exactly? Is this what I would call typing some text into the search box and pressing enter? And how does entering a datamining query allow one to gain control of more IPs? And being blocked - from what? Totally confused. For example— a friend of mine was querying google maps to find out their estimated travel time between every pair of US cities over some size threshold. After about a month of this they blocked her IP and she moved to using tor, spreading the traffic across many exits (which, as far as I know they never ended up blocking). People do bulk google queries to look for sites to spam (e.g. by googling for UI elements from wiki software plus keywords useful for their spammish purposes). These are the datamining things I was referring to. Another example, some people have operated fake search engines which do nothing but serve their own ads/malware and then direct the real queries back to google. I'm sure that there is a ton of potentially abusive behaviour which I've never seen or thought of but which google is aware of. I think it would be nice if captchas and blocking weren't the only anti-DOS/anti-abuse mechanisms used on the web today, but this is the world we live in. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Google and Tor.
Gregory Maxwell wrote: On Wed, Aug 25, 2010 at 11:31 AM, Matthew pump...@cotse.net wrote: People are running automated datamining queries _via tor_ in order to gain control of more IPs and avoid being blocked. I think it would be nice if captchas and blocking weren't the only anti-DOS/anti-abuse mechanisms used on the web today, but this is the world we live in. While I usually use scroogle or ixquick, on occasion I do a google query. Sometimes it works, frequently it is blocked. When they give me a captcha, I've learned to just give up right then (or maybe try with a new exit node). I have never had a successful result with a Google captcha ... it just keeps giving me new ones. So while your explanation for blocking makes sense, it doesn't explain why they don't fix their capthca. (Maybe it's tied to cookies, but I'm not going to allow google cookies for that one instance only to disable them again.) I realize there is nothing anybody on this list can do (unless a Google employee subscribes to the list). I'm just venting ... Cheers, Jim *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Google and Tor.
Thus spake Matthew (pump...@cotse.net): On numerous occasions when using Google with Tor (yes, I know there are other options like Scroogle) it claims I might be sending automated queries and gives me a CAPTCHA. Sometimes this allows me to search; other times I am caught in a loop and am constantly send back to the CAPTCHA screen. This has been a known problem with Google for ages. There are numerous ways we could improve this situation without requiring blanket exemptions for Tor Exits (such as client side puzzles, or more intelligent rate limiting algorithms that are more tolerant of our typically cookieless but legitimate users coming in large masses from the same IP). Unfortunately the DoS team at Google is unwilling to work with us to find alternate ways of limiting these captchas at the moment. Tor has many friends inside Google, but sadly the DoS team is independent enough from the rest of Google that regardles of Google's opinion of Tor or censorship circumvention, the DoS team is unwilling to devote any development resources to improving this problem, and have declined even meeting with us directly :( Astute students of human nature will note that this is the result you expect when you place a small group of people in a position of unassaillable control of a resource for security reasons... Our current solution is to automatically redirect Google Captcha requests to alternate search engines such as ixquick, scroogle, yahoo, or bing. This feature was introduced in Torbutton 1.2.5 and uses ixquick by default. However, Google's recent switch to using encrypted.google.com for SSL search caused our captcha detection code to break in Torbutton. So if you are using encrypted search and/or HTTPS Everywhere, your captchas will no longer be seamlessly redirected. This should be fixed in Torbutton 1.2.6. -- Mike Perry Mad Computer Scientist fscked.org evil labs pgpNcDwOSw9Vh.pgp Description: PGP signature
Re: Google and Tor.
On 8/25/2010 8:52 PM, Mike Perry wrote: Thus spake Matthew (pump...@cotse.net): On numerous occasions when using Google with Tor (yes, I know there are other options like Scroogle) it claims I might be sending automated queries and gives me a CAPTCHA. Sometimes this allows me to search; other times I am caught in a loop and am constantly send back to the CAPTCHA screen. This has been a known problem with Google for ages. (snip) Really? I've never had this problem until recently. For about 2 years now every Google CAPTCHA I've run into has been uneventful and let me through after the first try, only in the past month or so have I been getting caught in the CAPTCHA loop. ~Justin Aplin *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Google and Tor.
Thus spake Aplin, Justin M (jmap...@ufl.edu): On 8/25/2010 8:52 PM, Mike Perry wrote: Thus spake Matthew (pump...@cotse.net): On numerous occasions when using Google with Tor (yes, I know there are other options like Scroogle) it claims I might be sending automated queries and gives me a CAPTCHA. Sometimes this allows me to search; other times I am caught in a loop and am constantly send back to the CAPTCHA screen. This has been a known problem with Google for ages. (snip) Really? I've never had this problem until recently. For about 2 years now every Google CAPTCHA I've run into has been uneventful and let me through after the first try, only in the past month or so have I been getting caught in the CAPTCHA loop. Various horrible behaviors have come and go with this captcha system over the past 3 years or so. Sometimes you just get a 403 with no captcha, sometimes you have to solve a captcha, sometimes 2 captchas, sometimes infinite captchas, and sometimes it forgets your query and you have to start the whole process over again from a Google landing page. My point is that the whole system is problematic on a number of levels. I also personally believe that there are better ways of rate limiting and screening queries from high-user count IPs that do not involve cookies or captchas. I also question Google's threat model on this feature. Sure, they want to stop people from programmatically re-selling Google results without an API key in general, but there is A) no way people will be reselling Tor-level latency results, B) no way they can really expect determined competitors not to do competitive analysis of results using private IP ranges large enough to avoid DoS detection, C) no way that the total computational cost of the queries coming from Tor can justify denying so many users easy access to their site. This is why I'd love a chance to meet with the DoS team to discuss some of these points. However, I get the strong impression it is a very secretive group that is especially wary of discussing their methods, reasoning, or analysis and with anyone else, and is generally given a blank check to enact policy without proper in-depth cost/benefit analsysis because its actions are for security. -- Mike Perry Mad Computer Scientist fscked.org evil labs pgpGvcbwzdUPv.pgp Description: PGP signature
Re: Google and Tor.
On Wed, 25 Aug 2010 20:04:01 -0700 Mike Perry mikepe...@fscked.org wrote: I also question Google's threat model on this feature. Sure, they want to stop people from programmatically re-selling Google results without an API key in general, but there is A) no way people will be reselling Tor-level latency results, B) no way they can really expect determined competitors not to do competitive analysis of results using private IP ranges large enough to avoid DoS detection, C) no way that the total computational cost of the queries coming from Tor can justify denying so many users easy access to their site. If Tor exit nodes were allowed to bypass Google's CAPTCHA, someone could put up a low-bandwidth Tor exit node and then send their own automated queries directly to Google from their Tor exit's IP. Robert Ransom signature.asc Description: PGP signature
Re: Google and Tor
GMail doesn't do this anymore. You can sign up through Tor just fine. Yes, there was a time years ago where they were invite only :( Then they opened up. This does not refer to that historical thing. I tried making four different acct names over the span of a day about a day before I first posted this. Clearing cookies and newnym between each. Account creation tests between then and now have worked without issue. Don't know what google was up to when I posted Seems fine now. Thanks, sorry for the noise.
Re: Google and Tor
grarpamp wrote: GMail doesn't do this anymore. You can sign up through Tor just fine. Yes, there was a time years ago where they were invite only :( Then they opened up. This does not refer to that historical thing. I tried making four different acct names over the span of a day about a day before I first posted this. Clearing cookies and newnym between each. Account creation tests between then and now have worked without issue. Don't know what google was up to when I posted Seems fine now. Thanks, sorry for the noise. It may have been related to the traffic from those exit nodes that Google was seeing *at* *that* *time*. There was a time when Google's search engine would sometimes tell me something along the lines of we think you are a virus that was definitely time/exit-node dependent. (Now it is very rare that exiting from Tor does not cause me problems with Google's search.)
Re: Google and Tor
James Brown wrote: I use the gmail within Tor very easy but I have some problems sometimes with other services of Google. For maybe I couple of years it has been almost impossible for me to use Google's search via Tor. (It keeps calling me a virus.) Somebody eventually told me about Scroogle ( http://www.scroogle.org/scraper.html ) which I have had good luck with via Tor. I *think* that recently, after Google flags you as suspicious activity it allows you to proceed with a captcha *if* you accept cookies. Not a good way to remain anonymous unless you immediately delete the cookies. (When I first tried to use Tor I had some, now long forgotten, problem. Google-analytics was my motivation for solving the problem.) But about last two monthes there is problems with using the Yahoo mail through Tor. If you are talking about error 999 (Yahoo's term), I have occasionally had problems with that for a long time. Recently it seems to have become routine. You can immediately go to the captcha login for email (which I don't have trouble with from Tor) with: https://login.yahoo.com/config/login?.ab=1.done=http%3A//mail.yahoo.com (of course, Yahoo might break that link at any time) Be aware that although *login* to Yahoo mail is https, the other transmissions are in clear text. So you are exposing your email (both send and receive) to exit nodes. P.S. After seeing bao song's post, I remembered I have fiddled with Privoxy's settings to keep it from mangling Yahoo mail. But I have routed Yahoo's mail clear text straight to the Internet to avoid any exit node mischief. I send the https login via Tor because it it too difficult to separate from my other Yahoo traffic.
Re: Google and Tor
On 07/04/2009 04:12 AM, grarpamp wrote: AFAIK, google does not allow torizens to sign up for even gmail via Tor. It recently (perhaps always now?) insisted on sending a text to your cell phone to 'verify' you first. Or similarly breaking your anonymity and annoyance factor by linking you to two other email accounts via an 'invite'. So long as this proof continues to hold, I highly doubt google believes in Tor as a tool for good. Those concerned may wish to try a signup. GMail doesn't do this anymore. You can sign up through Tor just fine. -- Andrew Lewman The Tor Project pgp 0x31B0974B Website: https://torproject.org/ Blog: https://blog.torproject.org/ Identica/Twitter: torproject
Re: Google and Tor
grarpamp wrote: From some other thread: o Google was supportive of good uses of Tor, for its services AFAIK, google does not allow torizens to sign up for even gmail via Tor. It recently (perhaps always now?) insisted on sending a text to your cell phone to 'verify' you first. Or similarly breaking your anonymity and annoyance factor by linking you to two other email accounts via an 'invite'. So long as this proof continues to hold, I highly doubt google believes in Tor as a tool for good. Those concerned may wish to try a signup. I use the gmail within Tor very easy but I have some problems sometimes with other services of Google. But about last two monthes there is problems with using the Yahoo mail through Tor.
