United States law is opt-in for Fortune 500 companies.
2012/12/14 Jeffrey Walton noloa...@gmail.com
On Thu, Dec 13, 2012 at 7:52 AM, Philip Whitehouse phi...@whiuk.com
wrote:
I restate my email's second point.
Google is indexing robots.txt because (from all the examples I can see)
That paragraph says pretty much the exact opposite of what you understood.
Also, could we please stop refuting points nobody even made in the first
place? OP never claimed this to be a vulnerability, nor ever said
robots.txt is a proper security mechanism to hide files in public web
directories.
...@lists.grok.org.uk] On Behalf Of Christoph Gruber
Sent: Wednesday, December 12, 2012 3:19 AM
To: full-disclosure@lists.grok.org.uk
Subject: Re: [Full-disclosure] Google's robots.txt handling
On 12.12.2012 at 00:23 Lehman, Jim jim.leh...@interactivedata.com wrote:
It is possible to use white listing
I restate my email's second point.
Google is indexing robots.txt because (from all the examples I can see)
robots.txt doesn't contain a line to disallow indexing of robots.txt
It is possible that some web sites provide actual content in a file that
happens to be called robots.txt (e.g a
On Thu, Dec 13, 2012 at 7:52 AM, Philip Whitehouse phi...@whiuk.com wrote:
I restate my email's second point.
Google is indexing robots.txt because (from all the examples I can see)
robots.txt doesn't contain a line to disallow indexing of robots.txt
It is possible that some web sites
...@lists.grok.org.uk
[mailto:full-disclosure-boun...@lists.grok.org.uk] On Behalf Of Hurgel Bumpf
Sent: Monday, December 10, 2012 11:26 AM
To: full-disclosure@lists.grok.org.uk
Subject: [Full-disclosure] Google's robots.txt handling
Hi list,
i tried to contact google, but as they didn't answer my
On 12.12.2012 at 00:23 Lehman, Jim jim.leh...@interactivedata.com wrote:
It is possible to use white listing for robots.txt. Allow what you want
google to index and deny everything else. That way google doesn't make you a
goole dork target and someone browsing to your robots.txt file doesn't
I wouldn't consider this an issue. If Google didn't do this, someone
else would have (e.g. my rather old http://www.aushack.com/robanukah/
does it but I never bothered to index the web at large). I believe it
was suggested to Shodan and others, so it was only a matter of time.
If anything, Google
What we need is a robots2.txt that defines what users are allowed to access
the robots.txt file.
Problem solved.
On Mon, Dec 10, 2012 at 11:33 PM, Gynvael Coldwind gynv...@coldwind.plwrote:
Hey,
Here is an example:
An admin has a public webservice running with folders containing
This is not a strong argument. When you opt out of marketing companies store
your email on a blacklist. It's necessary.
If the contents is publicly visible then it is not a good place to put such
information you highlight below.
Moreover it only needs to be in robots.txt if its browsable. If
On Mon, Dec 10, 2012 at 3:21 PM, James Lay j...@slave-tothe-box.net wrote:
On 2012-12-10 12:25, Hurgel Bumpf wrote:
Hi list,
i tried to contact google, but as they didn't answer my email, i do
forward this to FD.
This security feature is not cleary a google vulnerability, but
On 10/12/12 19:25, Hurgel Bumpf wrote:
I tried to contact google, but as they didn't answer my email, I do forward
this to FD.
This shouldn't be a discussion about bad practice but the google feature
itself.
I seem to recall that the robots.txt exclusion standard was fairly
common before
On 2012-12-10 12:25, Hurgel Bumpf wrote:
Hi list,
i tried to contact google, but as they didn't answer my email, i do
forward this to FD.
This security feature is not cleary a google vulnerability, but
exposes websites informations that are not really intended to be
public.
/From/: Hurgel Bumpf l0rd_lunatic () yahoo com
/Date/: Mon, 10 Dec 2012 19:25:39 + (GMT)
Hi list,
i tried to contact google, but as they didn't answer my email, i do forward
this to FD.
This security feature is
Coldwind is right, u r talking about security through obscurity.
If u tell a pentester that u r using joomla and php together, he/she
will try yourwebsite.com/administrator
Since if u r ignorant and havent blocked access to it, your joomla
access page will show up and hydra/brutus will be able
If I understand the OP correctly, he is not stating that listing something
in robots.txt would make it inaccessible, but rather that Google indexes
the robots.txt files themselves, and makes the contexts of those available
for query. So, in a way, they make it easier for Google search results
Is this the case even when there is an entry in robots.txt for robots.txt
Philip Whitehouse
On 11 Dec 2012, at 12:22, Ulisses Montenegro ulisses.montene...@gmail.com
wrote:
If I understand the OP correctly, he is not stating that listing something in
robots.txt would make it inaccessible,
Hi list,
i tried to contact google, but as they didn't answer my email, i do forward
this to FD.
This security feature is not cleary a google vulnerability, but exposes
websites informations that are not really intended to be public.
(Additionally i have to say that i advocate robots.txt
On 2012-12-10 12:25, Hurgel Bumpf wrote:
Hi list,
i tried to contact google, but as they didn't answer my email, i do
forward this to FD.
This security feature is not cleary a google vulnerability, but
exposes websites informations that are not really intended to be
public.
Hey,
Here is an example:
An admin has a public webservice running with folders containing
sensitive informations. Enter these folders in his robots.txt and
protect them from the indexing process of spiders. As he doesn't
want the /admin/ gui to appear in the search results he also
20 matches
Mail list logo
