Re: [LIH] RE:shell account/robots.txt

[Shridhar Daithankar]

I guess it's difficult. Anybody can telnet to post 80 and type http request 
and get the page and also thr. any program just by sending http request to 
port 80.

How do you define browser access, as interpreted by web server? If it's going 
to be http request alone then it's difficult to stop 'non-browser' access.

Did I get your question right?

 Shridhar

On Friday 13 April 2001 23:30, shubhendu wrote:
> if i can make "robots = off"  in my local m/c and get whatever i wanted
> its not the proper way to protect webserver from the load of
> dowanloading through tools like wget ( what for robots.txt is made )
> so what could be foolproof method to prevent such  downloads
> along with provindg access to the pages through browsers
>
> shubh

----------------------------------------------
LIH is all for free speech.  But it was created
for a purpose.  Violations of the rules of
this list will result in stern action.