Hi Joachim, I can use a robots.txt to allow or deny bots accessing portions of the site. But about 80% of the site has data that needs to be accessed by bots so as to not affect the search rank. Since I have enabled a CAS gateway, every request is intercepted and redirected to cas. I need to modify this so that requests from bots are not redirected to CAS.
Not sure if robots.txt is the way to go for what I'm trying to accomplish. Prasanna. On Tue, Jul 13, 2010 at 9:01 PM, Joachim Fritschi < frits...@hrz.tu-darmstadt.de> wrote: > Hi, > > i guess a classic robots.txt should solve the issue for search engines. > > Cheers, > > Joachim > > Am 13.07.2010 11:18, schrieb prasanna h: > >> Hi All, >> >> We use CAS as a gateway where each request is intercepted and redirected >> to CAS to check for a ticket. I noticed that the redirect to CAS happens >> for users as well as bots. Has anyone using CAS encountered this and if >> yes, can you let me know the solution? >> >> Right now, I'm planning to check the user-agent against a list of common >> bots to determine whether to redirect to CAS. >> >> Looking forward to your thoughts as well. >> >> Prasanna >> >> -- >> You are currently subscribed tocas-...@lists.jasig.org <mailto: >> cas-dev@lists.jasig.org> as: frits...@hrz.tu-darmstadt.de >> To unsubscribe, change settings or access archives, see >> http://www.ja-sig.org/wiki/display/JSG/cas-dev >> >> > > -- > Joachim Fritschi > Hochschulrechenzentrum (HRZ) > L1|01 Raum 248 > Petersenstr. 30 > 64287 Darmstadt > > Tel. +49 6151 16-5638 > Fax. +49 6151 16-3050 > E-Mail: frits...@hrz.tu-darmstadt.de > > -- You are currently subscribed to cas-dev@lists.jasig.org as: arch...@mail-archive.com To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-dev
