You can tell analog to include a META tag as header file with the HEADERFILE
command. I know this goes at the top of the <BODY> but you could give it a shot. I
don't know how your spider (or others) handle META tags outside of the <HEAD></HEAD>
but it might work. If it doesn't you could try encapsulating it in another
<HEAD></HEAD> section (right in the middle of the <BODY>, yes!).
Of course, I realize this is all really bad HTML and illegal in all W3C HTML drafts.
I'm not suggesting this as anything close to a permanent solution, just something to
get you by for the short term.
HTH
--
Jeremy Wadsack
OutQuest Magazine
a Wadsack-Allen publication
Aengus Lawlor wrote:
> These test files aren't always in the same location (I never said
> consistency was my strong point!) and anyway, users can always override
> entries in the central analog.cfg by specifying their own .cfg file.
> (I haven't tried to put that FILEEXCLUDE in the mandatory.cfg file,
> though).
>
> The reason I gave for not spidering log reports may not be a very good
> one, though. Old Analog reports are often left on line, for a variety of
> reasons, and so they provide links to documents that no longer exist. I
> end up with 404 errors that aren't due to dead links in actual content
> pages. Our search engine can also show us what page links to any given
> page, but that information is a bit less meaningful if it includes the
> 12 archived Monthly log file reports.
>
> I'd prefer to not index Analog reports. I can deal with the problem
> fairly well by specifying my Analog report directories in the ROBOTS.TXT
> file. But it would be handy to have Analog generate the exclusion
> commands for me.
>
> Aengus
>
> ______________________________ Reply Separator _________________________________
> Subject: Re: [analog-help] ROBOTS Meta tag
> Author: [EMAIL PROTECTED] at Internet
> Date: 3/26/99 3:57 PM
>
> Why can't you just add a
>
> FILEEXCLUDE /test/*
>
> to your analog.cfg file as explained in docs/include.html
>
> Regards,
>
> Roger Brown
>
> On Fri, 26 Mar 1999, Aengus Lawlor wrote:
>
> > I'm forever creating "test" web pages that aren't linked to, so that our
> > Intranet spider doesn't find them and index them. (Security through
> > obscurity). But recently I found that some of them were in our search
> > engine, and it turned out that an Analog report had been posted and
> > indexed, and the spider followed the links in the request report to find
> > my "test" pages.
> >
> > While I immediately added the directory that the Analog report was in to
> > the servers ROBOTS.TXT file, it would be handy if Analog could
> > automatically add the Robot Exclusion meta tag to its output. (Now that
> > I've made anlgform.exe available on the server, I can't always keep
> > track where an Analog report will be posted. (And I don't have the tools
> > to modify and recompile the code with the addition).
> >
> > The Robots exclusion tag looks like this:
> >
> > <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
> >
> > It's explained in greater detail at:
> > http://info.webcrawler.com/mak/projects/robots/exclusion.html#meta
> >
> > Aengus
> >
> > --------------------------------------------------------------------
> > This is the analog-help mailing list. To unsubscribe from this
> > mailing list, send mail to [EMAIL PROTECTED]
> > with "unsubscribe analog-help" in the main BODY OF THE MESSAGE.
> > --------------------------------------------------------------------
> >
>
> --------------------------------------------------------------------
> This is the analog-help mailing list. To unsubscribe from this
> mailing list, send mail to [EMAIL PROTECTED]
> with "unsubscribe analog-help" in the main BODY OF THE MESSAGE.
> --------------------------------------------------------------------
>
> ------------------------------------------------------------------------
> Received: from nmho05u.rohmhaas.com ([136.141.252.23]) by ima1.rohmhaas.com with
> SMTP
> (IMA Internet Exchange 3.11) id 0017F4B4; Fri, 26 Mar 1999 18:06:38 -0500
> Received: by nmho05u.rohmhaas.com; id SAA04755; Fri, 26 Mar 1999 18:08:32 -0500
> (EST)
> Received: from mb3.mailbank.com(209.133.104.8) by nmho05u.rohmhaas.com via smap
> (3.2)
> id xma004749; Fri, 26 Mar 99 18:08:15 -0500
> Received: from gateway1.isite.net (gateway1.isite.net [198.207.204.66])
> by mb3.mailbank.com (8.9.1a/8.9.1) with ESMTP id PAA08100
> for <[EMAIL PROTECTED]>; Fri, 26 Mar 1999 15:06:14 -0800
> Received: from proxy1.noc.isite.net ([172.16.1.11] (may be forged)) by
> gateway1.isite.net (8.8.6/8.8) with ESMTP id OAA25563; Fri, 26 Mar 1999 14:58:24
> -0800 (PST)
> Received: from mail2.noc.isite.net (mail2.noc.isite.net [172.16.1.22]) by
> proxy1.noc.isite.net (8.8.6/8.8) with ESMTP id OAA04189; Fri, 26 Mar 1999
> 14:58:23 -0800 (PST)
> Received: (from majordom@localhost) by mail2.noc.isite.net (8.8.6/8.8) id
> OAA04052 for analog-help-localoutlist; Fri, 26 Mar 1999 14:58:10 -0800 (PST)
> Received: from proxy1.noc.isite.net (proxy1.noc.isite.net [172.16.1.11]) by
> mail2.noc.isite.net (8.8.6/8.8) with ESMTP id OAA04048 for
> <[EMAIL PROTECTED]>; Fri, 26 Mar 1999 14:58:05 -0800 (PST)
> Received: from gateway1.isite.net (gateway1.isite.net [198.207.204.66]) by
> proxy1.noc.isite.net (8.8.6/8.8) with ESMTP id OAA04185 for
> <[EMAIL PROTECTED]>; Fri, 26 Mar 1999 14:58:05 -0800 (PST)
> Received: from rogb.iserver.net (rogb.iserver.net [192.41.50.237]) by
> gateway1.isite.net (8.8.6/8.8) with ESMTP id OAA25559 for
> <[EMAIL PROTECTED]>; Fri, 26 Mar 1999 14:58:04 -0800 (PST)
> Received: from localhost (rogb@localhost) by rogb.iserver.net (8.8.5) id
> PAA24695; Fri, 26 Mar 1999 15:57:54 -0700 (MST)
> Date: Fri, 26 Mar 1999 15:57:53 -0700 (MST)
> From: Roger L Brown <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Subject: Re: [analog-help] ROBOTS Meta tag
> In-Reply-To: <>
> Message-ID: <>
> MIME-Version: 1.0
> Content-Type: TEXT/PLAIN; charset=US-ASCII
> Sender: [EMAIL PROTECTED]
> Precedence: bulk
> Reply-To: [EMAIL PROTECTED]
--------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe analog-help" in the main BODY OF THE MESSAGE.
--------------------------------------------------------------------
