Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-16 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Paul A. Rubin wrote:
I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.



That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)




sighRan it from my office.  After a few hours, running with the 
maximum 100 threads, it had identified about 30,000+ links and checked 
about 16,000 of them (excluding links that required authentication, 
since I didn't have a password).  Watching progress, I know that it 
found at least a half-dozen or so broken links in that time.  So I left 
it running overnight, and when I came in today it had finished and 
wanted to know if I wanted a report.  I said yes and it got stuck in a 
massive CPU-cycle-sucking exercise that it did not survive./sigh


I read elsewhere in the thread that you had validator running.  Do you 
still need me to run a link check here?  If so, I think the job needs to 
be partitioned (scan the Wiki to a modest depth on the first pass, then 
scan deeper parts in separate runs).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-16 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Paul A. Rubin wrote:
I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.



That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)




sighRan it from my office.  After a few hours, running with the 
maximum 100 threads, it had identified about 30,000+ links and checked 
about 16,000 of them (excluding links that required authentication, 
since I didn't have a password).  Watching progress, I know that it 
found at least a half-dozen or so broken links in that time.  So I left 
it running overnight, and when I came in today it had finished and 
wanted to know if I wanted a report.  I said yes and it got stuck in a 
massive CPU-cycle-sucking exercise that it did not survive./sigh


I read elsewhere in the thread that you had validator running.  Do you 
still need me to run a link check here?  If so, I think the job needs to 
be partitioned (scan the Wiki to a modest depth on the first pass, then 
scan deeper parts in separate runs).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-16 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Paul A. Rubin wrote:
I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.



That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)




Ran it from my office.  After a few hours, running with the 
maximum 100 threads, it had identified about 30,000+ links and checked 
about 16,000 of them (excluding links that required authentication, 
since I didn't have a password).  Watching progress, I know that it 
found at least a half-dozen or so broken links in that time.  So I left 
it running overnight, and when I came in today it had finished and 
wanted to know if I wanted a report.  I said yes and it got stuck in a 
massive CPU-cycle-sucking exercise that it did not survive.


I read elsewhere in the thread that you had validator running.  Do you 
still need me to run a link check here?  If so, I think the job needs to 
be partitioned (scan the Wiki to a modest depth on the first pass, then 
scan deeper parts in separate runs).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
[EMAIL PROTECTED] wrote:

 On Wed, 14 Dec 2005, Angus Leeming wrote:
 http://validator.w3.org/checklink/
 
 Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
 anyway, I'll keep it in mind while looking at other options.

You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html

The slowness is on purpose: Otherwise the linkchecker could be misused for
DoS attacks.


Georg



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Wed, 14 Dec 2005, Angus Leeming wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Todd Denniston wrote:



Oh, and it would be a good idea for someone to make sure all the links
work correctly now...


Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').


http://validator.w3.org/checklink/



Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.


/Christian







I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.  After five minutes, it had checked about 370 links 
and detected over 2,000.  (Hard to tell how close to done it was, 
because of course it finds new links each time it parses a page.)  Might 
run faster if run on the server or on a machine closer to the server (in 
the sense of fewer hops).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Paul A. Rubin wrote:

 Does anyone know of a tool which would be useful to traverse a web site
 and check which links doesn't work?  (In this case we are primarily
 interested in links starting with 'http://wiki.lyx.org/').

 I've had good luck with Xenu Link Sleuth 
 (http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
 program (although the web site says it appears to run under Wine). 
 Tried it on the Wiki, using 30 threads (could do more) and checking 
 internal links only.

That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)

I'm not able to run it any closer than you on a windows machine. 
Actually... I guess that bit depends on our geographic locations :-)

 Might run faster if run on the server or on a machine closer to the
 server (in the sense of fewer hops).

Speed isn't really an issue here, as long as it finishes before say
version LyX 1.4 arrives ;-)

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 [EMAIL PROTECTED] wrote:
 
  On Wed, 14 Dec 2005, Angus Leeming wrote:
  http://validator.w3.org/checklink/
  
  Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
  anyway, I'll keep it in mind while looking at other options.
 
 You can make exceptions in robots.txt for exactly this client. See
 http://validator.w3.org/docs/checklink.html

Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Todd Denniston

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Georg Baum wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Angus Leeming wrote:


http://validator.w3.org/checklink/


Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
anyway, I'll keep it in mind while looking at other options.


You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html



Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian



Did you notice the install it locally link?
http://validator.w3.org/docs/checklink.html#install
IIF you have all the prereqs installed you could mod the checker to take out 
the 1 second wait it has and run that instead of the one they have online.


Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
Todd Denniston wrote:

 Did you notice the install it locally link?
 http://validator.w3.org/docs/checklink.html#install
 IIF you have all the prereqs installed you could mod the checker to take
 out the 1 second wait it has and run that instead of the one they have
 online.

But you should only do that if you restrict the check to local links.
Otherwise it is rude to the ckecked servers. Anyway, the delay should not
matter at all if you install it locally and run it from a cron job every
few days.


Georg




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Todd Denniston wrote:

 [EMAIL PROTECTED] wrote:
  On Thu, 15 Dec 2005, Georg Baum wrote:
  
  
 [EMAIL PROTECTED] wrote:
 
 
 On Wed, 14 Dec 2005, Angus Leeming wrote:
 
 http://validator.w3.org/checklink/
 
 Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
 anyway, I'll keep it in mind while looking at other options.
 
 You can make exceptions in robots.txt for exactly this client. See
 http://validator.w3.org/docs/checklink.html
  
  
  Thanks for the info, I've modified robots.txt now and I'm looking into it
  further (trying to see if I can validate the HTML while I'm at it).
  
  /Christian
  
 
 Did you notice the install it locally link?

Yes... some dependencies aren't satisfied at the server where it should
ideally be installed. I'll ask Lars about this.

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 Todd Denniston wrote:
 
  Did you notice the install it locally link?
  http://validator.w3.org/docs/checklink.html#install
  IIF you have all the prereqs installed you could mod the checker to take
  out the 1 second wait it has and run that instead of the one they have
  online.
 
 But you should only do that if you restrict the check to local links.
 Otherwise it is rude to the ckecked servers. Anyway, the delay should not
 matter at all if you install it locally and run it from a cron job every
 few days.

I've got it working locally now (the dependencies that were missing aren't 
critical).

I'm also only interested in checking local links, so that should be fine.

Not sure this has to be run very often though... I thought just now and 
then. There's not that much traffic on the wiki anyway [*].

Thanks for the help guys!
/Christian

[1] One idea might be to run the test only on pages that have actually 
changed. 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 Todd Denniston wrote:
 
  Did you notice the install it locally link?
  http://validator.w3.org/docs/checklink.html#install
  IIF you have all the prereqs installed you could mod the checker to take
  out the 1 second wait it has and run that instead of the one they have
  online.
 
 But you should only do that if you restrict the check to local links.

You wouldn't happen to know how to do that? I can't figure it out from the 
manual, and perl isn't my strong suit.

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
[EMAIL PROTECTED] wrote:

 On Wed, 14 Dec 2005, Angus Leeming wrote:
 http://validator.w3.org/checklink/
 
 Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
 anyway, I'll keep it in mind while looking at other options.

You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html

The slowness is on purpose: Otherwise the linkchecker could be misused for
DoS attacks.


Georg



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Wed, 14 Dec 2005, Angus Leeming wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Todd Denniston wrote:



Oh, and it would be a good idea for someone to make sure all the links
work correctly now...


Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').


http://validator.w3.org/checklink/



Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.


/Christian







I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.  After five minutes, it had checked about 370 links 
and detected over 2,000.  (Hard to tell how close to done it was, 
because of course it finds new links each time it parses a page.)  Might 
run faster if run on the server or on a machine closer to the server (in 
the sense of fewer hops).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Paul A. Rubin wrote:

 Does anyone know of a tool which would be useful to traverse a web site
 and check which links doesn't work?  (In this case we are primarily
 interested in links starting with 'http://wiki.lyx.org/').

 I've had good luck with Xenu Link Sleuth 
 (http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
 program (although the web site says it appears to run under Wine). 
 Tried it on the Wiki, using 30 threads (could do more) and checking 
 internal links only.

That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)

I'm not able to run it any closer than you on a windows machine. 
Actually... I guess that bit depends on our geographic locations :-)

 Might run faster if run on the server or on a machine closer to the
 server (in the sense of fewer hops).

Speed isn't really an issue here, as long as it finishes before say
version LyX 1.4 arrives ;-)

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 [EMAIL PROTECTED] wrote:
 
  On Wed, 14 Dec 2005, Angus Leeming wrote:
  http://validator.w3.org/checklink/
  
  Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
  anyway, I'll keep it in mind while looking at other options.
 
 You can make exceptions in robots.txt for exactly this client. See
 http://validator.w3.org/docs/checklink.html

Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Todd Denniston

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Georg Baum wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Angus Leeming wrote:


http://validator.w3.org/checklink/


Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
anyway, I'll keep it in mind while looking at other options.


You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html



Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian



Did you notice the install it locally link?
http://validator.w3.org/docs/checklink.html#install
IIF you have all the prereqs installed you could mod the checker to take out 
the 1 second wait it has and run that instead of the one they have online.


Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
Todd Denniston wrote:

 Did you notice the install it locally link?
 http://validator.w3.org/docs/checklink.html#install
 IIF you have all the prereqs installed you could mod the checker to take
 out the 1 second wait it has and run that instead of the one they have
 online.

But you should only do that if you restrict the check to local links.
Otherwise it is rude to the ckecked servers. Anyway, the delay should not
matter at all if you install it locally and run it from a cron job every
few days.


Georg




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Todd Denniston wrote:

 [EMAIL PROTECTED] wrote:
  On Thu, 15 Dec 2005, Georg Baum wrote:
  
  
 [EMAIL PROTECTED] wrote:
 
 
 On Wed, 14 Dec 2005, Angus Leeming wrote:
 
 http://validator.w3.org/checklink/
 
 Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
 anyway, I'll keep it in mind while looking at other options.
 
 You can make exceptions in robots.txt for exactly this client. See
 http://validator.w3.org/docs/checklink.html
  
  
  Thanks for the info, I've modified robots.txt now and I'm looking into it
  further (trying to see if I can validate the HTML while I'm at it).
  
  /Christian
  
 
 Did you notice the install it locally link?

Yes... some dependencies aren't satisfied at the server where it should
ideally be installed. I'll ask Lars about this.

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 Todd Denniston wrote:
 
  Did you notice the install it locally link?
  http://validator.w3.org/docs/checklink.html#install
  IIF you have all the prereqs installed you could mod the checker to take
  out the 1 second wait it has and run that instead of the one they have
  online.
 
 But you should only do that if you restrict the check to local links.
 Otherwise it is rude to the ckecked servers. Anyway, the delay should not
 matter at all if you install it locally and run it from a cron job every
 few days.

I've got it working locally now (the dependencies that were missing aren't 
critical).

I'm also only interested in checking local links, so that should be fine.

Not sure this has to be run very often though... I thought just now and 
then. There's not that much traffic on the wiki anyway [*].

Thanks for the help guys!
/Christian

[1] One idea might be to run the test only on pages that have actually 
changed. 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

 Todd Denniston wrote:
 
  Did you notice the install it locally link?
  http://validator.w3.org/docs/checklink.html#install
  IIF you have all the prereqs installed you could mod the checker to take
  out the 1 second wait it has and run that instead of the one they have
  online.
 
 But you should only do that if you restrict the check to local links.

You wouldn't happen to know how to do that? I can't figure it out from the 
manual, and perl isn't my strong suit.

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
[EMAIL PROTECTED] wrote:

> On Wed, 14 Dec 2005, Angus Leeming wrote:
>> http://validator.w3.org/checklink/
> 
> Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
> anyway, I'll keep it in mind while looking at other options.

You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html

The slowness is on purpose: Otherwise the linkchecker could be misused for
DoS attacks.


Georg



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Paul A. Rubin

[EMAIL PROTECTED] wrote:

On Wed, 14 Dec 2005, Angus Leeming wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Todd Denniston wrote:



Oh, and it would be a good idea for someone to make sure all the links
work correctly now...


Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').


http://validator.w3.org/checklink/



Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.


/Christian







I've had good luck with Xenu Link Sleuth 
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
program (although the web site says it appears to run under Wine). 
Tried it on the Wiki, using 30 threads (could do more) and checking 
internal links only.  After five minutes, it had checked about 370 links 
and detected over 2,000.  (Hard to tell how close to done it was, 
because of course it finds new links each time it parses a page.)  Might 
run faster if run on the server or on a machine closer to the server (in 
the sense of fewer hops).


Paul



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Paul A. Rubin wrote:

> >>>Does anyone know of a tool which would be useful to traverse a web site
> >>>and check which links doesn't work?  (In this case we are primarily
> >>>interested in links starting with 'http://wiki.lyx.org/').

> I've had good luck with Xenu Link Sleuth 
> (http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only 
> program (although the web site says it appears to run under Wine). 
> Tried it on the Wiki, using 30 threads (could do more) and checking 
> internal links only.

That sounds good. Could you let it run to completion and let me know the 
result? (Send the log privately if it's too big)

I'm not able to run it any closer than you on a windows machine. 
Actually... I guess that bit depends on our geographic locations :-)

> Might run faster if run on the server or on a machine closer to the
> server (in the sense of fewer hops).

Speed isn't really an issue here, as long as it finishes before say
version LyX 1.4 arrives ;-)

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

> [EMAIL PROTECTED] wrote:
> 
> > On Wed, 14 Dec 2005, Angus Leeming wrote:
> >> http://validator.w3.org/checklink/
> > 
> > Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
> > anyway, I'll keep it in mind while looking at other options.
> 
> You can make exceptions in robots.txt for exactly this client. See
> http://validator.w3.org/docs/checklink.html

Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Todd Denniston

[EMAIL PROTECTED] wrote:

On Thu, 15 Dec 2005, Georg Baum wrote:



[EMAIL PROTECTED] wrote:



On Wed, 14 Dec 2005, Angus Leeming wrote:


http://validator.w3.org/checklink/


Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
anyway, I'll keep it in mind while looking at other options.


You can make exceptions in robots.txt for exactly this client. See
http://validator.w3.org/docs/checklink.html



Thanks for the info, I've modified robots.txt now and I'm looking into it
further (trying to see if I can validate the HTML while I'm at it).

/Christian



Did you notice the install it locally link?
http://validator.w3.org/docs/checklink.html#install
IIF you have all the prereqs installed you could mod the checker to take out 
the 1 second wait it has and run that instead of the one they have online.


Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread Georg Baum
Todd Denniston wrote:

> Did you notice the install it locally link?
> http://validator.w3.org/docs/checklink.html#install
> IIF you have all the prereqs installed you could mod the checker to take
> out the 1 second wait it has and run that instead of the one they have
> online.

But you should only do that if you restrict the check to local links.
Otherwise it is rude to the ckecked servers. Anyway, the delay should not
matter at all if you install it locally and run it from a cron job every
few days.


Georg




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Todd Denniston wrote:

> [EMAIL PROTECTED] wrote:
> > On Thu, 15 Dec 2005, Georg Baum wrote:
> > 
> > 
> >>[EMAIL PROTECTED] wrote:
> >>
> >>
> >>>On Wed, 14 Dec 2005, Angus Leeming wrote:
> >>>
> http://validator.w3.org/checklink/
> >>>
> >>>Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks
> >>>anyway, I'll keep it in mind while looking at other options.
> >>
> >>You can make exceptions in robots.txt for exactly this client. See
> >>http://validator.w3.org/docs/checklink.html
> > 
> > 
> > Thanks for the info, I've modified robots.txt now and I'm looking into it
> > further (trying to see if I can validate the HTML while I'm at it).
> > 
> > /Christian
> > 
> 
> Did you notice the install it locally link?

Yes... some dependencies aren't satisfied at the server where it should
ideally be installed. I'll ask Lars about this.

/Christian

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

> Todd Denniston wrote:
> 
> > Did you notice the install it locally link?
> > http://validator.w3.org/docs/checklink.html#install
> > IIF you have all the prereqs installed you could mod the checker to take
> > out the 1 second wait it has and run that instead of the one they have
> > online.
> 
> But you should only do that if you restrict the check to local links.
> Otherwise it is rude to the ckecked servers. Anyway, the delay should not
> matter at all if you install it locally and run it from a cron job every
> few days.

I've got it working locally now (the dependencies that were missing aren't 
critical).

I'm also only interested in checking local links, so that should be fine.

Not sure this has to be run very often though... I thought just now and 
then. There's not that much traffic on the wiki anyway [*].

Thanks for the help guys!
/Christian

[1] One idea might be to run the test only on pages that have actually 
changed. 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-15 Thread chr
On Thu, 15 Dec 2005, Georg Baum wrote:

> Todd Denniston wrote:
> 
> > Did you notice the install it locally link?
> > http://validator.w3.org/docs/checklink.html#install
> > IIF you have all the prereqs installed you could mod the checker to take
> > out the 1 second wait it has and run that instead of the one they have
> > online.
> 
> But you should only do that if you restrict the check to local links.

You wouldn't happen to know how to do that? I can't figure it out from the 
manual, and perl isn't my strong suit.

/C

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Todd Denniston wrote:

 Oh, and it would be a good idea for someone to make sure all the links
 work correctly now...

Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').

I believe 'wget' might be used, but I'm not versed well enough in it to 
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Angus Leeming
[EMAIL PROTECTED] wrote:

 On Wed, 14 Dec 2005, Todd Denniston wrote:
 
 Oh, and it would be a good idea for someone to make sure all the links
 work correctly now...
 
 Does anyone know of a tool which would be useful to traverse a web site
 and check which links doesn't work?  (In this case we are primarily
 interested in links starting with 'http://wiki.lyx.org/').

http://validator.w3.org/checklink/

-- 
Angus



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Stephen Harris


- Original Message - 
From: [EMAIL PROTECTED]

To: lyx-users@lists.lyx.org
Sent: Wednesday, December 14, 2005 11:06 AM
Subject: How to verify links on wiki.lyx? (Was: ...)



I believe 'wget' might be used, but I'm not versed well enough in it to
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

--
Christian Ridderström, +46-8-768 39 44 
http://www.md.kth.se/~chr





Hello Christian,

I borrowed this subject to ask you a vaguely related question.
Is there away for the Wiki to automatically convert a .pdf file
into the Wiki format? A pdf2wiki ?

Regards,
Stephen 





Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Angus Leeming wrote:

 [EMAIL PROTECTED] wrote:
 
  On Wed, 14 Dec 2005, Todd Denniston wrote:
  
  Oh, and it would be a good idea for someone to make sure all the links
  work correctly now...
  
  Does anyone know of a tool which would be useful to traverse a web site
  and check which links doesn't work?  (In this case we are primarily
  interested in links starting with 'http://wiki.lyx.org/').
 
 http://validator.w3.org/checklink/

Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.

/Christian
 
 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Todd Denniston wrote:

 Oh, and it would be a good idea for someone to make sure all the links
 work correctly now...

Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').

I believe 'wget' might be used, but I'm not versed well enough in it to 
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Angus Leeming
[EMAIL PROTECTED] wrote:

 On Wed, 14 Dec 2005, Todd Denniston wrote:
 
 Oh, and it would be a good idea for someone to make sure all the links
 work correctly now...
 
 Does anyone know of a tool which would be useful to traverse a web site
 and check which links doesn't work?  (In this case we are primarily
 interested in links starting with 'http://wiki.lyx.org/').

http://validator.w3.org/checklink/

-- 
Angus



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Stephen Harris


- Original Message - 
From: [EMAIL PROTECTED]

To: lyx-users@lists.lyx.org
Sent: Wednesday, December 14, 2005 11:06 AM
Subject: How to verify links on wiki.lyx? (Was: ...)



I believe 'wget' might be used, but I'm not versed well enough in it to
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

--
Christian Ridderström, +46-8-768 39 44 
http://www.md.kth.se/~chr





Hello Christian,

I borrowed this subject to ask you a vaguely related question.
Is there away for the Wiki to automatically convert a .pdf file
into the Wiki format? A pdf2wiki ?

Regards,
Stephen 





Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Angus Leeming wrote:

 [EMAIL PROTECTED] wrote:
 
  On Wed, 14 Dec 2005, Todd Denniston wrote:
  
  Oh, and it would be a good idea for someone to make sure all the links
  work correctly now...
  
  Does anyone know of a tool which would be useful to traverse a web site
  and check which links doesn't work?  (In this case we are primarily
  interested in links starting with 'http://wiki.lyx.org/').
 
 http://validator.w3.org/checklink/

Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.

/Christian
 
 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Todd Denniston wrote:

> Oh, and it would be a good idea for someone to make sure all the links
> work correctly now...

Does anyone know of a tool which would be useful to traverse a web site
and check which links doesn't work?  (In this case we are primarily
interested in links starting with 'http://wiki.lyx.org/').

I believe 'wget' might be used, but I'm not versed well enough in it to 
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr




Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Angus Leeming
[EMAIL PROTECTED] wrote:

> On Wed, 14 Dec 2005, Todd Denniston wrote:
> 
>> Oh, and it would be a good idea for someone to make sure all the links
>> work correctly now...
> 
> Does anyone know of a tool which would be useful to traverse a web site
> and check which links doesn't work?  (In this case we are primarily
> interested in links starting with 'http://wiki.lyx.org/').

http://validator.w3.org/checklink/

-- 
Angus



Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread Stephen Harris


- Original Message - 
From: <[EMAIL PROTECTED]>

To: <lyx-users@lists.lyx.org>
Sent: Wednesday, December 14, 2005 11:06 AM
Subject: How to verify links on wiki.lyx? (Was: ...)



I believe 'wget' might be used, but I'm not versed well enough in it to
say how it's actually done.

regards
/Christian

(I'll also go and ask on the wiki list about this in general)

--
Christian Ridderström, +46-8-768 39 44 
http://www.md.kth.se/~chr





Hello Christian,

I borrowed this subject to ask you a vaguely related question.
Is there away for the Wiki to automatically convert a .pdf file
into the Wiki format? A pdf2wiki ?

Regards,
Stephen 





Re: How to verify links on wiki.lyx? (Was: ...)

2005-12-14 Thread chr
On Wed, 14 Dec 2005, Angus Leeming wrote:

> [EMAIL PROTECTED] wrote:
> 
> > On Wed, 14 Dec 2005, Todd Denniston wrote:
> > 
> >> Oh, and it would be a good idea for someone to make sure all the links
> >> work correctly now...
> > 
> > Does anyone know of a tool which would be useful to traverse a web site
> > and check which links doesn't work?  (In this case we are primarily
> > interested in links starting with 'http://wiki.lyx.org/').
> 
> http://validator.w3.org/checklink/

Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks 
anyway, I'll keep it in mind while looking at other options.

/Christian
> 
> 

-- 
Christian Ridderström, +46-8-768 39 44   http://www.md.kth.se/~chr