Re: [basex-talk] Wasn't there a function, that would walk a website?

Vincent Lizzi wrote: > Hi Andreas, Christian, > > Here attached is a module that I wrote a while ago to limit the rate of > requests sent to a web server. This module has been useful in accessing APIs > where the SLA does not allow more than a certain number requests per minute, > and might be

Re: [basex-talk] Wasn't there a function, that would walk a website?

scenario. Although Cristian's crawler module already has a sleep built in to it. Cheers, Vincent -Original Message- From: BaseX-Talk On Behalf Of Christian Grün Sent: Wednesday, August 01, 2018 3:57 AM To: Andreas Mixich Cc: BaseX Subject: Re: [basex-talk] Wasn't there a func

[basex-talk] Wasn't there a function, that would walk a website?

Am 01.08.2018 um 09:56 schrieb Christian Grün: > Just for fun, I wrote a little crawler in XQuery (see the attached files). Very interesting, indeed! Nice to see an example of lazy:cache and prof:dump. I did not use them, so far, and that is some good news to see them in action. > it should surel

Re: [basex-talk] Wasn't there a function, that would walk a website?

Hi Andreas, Just for fun, I wrote a little crawler in XQuery (see the attached files). Please note that it’s just a stub; and it should surely be used decently, otherwise the remote server might block further access. Cheers, Christian On Wed, Aug 1, 2018 at 8:08 AM Andreas Mixich wrote: > > A

Re: [basex-talk] Wasn't there a function, that would walk a website?

Am 31.07.2018 um 08:51 schrieb Christian Grün: > I guess you were dreaming ;) But it should definitely be possible to > realize this in XQuery without too many lines of code.. Ok, then that's what I am going to do. Thanks for clarification. -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andr

Re: [basex-talk] Wasn't there a function, that would walk a website?

I guess you were dreaming ;) But it should definitely be possible to realize this in XQuery without too many lines of code.. Andreas Mixich schrieb am Di., 31. Juli 2018, 01:43: > Hi, > > somehow it deems to me, that I once saw a function in BaseX extensive > module library, that would recursi

[basex-talk] Wasn't there a function, that would walk a website?

Hi, somehow it deems to me, that I once saw a function in BaseX extensive module library, that would recursively read a web-page from remote, following all the links in it. But now I can't find it. Am I dreaming? -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich