I am working on crawling data. I can see it is taking a lot of time because xquery is pulling web pages one by one. But popular crawlers like apache Nutch has multithreading support so it can open multiple socket connections to multiple sites so that it can crawl very fast. Can we make add multithreading support basex? So it can run multiple xquery functions in parallel and then merge results? Initially, merging results can be done manually. At least, if we can distribute loads to different xquery functions, that would be great.
Erol Akarsu
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
