wangxu wrote:
Have anybody thought of replacing CrawlDb with any kind of Rational
DB,mysql,for example?
Crawldb is so difficult to manipulate.
I often have the requirements to edit several entries in crawdb;
But that would cost too much waiting for the mapReduce.
Please make the following test using your favorite relational DB:
* create a table with 300 mln rows and 10 columns of mixed type
* select 1 mln rows, sorted by some value
* update 1 mln rows to different values
If you find that these operations take less time than with the current
crawldb then we will have to revisit this issue. :)
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
