Andrzej Bialecki wrote:
wangxu wrote:
Have anybody thought of replacing CrawlDb with any kind of Rational
DB,mysql,for example?
Crawldb is so difficult to manipulate.
I often have the requirements to edit several entries in crawdb;
But that would cost too much waiting for the mapReduce.
Please make the following test using your favorite relational DB:
* create a table with 300 mln rows and 10 columns of mixed type
* select 1 mln rows, sorted by some value
* update 1 mln rows to different values
If you find that these operations take less time than with the current
crawldb then we will have to revisit this issue. :)
That is so funny.
