> for the record, there urls are 126M and they are all from from unique host, > no host repeteated here. I'm partitioning by host ( in first round now it's > not going to have effect, but for the next rounds ) partitioning 10 urls by > host. > ¿Can be that the problem? >
no, limiting the number of urls by host is what I was asking about, which is not the same as partioning by host. The latter is definitely not the source of the problem. -- * *Open Source Solutions for Text Engineering http://digitalpebble.blogspot.com/ http://www.digitalpebble.com
