jnioche opened a new pull request, #2291: URL: https://github.com/apache/stormcrawler/pull/2291
Fixes #2289, partial revert of #1854 Default now set to 1M The size should be set to the worker’s heap at about 200 bytes per entry, assuming an average URL size of 59-characters. 1M URLs in the cache would take ~200MB per instance of StatusUpdaterBolt. Cheaper entries: the cache could store a 64-bit hash of the URL instead of the String, which takes about a half the memory per entry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
