jnioche commented on PR #2241: URL: https://github.com/apache/stormcrawler/pull/2241#issuecomment-6265643904
thanks @dpol1, 2 observations: **Every operation is now serialized twice.** The second time happens on the bolt’s thread, for size counting. That’s cheap for status and delete operations, but IndexerBolt sends full page text, so it may cost noticeable CPU. The legacy module estimated the size instead of serializing. **DeletionBolt doesn’t handle the new exception from add().** At DeletionBolt.java:112-114 it calls waitAck.addTuple and then addToProcessor with no try/catch. A serialization error now escapes execute(), which in Storm kills the worker, and leaves a stale entry in WaitAckCache. Serializing a delete operation is unlikely to fail, but for consistency it should do what IndexerBolt and now StatusUpdaterBolt do: invalidate the entry, then fail the tuple. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
