jnioche commented on PR #1973: URL: https://github.com/apache/stormcrawler/pull/1973#issuecomment-4877752358
> thanks @dpol1, I like this very much. > > Instead of having a new bolt, just for handling block events, what about a generic QueueBolt, just like the [one in the OpenSearch module ](external/opensearch/src/main/java/org/apache/stormcrawler/opensearch/persistence/QueueBolt.java) but with a pluggable mechanism to handle the blocking logic within it? > > URLFrontier handles the queues itself and does not need to be told about them like OpenSearch, so we could do without the QueueBolt but we probably don't want the duplication of HostBlocks implementations for every single type of backend. Instead we would have a single implementation in core and let each QueueBolt convert it into the appropriate action. > > Make things a bit more complex now, but easier to handle later. Maybe I am making it more complicated than necessary. We can have some duplication initially, learn from it and make things more generic afterwards. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
