Hi
I woud like to use Kombu to consume message on several queues on RabbitMQ
In Kombu tutorial, they use a loop to drain event (see below
I'm a beginner in twisted/scrapy and my question is how/where should I put
this loop "while" ?
I made a try with the code below but I'm not sure this is the right twisted
way to do it
reactor.callInThread(self.loop) <--- called when engine is started
def loop(self):
while True:
try:
self.connection.drain_events()
except Exception as e:
logger.error("Error occured while draining events", exc_info=True)
pass
def process_media(self, body, message):
<create req>
self.crawler.engine.crawl(req, spider=self)
Kombu sample
Url :
http://docs.celeryproject.org/projects/kombu/en/latest/introduction.html
# Consume from several queues on the same channel:video_queue = Queue('video',
exchange=media_exchange, key='video')image_queue = Queue('image',
exchange=media_exchange, key='image')
with connection.Consumer([video_queue, image_queue],
callbacks=[process_media]) as consumer:
while True:
connection.drain_events()
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
