Hi Ганц, I think I understand now. I'm not sure if both processes (data updater that process events and the rest of the Mojo workers that just querys Cache::FastMmap data answering to http requests) can be written in a "non-blocking" way. (Cache::FastMmap doesn't seem to support non-blocking calls).
If some of those processes will block the ioloop, probably the best you can do is to run the master process by itself, in a different script, and the rest of the processes as a Mojolicious app through hypnotoad. For the updater process you can use Mojo:: modules and probably the Mojo::IOLoop loop as showed in many Mojolicious stand alone examples, and then you can execute it as a service with systemd for instance. But if you can write them in a non-blocking way, for instance replacing the Cache::FastMmap data sharing with the Redis equivalents, and using Mojo::Redis2 whithout blocking the ioloop, I think to run everything together as I mentioned earlier (with the Mojo::IOLoop->next_tick trick for the "master" part and just the hppt gets and posts as a regular Mojo app for the querys) should just work ok. The reason is that all works will be distribuited among identical workers, and if they don't block the ioloop it doesn't matter than a worker ioloop has updates and querys jobs mixed because there will not be penalties translated from one to the other. Even the code will probably not be duplicated (thanks to COW). So it doesn't really matters if one is a heavy lifting process or not, the important thing to run it inside a Mojolicious app is if it will block or not your ioloop. With that said, it is also true that if your current code actually blocks but never takes forever to go back to the loop (like 100 mS or so), you can consider it non-blocking and just add more workers to compensate the small blocking time to time, and nobody will notice... El martes, 18 de abril de 2017, 9:44:26 (UTC-3), Ганц Аккерман escribió: > > Daniel, as a simple explanation: > > I have: > > $ pstree -p $(</opt/WebServices/druid2/run/hypnotoad.pid) > /opt/WebService(7038)─┬─/opt/WebService(7039) > ├─/opt/WebService(7040) > ├─/opt/WebService(7041) > ├─/opt/WebService(7042) > ├─/opt/WebService(7043) > ├─/opt/WebService(7044) > ├─/opt/WebService(7045) > ├─/opt/WebService(7046) > ├─/opt/WebService(7047) > ├─/opt/WebService(7048) > ├─/opt/WebService(7049) > ├─/opt/WebService(7050) > ├─/opt/WebService(7051) > ├─/opt/WebService(7052) > ├─/opt/WebService(7053) > └─/opt/WebService(7054) > > I need to subscribe to update notification records from Redis and update > cache - only in process #7038 > Workers #7040..7054 must be used only to read cache and send replies to > users in a predictable time. > In my opinion, workers must not be involved in any heavy processing, > because its area of responsibility is user requests - it must reply as soon > as it possible minimizing resources needed to assemble responses. > > 2017-04-18 3:14 GMT+03:00 Daniel Mantovani <[email protected] <javascript:> > >: > >> Hi, >> >> I am not sure I fully understand your question, but if you already have >> Redis and you want that only one spawned process receives an event, and >> that on next event next spawned process do the same, you could use queue >> commands from Redis for that (instead of PUB/SUB that would be addressed to >> all childs that are listening to the published key). >> >> So your "publisher" should send events to a queue using for instance, >> RPUSH somelist someevent, and your mojo app should be listening with BLPOP >> somelist 0 (0 means indefinitelly). >> >> For the listening inside your Mojo app, you should take a look on >> Mojo::Redis2. >> >> Just as an example, you could do something like: >> >> use Mojolicious::Lite; >> use Mojo::Redis2; >> >> helper redis => sub { shift->stash->{redis} ||= Mojo::Redis2->new; }; >> >> Mojo::IOLoop->next_tick(sub { >> my $c = app->build_controller; >> my $element_get; >> $element_get = sub { >> $c->redis->blpop('myqueue', 0, sub { >> my ($self, $err, $res) = @_; >> $c->app->log->info("Worker $$ received an event from $res->[0]: >> $res->[1]"); >> $element_get->(); # get next element from 'myqueue' >> }); >> }; >> $c->app->log->info("Worker $$ started"); >> $element_get->(); >> }); >> >> get '/' => {text => 'Hello Word!!!'}; >> >> app->start; >> >> That way, you will not block the ioloop, and only one child will take >> next event. >> The child that gets the event should rotate just because it is the way >> Redis queues work. >> As stated in the cookbook, you can use "next_tick" to run code whenever a >> new process is spawned. >> >> You can check this code running it with hypnotoad and opening on another >> terminal a redis-cli instance and sending strings to the 'myqueue' list, >> like: >> >> > rpush myqueue whatever "whatever 2" "whatever 3" "whatever 4" "whatever >> 5" >> >> Hope this helps, >> >> >> El jueves, 13 de abril de 2017, 10:46:24 (UTC-3), Ганц Аккерман escribió: >>> >>> Hello! >>> >>> In my hypnotoad application i want to update cache (Cache::FastMmap) on >>> some event, which fired by Redis subscription. >>> But now i have to subscribe on Redis events and update local perl hash >>> in every spawned process, which at the same time serve users requests. >>> How to use only one process to support actualized cache state, so all >>> workers can read information from that cache, but not update it? >>> >>> I think, i need to subscribe on Redis events and maintain cache in the >>> main process, but Mojolicious Cookbook tell me that is impossible (IOLoop >>> not running in the main process). >>> I think Minion is not suitable for me because we use only Redis+MariaDB >>> in our production environments and i cant install PostgreSQL for Minion. >>> So i m in doubt, how to operate cache in traditional way (one auxiliary >>> updater process - many frontend processes-readers). >>> >>> Tell me please, whether is it possible in Mojolicious application >>> running on Mojolicious::Server::Prefork/hypnotoad? >>> >>> Thank you! >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Mojolicious" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/mojolicious/djY6qVG1sTo/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/mojolicious. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
