Re: [Rpm-maint] [rpm-software-management/rpm] RFE: run rpm scripts on a different thread using MQTT pub/sub message queues (#519)

2018-08-21 Thread Jeff Johnson
Closed #519.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/519#event-1800134078___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint


Re: [Rpm-maint] [rpm-software-management/rpm] RFE: run rpm scripts on a different thread using MQTT pub/sub message queues (#519)

2018-08-17 Thread Jeff Johnson
To forestall the obvious objections to using MQTT, please note that I also have 
working code for SysV messages, POSIX message queues, AMQP, ZeroMQ, etc.

I also am quite capable of an implementation of asynchronous RPC similar to 
above in XMLRPC, jabber, or UNIX domain sockets, or protocol du jour.

The type of transport used is not the issue.

The example provided provides an outline of a solution to a known problem in 
order to ask an important question.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/519#issuecomment-413930567___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint


[Rpm-maint] [rpm-software-management/rpm] RFE: run rpm scripts on a different thread using MQTT pub/sub message queues (#519)

2018-08-17 Thread Jeff Johnson
This RFE tries to provide a concrete example to known bottlenecks with rpm 
install/update mentioned in issue #517.

MQTT is Yet Another Message Queue (YAMQ) like ZeroMQ, or AMQP, or M$ MQ 
developed by IBM with a client implementation maintained in the Apache 
Foundation.

Message queues are typically (ZeroMQ is an exception) implemented with a broker 
to ensure reliable delivery of messages in sequential order, using 
publish/subscribe methods for producers/consumers.

With MQTT the subscribe consumer is delivered a message through a callback on a 
different thread.

One approach to running scripts on a different thread would involve creating 
both a publish/subscribe mailbox within RPM, and sending the script to be 
executed through MQTT to be run asynchronously.

The subscribe code would take the message body (I.e. the rpm script to run) and 
either invoke /bin/sh or run embedded lua on a different thread. The return 
code would then be sent back to the original publisher to be collected.

Using MQTT in this fashion is just an obvious implementation of asynchronous 
RPC. The benefit comes from the simplicity by which the scriptlet runs on a 
different thread, and rpm execution can proceed without blocking on waitpid, 
nor implementing thread pools, or using locks, or worrying much about thread 
safety of rpmlib since MQTT messages hide all the gory details.

I have working code for MQTT, the refactoring to achieve asynchronous execution 
is obvious, and I can provide an implementation, with measurements, if there is 
interest. Yadda yadda.

The real point of this RFE is to supply a concrete illustration of an 
alternative path to parallelism for issue #517 and to reask the question:

What implementations are deemed acceptable to RPM?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/519___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint