Saggi Mizrahi has uploaded a new change for review. Change subject: [WIP] Implement a process to do dangerous IO in C ......................................................................
[WIP] Implement a process to do dangerous IO in C This replaces the process pool with a process that can serve multiple requests written in C. This implementation is much more scalable and lightweight. Should solve bugs related to running out of helpers, logging getting suck, python forking deadlocking, running out of memory and other things as well. The communication between VDSM and the IOProcess is done with json objects. The IOProcess starts with 3 thread: 1. requestReader - reads requests from the pipe, builds a DOM representation of it and queues it up for handling 2. responseWriter - gets response DOMs from the queue converts them to a JSON string and send it over the pipe 3. requestHandler - pops requests from the queue and provisions threads for handling them. Currently we I just allocate a new thread per request. If there is ever a need to have a thread pool this is where the load balancing is going to sit. Each request gets the are as a JsonNode and returns a response that is a JsonNode as well. Most exported functions are pretty trivial and are a good example on how to write new ones. Unlink the ProcessPoolHelper, high level commands sit of the OopWrapper and are run from the client side instead of being implemented in C on the IOProcess side. Change-Id: Ie4664d5330debbe38ba33b74ebb586ac42913b4a Signed-off-by: Saggi Mizrahi <[email protected]> --- M configure.ac M tests/Makefile.am A tests/ioprocessTests.py A tests/outOfProcessTests.py D tests/processPoolTests.py M vdsm.spec.in M vdsm/constants.py.in M vdsm/storage/Makefile.am M vdsm/storage/fileSD.py M vdsm/storage/fileUtils.py M vdsm/storage/fileVolume.py A vdsm/storage/ioprocess.py A vdsm/storage/ioprocess/.gitignore A vdsm/storage/ioprocess/Makefile.am A vdsm/storage/ioprocess/exported-functions.c A vdsm/storage/ioprocess/exported-functions.h A vdsm/storage/ioprocess/ioprocess.c A vdsm/storage/ioprocess/json-dom-generator.c A vdsm/storage/ioprocess/json-dom-generator.h A vdsm/storage/ioprocess/json-dom-parser.c A vdsm/storage/ioprocess/json-dom-parser.h A vdsm/storage/ioprocess/json-dom.c A vdsm/storage/ioprocess/json-dom.h M vdsm/storage/misc.py M vdsm/storage/nfsSD.py M vdsm/storage/outOfProcess.py D vdsm/storage/processPool.py M vdsm/storage/sd.py M vdsm/storage/sp.py M vdsm/storage/task.py 30 files changed, 3,018 insertions(+), 666 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/3946/1 -- To view, visit http://gerrit.ovirt.org/3946 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie4664d5330debbe38ba33b74ebb586ac42913b4a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <[email protected]> _______________________________________________ vdsm-patches mailing list [email protected] https://fedorahosted.org/mailman/listinfo/vdsm-patches
