Re: [PATCH v2 master 4/7] Implement a module performing PAM authentication

:03 PM, 'Klaus Aehlig' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > On Wed, Jan 13, 2016 at 12:42:31PM +0100, 'Oleg Ponomarev' via > ganeti-devel wrote: > > As proposed in doc/design-rapi-pam.rst, implement ValidateRequest > > function that interacts with PAM in orde

Re: [PATCH master 1/7] Add design document for PAM RAPI support

Thanks for the review! On 01/12/2016 02:44 PM, Klaus Aehlig wrote: Really a configure flag? Why not a run-time option of the daemon? Note that this very patch series actually implements a run-time option of the RAPI daemon, and not a flag at configure time. Yes, that's my fail. I forgot to

Re: [PATCH master 1/7] Add design document for PAM RAPI support

Please, consider the following interdiff: diff --git a/doc/design-rapi-pam.rst b/doc/design-rapi-pam.rst index b5bc5ed..5d2679e 100644 --- a/doc/design-rapi-pam.rst +++ b/doc/design-rapi-pam.rst @@ -5,14 +5,14 @@ RAPI authentication and authorization using PAM .. contents:: :depth: 4 This

[PATCH v2 master 4/7] Implement a module performing PAM authentication

As proposed in doc/design-rapi-pam.rst, implement ValidateRequest function that interacts with PAM in order to perform authentication and then authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + lib/errors.py | 6 +

[PATCH v2 master 6/7] Add an authenticated user name to an opcode's reason trail

The entry will have the following format: ("gnt:library:rlib2", "RAPI-Auth:user", *time*). This entry in the reason trail might be useful in order to detect malicious rapi users or restrict access for some operations by filters. Note that, in the last case, it's necessary to restrict access to

[PATCH master 4/7] Implement a module performing PAM authentication

As proposed in doc/design-rapi-pam.rst, implement ValidateRequest function that interacts with PAM in order to perform authentication and then authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + lib/rapi/auth/pam.py| 351

[PATCH master 4/7] Implement a module performing PAM authentication

As proposed in doc/design-rapi-pam.rst, implement ValidateRequest function that interacts with PAM in order to perform authentication and then authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + lib/rapi/auth/pam.py| 351

Ganeti Basic PAM module

Hi, ganeti-eng, I've just completed a PAM module that (in combination with --pam-authentication) has almost the same behavior as the current authentication implementation (preserved in lib/rapi/basic_auth.py). The module is available in https://github.com/onponomarev/ganeti-basic-pam repository.

[PATCH master 7/7] Add ``user`` predicate to Ganeti query language

The new predicate contains the only one parameter ``user`` that equals to the authorized RAPI user submitted a job. If the user is unauthorized (RAPI authentication disabled) or a job hasn't been submitted via RAPI, the parameter will contain an empty string. Signed-off-by: Oleg Ponomarev

[PATCH master 2/7] Move RAPI users config parsing to a separate file

As this functinoality will be used in a PAM module, move it into an independent module. Signed-off-by: Oleg Ponomarev --- Makefile.am| 3 +- lib/http/auth.py | 53 - lib/rapi/testutils.py |

[PATCH master 6/7] Add an authenticated user name to an opcode's reason trail

The entry will have the following format: ("gnt:library:rlib2", "RAPI-Auth:user", *time*). This entry in the reason trail might be useful in order to detect malicious rapi users or restrict access for some operations by filters. Note, that in the last case it's necessary to restrict access to

[PATCH master 1/7] Add design document for PAM RAPI support

The document propose a way of integration PAM into Ganeti RAPI in order to support flexible authentification and authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + doc/design-optables.rst | 2 + doc/design-rapi-pam.rst | 132

[PATCH master 5/7] Add a new --pam-authentication option to RAPI daemon

When the option is passed, RAPI daemon uses PamAuthenticator instead of BasicAuthenticator. Signed-off-by: Oleg Ponomarev --- lib/server/rapi.py | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/server/rapi.py b/lib/server/rapi.py index

[PATCH master 0/7] Add PAM support to Ganeti

Thee implementation is now ready. All the current tests are adapted to the new implementation. New functionality (pam authentication) has been tested with an artificial pam module. Oleg Ponomarev (7): Add design document for PAM RAPI support Move RAPI users config parsing to a separate file

[PATCH master 3/7] Incapsulate RAPI authentication into a separate module

Functions and classes related to authentication moved to rapi.auth module and are abstracted by RapiAuthenticator class in order to simplify implementation of other authentication modules. Signed-off-by: Oleg Ponomarev --- Makefile.am| 11 ++-

[PATCH master 4/7] Implement a module performing PAM authentication

As proposed in doc/design-rapi-pam.rst, implement ValidateRequest function that interacts with PAM in order to perform authentication and then authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + lib/rapi/auth/pam.py| 351

Re: [PATCH stable-2.16] Fix error message in attachInstanceDiskChecks

LGTM, thanks. On 12/14/2015 03:17 PM, 'Lisa Velden' via ganeti-devel wrote: Name the instance where disks are already attached to, which is not necessarily the instance where we want to attach a disk to. This fixes issue 1151. Signed-off-by: Lisa Velden ---

[PATCH master] Add design document for PAM RAPI support

The document propose the way of integration PAM into Ganeti RAPI in order to support flexible authentification and authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + doc/design-optables.rst | 2 + doc/design-rapi-pam.rst | 134

[PATCH master] Fix node root path in global hooks QA tests

As the root path is different on a virtual cluster, it is modified with qa_utils.MakeNodePath. Signed-off-by: Oleg Ponomarev --- qa/qa_global_hooks.py | 35 +++ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git

[PATCH master] Add design document for PAM RAPI support

The document propose a way of integration PAM into Ganeti RAPI in order to support flexible authentification and authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + doc/design-optables.rst | 2 + doc/design-rapi-pam.rst | 132

Re: [PATCH master 06/10] Implement forkPostHooksProcess function

Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: forkPostHooksProcess function will be used to create process running global POST hooks for opcodes which job processes have disappeared. S

Re: [PATCH master 02/10] Run global hooks in case of succesfull opcode execution

On 12/04/2015 07:34 PM, Hrvoje Ribicic wrote: Furthermore, why assign it a value of None instead of self._GetNodeId by default? This way you do not have to check for the None value later on, but just invoke the function. Unfortunately, I can't because self is not defined yet in the

[PATCH] Add design document for PAM RAPI support

The document propose the way of integration PAM into Ganeti RAPI in order to support flexible authentification and authorization. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + doc/design-optables.rst | 2 + doc/design-rapi-pam.rst | 133

Re: [PATCH] Always pass node uuids but not node names to hooksmaster

-9]{4}-[a-f0-9]{12}$" +dummyUuid :: String +dummyUuid = "deadbeef-dead-beed-dead-beefdeadbeef" + -- * Luxi constants luxiSocketPerms :: Int On 11/26/2015 09:12 AM, Helga Velroyen wrote: LGTM, thanks On Wed, 25 Nov 2015 at 14:10 'Oleg Ponomarev' via gane

Re: [PATCH stable-2.16] Fix line length

> > If you still need a review for this and it's exactly the same as the patch > for stable-2.14, then LGTM. > > On Fri, Nov 27, 2015 at 11:50 AM, 'Lisa Velden' via ganeti-devel < > ganeti-devel@googlegroups.com> wrote: > >> As the previous patch series was wrongly pushed to stable-2.16, >> fix

Re: [PATCH] Always pass node uuids but not node names to hooksmaster

2015 09:12 AM, Helga Velroyen wrote: LGTM, thanks On Wed, 25 Nov 2015 at 14:10 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: Currently hooksmaster is able to deal with both uuids and names. While uuids are

[PATCH master 12/12] Add unit tests for global hooks

The tests check the following: - global hooks should be run on the master node even in case of empty nodes list; - global hooks should be run on the master node separately from the other nodes; - the environmental variable IS_MASTER should be set to "master" when executing on the

Re: [PATCH stable-2.14 1/3] Improve tests for attaching disks

LGTM with the small nit. (but it's OK to just ignore it). On 11/26/2015 01:36 PM, 'Lisa Velden' via ganeti-devel wrote: by associating disks and instances to a specific node. Signed-off-by: Lisa Velden --- test/py/cmdlib/instance_unittest.py | 20 1

Re: [PATCH stable-2.14 2/3] Allow disk attachment to diskless instances

LGTM with a small nit. On 11/26/2015 01:36 PM, 'Lisa Velden' via ganeti-devel wrote: As only DRBD disks can be associated to more nodes than the instance where we want to attach the disk to, we have to check for that, too. The commit message as is introduces one more check in addition to the

Re: [PATCH stable-2.14 3/3] Add more detach/attach sequence tests

LGTM, but it's not clear why did you decide to use mock_uuid_113{5,6} and repeated it so many times. From my point of view, it's better to create a variable mock_uuid="..." and then use it. Anyway it's not that important and you can just ignore the remark. On 11/26/2015 01:36 PM, 'Lisa

Re: [PATCH stable-2.14 1/3] Improve tests for attaching disks

LGTM, thanks On 11/26/2015 05:31 PM, Lisa Velden wrote: interdiff that also adds the varriables for the third patch: diff --git a/test/py/cmdlib/instance_unittest.py b/test/py/cmdlib/instance_unittest.py index e91f4c4..7da8293 100644 --- a/test/py/cmdlib/instance_unittest.py +++

Re: [PATCH stable-2.14 2/3] Allow disk attachment to diskless instances

LGTM, thanks On 11/26/2015 05:35 PM, Lisa Velden wrote: Suggestion for a new commit message: Allow disk attachment to diskless instances As only DRBD disks can be associated to more nodes than the instance where we want to attach the disk to, we have to change the check for associated nodes,

Re: [PATCH stable-2.14 3/3] Add more detach/attach sequence tests

LGTM, thanks On 11/26/2015 05:39 PM, Lisa Velden wrote: interdiff: diff --git a/test/py/cmdlib/instance_unittest.py b/test/py/cmdlib/instance_unittest.py index cc668e8..04dffbb 100644 --- a/test/py/cmdlib/instance_unittest.py +++ b/test/py/cmdlib/instance_unittest.py @@ -2912,14 +2912,14 @@

Re: [PATCH master 10/10] Add QA tests for global hooks

On 11/25/2015 04:20 PM, Hrvoje Ribicic wrote: General remarks: Consider adding another node to the mix - the QA is missing checks for global hooks running on both master and other nodes, and running on master despite not being executed on other nodes. You can allow the LUTestDelay to run

[PATCH master 11/11] Add QA tests for global hooks

Test 3 main cases of global post hooks usage: - successful LU execution; - LU with the prerequisites failed; - disappeared LU process. All the tests are performed on the master node. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + qa/ganeti-qa.py

Re: [PATCH stable-2.15] Compute lock allocation strictly

LGTM with nit. Also I'm almost sure about how all these optimizations affect the performance (maybe the compiler will perform some of them automatically, especially in foldl/foldl' case) but anyway it will not break the functionality. On 11/26/2015 06:41 PM, 'Klaus Aehlig' via ganeti-devel

Re: [PATCH master 08/10] Execute global POST hooks for disappeared job processes

*Commit message fixed* On 11/24/2015 08:36 PM, Hrvoje Ribicic wrote: LGTM with a single nit. On Fri, Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: Start execution of global POST h

Re: [PATCH master 07/10] Implement python process ececuting global POST hooks

ot;Removing livelock file %s", livelock_name.GetPath()) On 11/24/2015 08:35 PM, Hrvoje Ribicic wrote: LGTM with nits. Fix patch title - executing. On Fri, Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.

Re: [PATCH master 03/10] Execute POST-phase global hooks in case of failure

ibicic wrote: LGTM with nitpick On Fri, Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: All the errors encountered during an opcode execution are reported via exceptions. I

Re: [PATCH master 05/10] Divide forkJobProcess into several functions

"Writing job id to the client" jidStr _ <- recv "Waiting for the job to ask for the lock file name" On 11/24/2015 08:24 PM, Hrvoje Ribicic wrote: On Fri, Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-deve

Re: [PATCH master 09/10] Update design document for global hooks

Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: Make the design document consistent with the current global post hooks implementation. Signed-off-by: Oleg Ponomarev <oponoma...@google.com <mailto:opo

[PATCH] Always pass node uuids but not node names to hooksmaster

Currently hooksmaster is able to deal with both uuids and names. While uuids are passed in most cases, the node list uses the legacy naming *node_names* inside the hooksmaster implementation which leads to misunderstandings. This patch modifies the existing code to make it always pass node uuids

Re: [PATCH master 01/11] Always pass node uuid's but not node name's to hooksmaster

Remarks fixed. Finally, I decide to add assert and therefore fixed python tests. Here is the interdiff: diff --git a/lib/hooksmaster.py b/lib/hooksmaster.py index 159aa14..4f1d7bd 100644 --- a/lib/hooksmaster.py +++ b/lib/hooksmaster.py @@ -188,7 +188,7 @@ class HooksMaster(object):

Re: [PATCH master 01/11] Always pass node uuid's but not node name's to hooksmaster

On 11/24/2015 05:06 PM, Hrvoje Ribicic wrote: One possible improvement: we can consider asserting that all node_uuids passed are truly uuids whenever RunPhase is invoked. Or do you think this is too paranoid? I am not sure that it worth it because it will break several tests which use node

Re: [PATCH master 01/10] Add global hooks support to the HooksMaster

On 11/23/2015 06:21 PM, Hrvoje Ribicic wrote: On Fri, Nov 20, 2015 at 5:11 PM, 'Oleg Ponomarev' via ganeti-devel <ganeti-devel@googlegroups.com <mailto:ganeti-devel@googlegroups.com>> wrote: Add the *glob* argument to the RunPhase function. With *glob* set to True,

Re: [PATCH master 01/10] Add global hooks support to the HooksMaster

OK, and also in the hooksmaster itself even before my changes. On 11/23/2015 06:21 PM, Hrvoje Ribicic wrote: How come? Where do the different value types come from? I would rather fix this at the source rather than here.

Re: [PATCH master 01/10] Add global hooks support to the HooksMaster

quot;GANETI_IS_MASTER"] = constants.GLOBAL_HOOKS_NOT_MASTER +ret.update(self._CheckParamsAndExecHooks(node_list, hpath, phase, env)) return ret def RunPhase(self, phase, node_uuids=None, is_global=False, On 11/23/2015 06:21 PM, Hrvoje Ribicic wrote: On Fri, Nov 20, 2015 at 5:

Re: [PATCH master 01/10] Add global hooks support to the HooksMaster

+env["GANETI_IS_MASTER"] = constants.GLOBAL_HOOKS_NOT_MASTER +ret.update(self._CheckParamsAndExecHooks(node_list, hpath, phase, env)) return ret def RunPhase(self, phase, node_uuids=None, is_global=False, On 11/23/2015 06:21 PM, Hrvoje Ribicic wrote: On Fri, Nov 20, 2015 at

[PATCH master 01/11] Always pass node uuid's but not node name's to hooksmaster

Currently hooksmaster is able to deal with both uuids and names. Besides, in most cases, uuids are passed. In spite of it, the node list has legacy naming *node_names* inside the hooksmaster implementation which leads to misunderstandings. This patch modifies the existing code to make it always

Re: Merge 'stable-2.15' into 'stable-2.16'

LGTM, but I still have two remarks: On 11/20/2015 01:42 PM, 'Helga Velroyen' via ganeti-devel wrote: backend.py --logging.debug("Add ssh key of node '%s'.", node_name) Why did you decide to remove the logging.debug call? Disk.hs ++ addInst i = csConfigDataL . configInstancesL ++

[PATCH master 01/10] Add global hooks support to the HooksMaster

Add the *glob* argument to the RunPhase function. With *glob* set to True, HooksMaster runs global hooks instead of the opcode's hooks. The global hooks should be placed in the "global" subdirectory of the hooks directory. Global hooks run separately for the master node and for the others because

[PATCH master 10/10] Add QA tests for global hooks

Test 3 main cases of global post hooks usage: - successful LU execution; - LU with the prerequisites failed; - disappeared LU process. All the tests are performed on the master node. Signed-off-by: Oleg Ponomarev --- Makefile.am | 1 + qa/ganeti-qa.py

[PATCH master 06/10] Implement forkPostHooksProcess function

forkPostHooksProcess function will be used to create process running global POST hooks for opcodes which job processes have disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/Path.hs | 7 +++ src/Ganeti/Query/Exec.hs | 51

[PATCH master 00/10] Implement global hooks (and the QA tests)

Due to too long discussion https://groups.google.com/forum/#!topic/ganeti-devel/9_QgaRD_zeU which has become unmaintainable, new thread started. Compared to the last patch series: - fixed all the riba's remarks - added the new IS_MASTER enviromental variable - python tests are modified in

[PATCH master 03/10] Execute POST-phase global hooks in case of failure

All the errors encountered during an opcode execution are reported via exceptions. Intercept all the exceptions and execute POST global hooks with ERROR status. Signed-off-by: Oleg Ponomarev --- lib/hooksmaster.py | 18 ++-- lib/mcpu.py| 83

[PATCH master 02/10] Run global hooks in case of succesfull opcode execution

Run PRE and POST global hooks in case of succesful job execution on the master node and on the node_list provided by the logical unit. POST hooks are always executed with status equal to SUCCESS. Also fix the python tests: - as currently at least global hooks are always executed and

[PATCH master 04/10] Make SafeLoadJobFromDisk a @staticmethod

SafeLoadJobFromDisk will be used in further patches in order to load job files for jobs whose processes have disappeared. The functionality will be used to run global post hooks for such jobs. Signed-off-by: Oleg Ponomarev --- lib/jqueue/__init__.py | 23

[PATCH master 05/10] Divide forkJobProcess into several functions

forkJobProcess implementation consist of several steps. Move each logical consistent step into the generalized function in order to reuse these code fragments in forkPostHooksProcess (will be the next commit). Signed-off-by: Oleg Ponomarev --- src/Ganeti/Query/Exec.hs |

[PATCH master 09/10] Update design document for global hooks

Make the design document consistent with the current global post hooks implementation. Signed-off-by: Oleg Ponomarev --- doc/design-global-hooks.rst | 60 + 1 file changed, 44 insertions(+), 16 deletions(-) diff --git

[PATCH master 07/10] Implement python process ececuting global POST hooks

Implement ppost_hooks_exec.py which has already been reffered to by forkPostHooksProcess. This python process will be used in order to run global POST hooks for opcodes whose processes have disappeared. Signed-off-by: Oleg Ponomarev --- Makefile.am | 3

[PATCH stable-2.12] Calculate correct affected nodes set in LUInstanceChangeGroup

This is the fix for the issue 1144. The nodes affected by the InstanceChangeGroup logical unit were calculated incorrectly and that broke 'gnt-instance change-group --to' operation. This patch fixes it. Signed-off-by: Oleg Ponomarev --- lib/cmdlib/instance.py | 2 +- 1

Re: [PATCH stable-2.15] Don't keep input for error messages

LGTM, thanks On 11/19/2015 01:04 PM, 'Klaus Aehlig' via ganeti-devel wrote: When generating error messages, the raw JSValue is rarely useful. However, keeping it for error messages---even if only in the unused branch of an if statement---prevents this value from going out of scope. Note: with

Re: [PATCH stable-2.15] Don't keep input for error messages

LGTM, thanks On 11/18/2015 04:21 PM, 'Klaus Aehlig' via ganeti-devel wrote: When generating error messages, the raw JSValue is rarely useful. However, keeping it for error messages---even if only in the unused branch of an if statement---prevents this value from going out of scope.

Re: [PATCH stable-2.15] Use dict.copy instead of deepcopy

LGTM, thanks On 11/18/2015 02:48 PM, 'Helga Velroyen' via ganeti-devel wrote: Due to a bug in python, deepcopy does not work on the dictionaries we use for SSH updates. This patch replaces the use of deepcopy by the built-in copy function of dictionaries. Signed-off-by: Helga Velroyen

Re: [PATCH stable-2.15] Pause watcher during performance QA

LGTM, thanks. On 11/17/2015 03:42 PM, 'Klaus Aehlig' via ganeti-devel wrote: Our performance QA tests are intended to alert us if some common task suddenly takes longer. To serve this purpose, they need to provide reproducible results. Hence avoid any interference with watcher-submitted jobs by

Merge branch 'stable-2.12' into stable-2.13

commit 0a204f1e2430956935a9bcd1eb9ab1dfdc02af8b Merge: d0e19c6 9073c5f Author: Oleg Ponomarev Date: Wed Nov 11 18:14:51 2015 +0100 Merge branch 'stable-2.12' into stable-2.13 * stable-2.12 Revert "Also consider connection time out a network error"

Merge branch 'stable-2.13' into stable-2.14

commit 1b069691608602a77adfb612f30f7f1992180b36 Merge: 95e1bd6 85e1c58 Author: Oleg Ponomarev Date: Wed Nov 11 19:01:36 2015 +0100 Merge branch 'stable-2.13' into stable-2.14 * stable-2.13 Extend timeout for gnt-cluster renew-crypto * stable-2.12

Merge branch 'stable-2.11' into stable-2.12

commit 56742a761ba290b35ba8c81305195659098f7a17 Merge: 0c401de 9614023 Author: Oleg Ponomarev Date: Wed Nov 11 17:04:40 2015 +0100 Merge branch 'stable-2.11' into stable-2.12 * stable-2.11 (no changes) * stable-2.10 Remove -X

Re: Merge branch 'stable-2.10' into stable-2.11

LGTM On 11/11/2015 04:53 PM, 'Klaus Aehlig' via ganeti-devel wrote: commit a3d8f6353f8658bd5fb6a799716d7f1fd44906c3 Merge: b8be157 5c528a0 Author: Klaus Aehlig Date: Wed Nov 11 16:51:42 2015 +0100 Merge branch 'stable-2.10' into stable-2.11 * stable-2.10

[PATCH master 03/11] Execute POST-phase global hooks in case of failure

All the errors during an opcode execution are reported via exceptions. Except all the exceptions and execute POST global hooks with FAILED status. Signed-off-by: Oleg Ponomarev --- lib/hooksmaster.py | 4 +-- lib/mcpu.py| 73

[PATCH master 11/11] Update design document for global hooks

Make the design document consistent with the current global post hooks implementation. Signed-off-by: Oleg Ponomarev --- doc/design-global-hooks.rst | 44 1 file changed, 32 insertions(+), 12 deletions(-) diff --git

[PATCH master 06/11] Divide forkJobProcess into several functions

forkJobProcess implementation consist of several steps. Move each logical consistent step into the generalized function in order to reuse these code fragments in forkPostHooksProcess (will be the next commit). Signed-off-by: Oleg Ponomarev --- src/Ganeti/Query/Exec.hs |

[PATCH master 00/11] Implement global job hooks

Fixed several bugs uncovered during writing the draft of the qa tests. Also, now universal hooks are always executed at least on the master node even if the master node is missing in the node list generated by the opcode for the usual hooks. Oleg Ponomarev (11): Add global hooks support to the

[PATCH master 07/11] Implement forkPostHooksProcess function

forkPostHooksProcess function will be used to create process running global POST hooks for opcodes which job processes have disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/Path.hs | 7 +++ src/Ganeti/Query/Exec.hs | 51

[PATCH master 10/11] Execute global POST hooks for disappeared job processes

Start execution of global POST hooks as soon as scheduler finds out that job process has disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/JQScheduler.hs | 21 +++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git

[PATCH master 08/11] Implement python process ececuting global POST hooks

Implement ppost_hooks_exec.py which has already been reffered from forkPostHooksProcess. This python process will be used in order to run global POST hooks for opcodes which processes have disappeared. Signed-off-by: Oleg Ponomarev --- Makefile.am | 3

[PATCH master 04/11] Always execute global hooks at least on the master node

NoHooksLU opcodes don't provide node lists on which the hooks should be executed. Some opcodes porivde node lists which don't contain the master node. Thus, always add the master node to the node list for global hooks. Signed-off-by: Oleg Ponomarev --- lib/hooksmaster.py

[PATCH master 09/11] Implement python process ececuting global POST hooks

Implement ppost_hooks_exec.py which has already been reffered from forkPostHooksProcess. This python process will be used in order to run global POST hooks for opcodes which processes have disappeared. Signed-off-by: Oleg Ponomarev --- lib/jqueue/post_hooks_exec.py | 4

[PATCH master 05/11] Make SafeLoadJobFromDisk a @staticmethod

SafeLoadJobFromDisk will be used in further patches in order to load job files for jobs which processes are disappeared. That will be used to run global post hooks for such jobs. Signed-off-by: Oleg Ponomarev --- lib/jqueue/__init__.py | 23 ++-

[PATCH master 01/11] Add global hooks support to the HooksMaster

Add the *glob* argument to the RunPhase function. With *glob* set to True, HooksMaster runs global hooks instead of per-opcode's hooks. The global hooks should be placed in the "global" subdirectory of the hooks directory. Signed-off-by: Oleg Ponomarev ---

[PATCH master 02/11] Run global hooks in case of succesfull opcode execution

Run PRE and POST global hooks in case of succesfull job execution on the master node and on the node_list provided by the logical unit. POST hooks always always executed with status SUCCEEDED. On errors POST hooks will not be executed because all the errors cause related exeptions. Also fix test

[PATCH stable-2.15] Assign the variables before use

One more unassigned variable. Oleg Ponomarev (1): Assign the variables before use (bugfix for dee6adb9) lib/cmdlib/node.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.6.0.rc2.230.g3dd15c0

[PATCH stable-2.15] Assign the variables before use (bugfix for dee6adb9)

Signed-off-by: Oleg Ponomarev --- lib/cmdlib/node.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/cmdlib/node.py b/lib/cmdlib/node.py index c67c65d..c0eccce 100644 --- a/lib/cmdlib/node.py +++ b/lib/cmdlib/node.py @@ -1565,14 +1565,14 @@

[PATCH stable-2.15] Assign the variable before use (bugfix for dee6adb9)

Signed-off-by: Oleg Ponomarev --- lib/cmdlib/node.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/cmdlib/node.py b/lib/cmdlib/node.py index c67c65d..f9fbce3 100644 --- a/lib/cmdlib/node.py +++ b/lib/cmdlib/node.py @@ -1565,6 +1565,7 @@ class

[PATCH master 06/10] Divide forkJobProcess into several functions

forkJobProcess implementation consist of several steps. Move each logical consistent step into the generalized function in order to reuse these code fragments in forkPostHooksProcess (will be the next commit). Signed-off-by: Oleg Ponomarev --- src/Ganeti/Query/Exec.hs |

[PATCH master 10/10] Update design document for global hooks

Make the design document consistent with the current global post hooks implementation. Signed-off-by: Oleg Ponomarev --- doc/design-global-hooks.rst | 43 +++ 1 file changed, 31 insertions(+), 12 deletions(-) diff --git

[PATCH master 04/10] Execute global hooks even for NoHooksLU opcodes

NoHooksLU opcodes don't provide node lists on which the hooks should be executed. Execute global hooks just on master node for such opcodes. Signed-off-by: Oleg Ponomarev --- lib/mcpu.py | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git

[PATCH master 02/10] Run global hooks in case of succesfull opcode execution

Run PRE and POST global hooks in case of succesfull job execution. POST hooks always always executed with status SUCCEEDED. On errors POST hooks will not be executed because all the errors cause related exeptions. Also fix test which expects that hooks run only once. Now rpc called twice: for

[PATCH master 01/10] Add global hooks support to the HooksMaster

Add the *glob* argument to the RunPhase function. With *glob* set to True, HooksMaster runs global hooks instead of per-opcode's hooks. The global hooks should be placed in the "global" subdirectory of the hooks directory. Signed-off-by: Oleg Ponomarev ---

[PATCH master 03/10] Execute POST-phase global hooks in case of failure

All the errors during an opcode execution are reported via exceptions. Except all the exceptions and execute POST global hooks with FAILED status. Signed-off-by: Oleg Ponomarev --- lib/hooksmaster.py | 4 +-- lib/mcpu.py| 73

[PATCH master 05/10] Make SafeLoadJobFromDisk a @staticmethod

SafeLoadJobFromDisk will be used in further patches in order to load job files for jobs which processes are disappeared. That will be used to run global post hooks for such jobs. Signed-off-by: Oleg Ponomarev --- lib/jqueue/__init__.py | 23 ++-

[PATCH master 09/10] Execute global POST hooks for disappeared job processes

Start execution of global POST hooks as soon as scheduler finds out that job process has disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/JQScheduler.hs | 21 +++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git

[PATCH master 08/10] Implement python process ececuting global POST hooks

Implement ppost_hooks_exec.py which has already been reffered from forkPostHooksProcess. This python process will be used in order to run global POST hooks for opcodes which processes have disappeared. Signed-off-by: Oleg Ponomarev --- Makefile.am | 3

[PATCH master 00/10] Implement global hooks

Compared to the previous patch set, there are several improvals: - fixed the global post hooks results processing function (and qa tests work thanks to the fix) - python tests adaptated to the existance of global hooks (and py-tests work) - update the design document gor global

[PATCH master 07/10] Implement forkPostHooksProcess function

forkPostHooksProcess function will be used to create process running global POST hooks for opcodes which job processes have disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/Path.hs | 7 +++ src/Ganeti/Query/Exec.hs | 51

Re: [PATCH stable-2.13] Extend timeout for gnt-cluster renew-crypto

LGTM On 11/04/2015 02:06 PM, 'Hrvoje Ribicic' via ganeti-devel wrote: With particularly large clusters, the SSH renewal of key happening in renew-crypto can take a long time to complete. While this should be improved, an additional problem is that the RPC doing most of the work has a default

[PATCH master 3/9] Execute POST-phase global hooks in case of failure

All the errors during an opcode execution are reported via exceptions. Except all the exceptions and execute POST global hooks with FAILED status. Signed-off-by: Oleg Ponomarev --- lib/mcpu.py | 72 +++-- 1 file

[PATCH master 9/9] Execute global POST hooks for disappeared job processes

Start execution of global POST hooks as soon as scheduler finds out that job process has disappeared. Signed-off-by: Oleg Ponomarev --- src/Ganeti/JQScheduler.hs | 21 +++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git

[PATCH master 2/9] Run global hooks in case of succesfull opcode execution

Run PRE and POST global hooks in case of succesfull job execution. POST hooks always always executed with status SUCCEEDED. On errors POST hooks will not be executed because all the errors cause related exeptions. Also fix test which expects that hooks run only once. Now rpc called twice: for

[PATCH master 1/9] Add global hooks support to the HooksMaster

Add the *glob* argument to the RunPhase function. With *glob* set to True, HooksMaster runs global hooks instead of per-opcode's hooks. The global hooks should be placed in the "global" subdirectory of the hooks directory. Signed-off-by: Oleg Ponomarev ---

  1   2   3   >