[PATCH stable-2.17] Prevent watcher from holding lock on verify group

The ganeti-watcher holds the group file lock for too long, until after the execution of a group-verify-disk job. This locks for a long time if there are other jobs already running and blocking the verify from executing. When the lock is held, another ganeti-watcher run cannot be scheduled, so

Re: [PATCH stable-2.16] Special case WaitForJobChange to reduce heap use

ganeti-devel < ganeti-devel@googlegroups.com> wrote: > If you submit a workload to Ganeti that has a sequence of jobs that > modify the cluster state, then performs some action that takes a long > time to complete, each handler thread will retain the version of > ConfigData tha

[PATCH stable-2.16] Special case WaitForJobChange to reduce heap use

If you submit a workload to Ganeti that has a sequence of jobs that modify the cluster state, then performs some action that takes a long time to complete, each handler thread will retain the version of ConfigData that was current when the RPC call started. Potentially, each handler can retain

Re: [PATCH stable-2.15 1/2] KVM: handle gracefully too old psutil versions

On Wed, Jun 15, 2016 at 07:18:38AM +0200, Iustin Pop wrote: > From: Iustin Pop > > My previous pylint cleanups were done without psutil installed; as soon > I installed it, pylint showed that the wheezy's version of psutil is too > old (0.5.1), not having the cpu_count()

Re: [PATCH master 2/2] Fix running unittests without python-psutil

On Tue, Jun 14, 2016 at 03:07:42PM +0200, Iustin Pop wrote: > On 2016-06-14 13:36:38, Brian Foley wrote: > > On Tue, Jun 14, 2016 at 06:10:28AM +0200, Iustin Pop wrote: > > > From: Iustin Pop > > > > > > Python 2.7 has a very nice extension to the unittest module to support >

Re: [PATCH master 2/2] Fix running unittests without python-psutil

On Tue, Jun 14, 2016 at 06:10:28AM +0200, Iustin Pop wrote: > From: Iustin Pop > > Python 2.7 has a very nice extension to the unittest module to support > test skipping, but until we officially stop supporting it we pretend > these pass. Cool. I didn't know about that.

Re: [MERGE] stable-2.15 to stable-2.16

LGTM, although there is nothing to review :) On Tue, Jun 14, 2016 at 12:48 PM, 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > commit 2dc71a7814258445b2a8de49747aebc282642343 > Merge: 90281b4 d27c6da > Author: Brian Foley <bpfo...@google.com> >

[MERGE] stable-2.15 to stable-2.16

commit 2dc71a7814258445b2a8de49747aebc282642343 Merge: 90281b4 d27c6da Author: Brian Foley Date: Tue Jun 14 12:28:28 2016 +0100 Merge branch 'stable-2.15' into stable-2.16 Manually resolve conflicts. Signed-off-by: Brian Foley

Re: [PATCH stable-2.16 0/3] ...Summary of patch series...

On Tue, Jun 14, 2016 at 12:46:01PM +0100, Brian Foley wrote: > > ...body of cover letter... > > Brian Foley (3): > debugging allocation > disable ghc stats > Second attempt at WaitForJobChange patch Please ignore these. They were sent in error. Thanks, Brian.

[PATCH stable-2.16 0/3] ...Summary of patch series...

...body of cover letter... Brian Foley (3): debugging allocation disable ghc stats Second attempt at WaitForJobChange patch src/Ganeti/Query/Export.hs |2 +- src/Ganeti/Query/Filter.hs | 27 ++--- src/Ganeti/Query/Instance.hs |5 +- src/Ganeti/Query/Node.hs |5

[PATCH stable-2.16 2/3] disable ghc stats

Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/UDSServer.hs |8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/Ganeti/UDSServer.hs b/src/Ganeti/UDSServer.hs index 254d0e4..8abe8a3 100644 --- a/src/Ganeti/UDSServer.hs +++ b/src/Ganeti/UDSSer

[PATCH stable-2.16 1/3] debugging allocation

(cherry picked from commit 33745104fd9286477cbba13cd9565a53eb64c56d) Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/Query/Server.hs |5 +++-- src/Ganeti/UDSServer.hs| 12 ++-- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/src/Ganeti

Re: [PATCH stable-2.15 0/2] Make stable-2.15 distcheck-compliant

On Tue, Jun 14, 2016 at 06:04:00AM +0200, Iustin Pop wrote: > From: Iustin Pop > > This two patches (well, only the first, the second is cleanup) make > stable-2.15 > pass "make distcheck" on wheezy again. > LGTM, thanks, Brian.

Re: Issue 1175 in ganeti: ext3 filesystem hardcoded in fstab of import script

Comment #6 on issue 1175 by hostingn...@gmail.com: ext3 filesystem hardcoded in fstab of import script https://code.google.com/p/ganeti/issues/detail?id=1175 @ius... thanks for the tip but backports packages are not considered as stable, hence "backports", and as such I d

Re: [MERGE] stable-2.17 to master

2016-06-10 16:25 GMT+02:00 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com>: > commit 7eb49311e18865db76c4e8da5eb4b2e166db2d55 > Merge: a32eb3c 17a1c27 > Author: Brian Foley <bpfo...@google.com> > Date: Fri Jun 10 15:20:33 2016 +0100 > >

[MERGE] stable-2.17 to master

> diff --cc src/Ganeti/Codec.hs index a703ba8,6f54c0d..b3b4c87 --- a/src/Ganeti/Codec.hs +++ b/src/Ganeti/Codec.hs @@@ -42,12 -40,12 +42,12 @@@ module Ganeti.Code import Prelude () import Ganeti.Prelude - import Codec.Compression.Zlib (compress) + import Codec.Compression.Zlib i

Re: [MERGE] stable-2.16 to stable-2.17

2016-06-10 15:40 GMT+02:00 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com>: > commit b462d8c77bff0789e8a951288dea34226ab8b6d7 > Merge: 20c24a8 90281b4 > Author: Brian Foley <bpfo...@google.com> > Date: Fri Jun 10 14:35:13 2016 +0100 > > Merge br

Re: Issue 1175 in ganeti: ext3 filesystem hardcoded in fstab of import script

Comment #5 on issue 1175 by ius...@google.com: ext3 filesystem hardcoded in fstab of import script https://code.google.com/p/ganeti/issues/detail?id=1175 @hosting: FYI, there's a newer package in jeesie-backports; I'd recommend using that. -- You received this message because

[MERGE] stable-2.16 to stable-2.17

ian Foley <bpfo...@google.com> diff --cc src/Ganeti/Codec.hs index 9a41499,404c70b..6f54c0d --- a/src/Ganeti/Codec.hs +++ b/src/Ganeti/Codec.hs @@@ -37,21 -37,18 +37,23 @@@ module Ganeti.Code , decompressZlib ) where +import Prelude () +import Ganeti.Prelude + - import Codec.C

Re: [MERGE] stable-2.15 to stable 2.16

2016-06-10 15:28 GMT+02:00 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com>: > commit 5785f214a9e728465a4bfc1aef7ded306225cfa2 > Merge: 40cd52f 2429235 > Author: Brian Foley <bpfo...@google.com> > Date: Fri Jun 10 14:23:10 2016 +0100 > > Merge br

[MERGE] stable-2.15 to stable 2.16

commit 5785f214a9e728465a4bfc1aef7ded306225cfa2 Merge: 40cd52f 2429235 Author: Brian Foley Date: Fri Jun 10 14:23:10 2016 +0100 Merge branch 'stable-2.15' into stable-2.16 * stable-2.15 Fixup compatibility with GHC 7.4/base 4.5 Signed-off-by:

Re: [PATCH stable-2.15] Fixup compatibility with GHC 7.4/base 4.5

On Fri, Jun 10, 2016 at 01:51:56PM +0200, 'Iustin Pop' via ganeti-devel wrote: > It looks like commit c429dd26 introduced the use of > atomicModifyIORef', which is only present in base 4.6 (GHC 7.6). > Let's fix that by importing the actual implementation of said function > from base

[PATCH stable-2.15] Fixup compatibility with GHC 7.4/base 4.5

com> --- src/Ganeti/Compat.hs | 14 ++ src/Ganeti/JQScheduler.hs | 3 ++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/src/Ganeti/Compat.hs b/src/Ganeti/Compat.hs index e5276d1..8e002e7 100644 --- a/src/Ganeti/Compat.hs +++ b/src/Ganeti/Compat.hs @@ -41,10

Re: [PATCH master] Fixup compatibility with GHC 7.4/base 4.5

On Fri, Jun 10, 2016 at 12:07:45PM +0100, Brian Foley wrote: > On Fri, Jun 10, 2016 at 12:33:23PM +0200, 'Iustin Pop' via ganeti-devel wrote: > > On Fri, Jun 10, 2016 at 12:30:03PM +0200, Iustin Pop wrote: > > > It looks like commit c429dd26 introduced the use of atomicModifyIO

Re: [PATCH master] Fixup compatibility with GHC 7.4/base 4.5

On Fri, Jun 10, 2016 at 12:33:23PM +0200, 'Iustin Pop' via ganeti-devel wrote: > On Fri, Jun 10, 2016 at 12:30:03PM +0200, Iustin Pop wrote: > > It looks like commit c429dd26 introduced the use of atomicModifyIORef', > > which > > is only present in base 4.6 (GHC 7.6).

Re: [PATCH master 8/8] Ignore another bug in System.Time.diffClockTimes

On Thu, Jun 09, 2016 at 11:13:58PM +0200, Iustin Pop wrote: > From: Iustin Pop > > Reading the source for that function is scary: it uses local time conversions > in the context of what should be absolute (not local) time, so of course it > has > daylight savings time issues.

Re: [PATCH master 7/8] Replace getNode with getNodeByUuid in certain cases

On Thu, Jun 09, 2016 at 11:13:57PM +0200, Iustin Pop wrote: > From: Iustin Pop > > Do this in cases where this should already be the case: > - some functions are explicitly denoted to take UUIDs (documentation/parameter > name) > - some functions (e.g. clusterMasterNode,

Re: [PATCH master 6/8] Rework getInstDisks/getInstDisksFromObj

On Thu, Jun 09, 2016 at 11:13:56PM +0200, Iustin Pop wrote: > From: Iustin Pop > > The function getInstDisksFromObj takes an Instance object, extracts its uuids, > and passes that to getInstDisks. This function then looks up the given UUID in > the configuration, retrieves the

Re: [PATCH master] Fixup compatibility with GHC 7.4/base 4.5

On Fri, Jun 10, 2016 at 12:30:03PM +0200, Iustin Pop wrote: > It looks like commit c429dd26 introduced the use of atomicModifyIORef', which > is only present in base 4.6 (GHC 7.6). Let's temporarily fix that by adding a > small compat layer (which undoes the optimisations of using strict on 7.4,

Re: [PATCH master 4/8] Split getInstance into *ByUuid and *ByPartialName

On Fri, Jun 10, 2016 at 11:28:16AM +0100, Brian Foley wrote: > On Thu, Jun 09, 2016 at 11:13:54PM +0200, Iustin Pop wrote: > > From: Iustin Pop > > > > This function (and getNode, next patch) were two pain points when I tried to > > convert UUIDs to another type, since they're

[PATCH master] Fixup compatibility with GHC 7.4/base 4.5

for 7.4. Signed-off-by: Iustin Pop <ius...@google.com> --- src/Ganeti/Compat.hs | 10 ++ src/Ganeti/JQScheduler.hs | 5 +++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/src/Ganeti/Compat.hs b/src/Ganeti/Compat.hs index e5276d1..3ee3293 100644 --- a/src/

Re: [PATCH master 5/8] Split getNode into *ByUuid and *ByPartialName

On Thu, Jun 09, 2016 at 11:13:55PM +0200, Iustin Pop wrote: > From: Iustin Pop > > This follows the getInstance changes, with the same rationale. > > Signed-off-by: Iustin Pop LGTM

Re: [PATCH master 4/8] Split getInstance into *ByUuid and *ByPartialName

on getInstanceByName to > match the new functions better. > > Signed-off-by: Iustin Pop <ius...@google.com> It took me a little while to convince myself that some of the callers should use getInstanceByExactName, but I think you're correct. LGTM. > --- > src/Ganeti/Config.h

Re: [PATCH master 3/8] Switch TagSet from type synonim to a newtype

n Pop <ius...@google.com> Getting rid of all those extra Set.fromList <$> genTags and all those other explicit uses of Set is definitely an improvement. LGTM. Thanks! > --- > src/Ganeti/Objects.hs | 12 ++-- > src/Ganeti/Objects/Lens.hs | 3

Re: [PATCH master 2/8] Slightly improve the JSScheduler/prop_jobFiltering test

On Thu, Jun 09, 2016 at 11:13:52PM +0200, Iustin Pop wrote: > From: Iustin Pop > > For some reason, this test doesn't pass on Debian unstable. I don't know if it > passes on other versions of QuickCheck, but upon investigation, I think it was > always broken, but the use of

Re: [PATCH master 1/8] Fixup cabal dependencies relations

LGTM On Thu, Jun 9, 2016 at 10:13 PM, Iustin Pop wrote: > From: Iustin Pop > > In Haskell world (at least as far as I know), dependencies of type "<= > A.B.C" > are rarely used, because incrementing the fourth version component (i.e. D > in > A.B.C.D)

Re: [PATCH master 3/3] Fix KVM pinning tests to not depend on the local machine

y/ganeti.hypervisor.hv_kvm_unittest.py b/test/py/ >>> ganeti.hypervisor.hv_kvm_unittest.py >>> index c7a53b5..55ffb9b 100755 >>> --- a/test/py/ganeti.hypervisor.hv_kvm_unittest.py >>> +++ b/test/py/ganeti.hypervisor.hv_kvm_unittest.py >>> @@ -37,6 +37,7 @@

Re: [PATCH master 2/3] Don't use inexistent mock.assert_called_once()

LGTM, thanks for catching this bug! On Thu, Jun 9, 2016 at 11:33 PM, Iustin Pop wrote: > From: Iustin Pop > > Old versions of mock were not strict, thus allowing to call any method on > mocked objects, without complaining. More recent mock versions are >

Re: [PATCH master 1/3] Support python-mock versions later than about 1.1

; +try: > + # pylint: disable=W0212 > + _patcher = mock._patch_object > +except AttributeError: > + # pylint: disable=E1101 > + try: > +_patcher = mock.patch_object > + except AttributeError: > +_patcher = mock.patch.object > + > from ganeti import utils >

Re: [PATCH master 3/3] Fix KVM pinning tests to not depend on the local machine

ittest.py >> +++ b/test/py/ganeti.hypervisor.hv_kvm_unittest.py >> @@ -37,6 +37,7 @@ import socket >> import os >> import struct >> import re >> +from contextlib import nested >> >> from ganeti import serializer >> from ganeti import co

Re: [PATCH master 3/3] Fix KVM pinning tests to not depend on the local machine

ganeti.hypervisor.hv_kvm_unittest.py > index c7a53b5..55ffb9b 100755 > --- a/test/py/ganeti.hypervisor.hv_kvm_unittest.py > +++ b/test/py/ganeti.hypervisor.hv_kvm_unittest.py > @@ -37,6 +37,7 @@ import socket > import os > import struct > import re > +from contextlib impor

Re: On string data types

2016-06-07 13:10 GMT+02:00 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com>: > Hi Iustin, > > I'm half inclined to think that if we're going to change the type of UUID > we should do it 'properly', ie change to a strict, unpacked 16-byte array > that impl

Re: On string data types

that lets us distinguish between UUIDs for different Ganeti object types, eg Disks, Nodes, Instances, NICs etc. Efficiency-wise, yeah, we do have the problem where we don't have a suite of microbenchmarks at the moment, and we don't really know which functions are critical to overall performance. As you

Re: Issue 1025 in ganeti: Could not migrate instance ...: hypervisor returned failure

Comment #4 on issue 1025 by iust...@gmail.com: Could not migrate instance ...: hypervisor returned failure https://code.google.com/p/ganeti/issues/detail?id=1025 Hi Charlie, The commands.log and master-daemon.log are not the most interesting logs for this problem. Can you post the ganeti

Issue 1182 in ganeti: Recommended /etc/xen/scripts/vif-ganeti causes new instance creation to fail

Status: New Owner: New issue 1182 by ch...@simply-italian.co.uk: Recommended /etc/xen/scripts/vif-ganeti causes new instance creation to fail https://code.google.com/p/ganeti/issues/detail?id=1182 What software version are you running? Please provide the output of "gnt- cl

Re: Issue 1025 in ganeti: Could not migrate instance ...: hypervisor returned failure

Comment #3 on issue 1025 by charlieb...@gmail.com: Could not migrate instance ...: hypervisor returned failure https://code.google.com/p/ganeti/issues/detail?id=1025 I have got a Ganeti cluster (v2.6.2) with 4 VMs and only one fails with migration : # gnt-instance migrate vm.test.lan

Re: Issue 1175 in ganeti: ext3 filesystem hardcoded in fstab of import script

Comment #4 on issue 1175 by ius...@google.com: ext3 filesystem hardcoded in fstab of import script https://code.google.com/p/ganeti/issues/detail?id=1175 @hosting: will see about it. -- You received this message because this project is configured to send all issue notifications

Re: [PATCH instance-debootstrap 1/2] Add NEWS entry for current 0.15 release

LGTM, Thanks for cleaning this up! On Monday, May 30, 2016 at 9:53:37 PM UTC+1, Iustin Pop wrote: > > From: Iustin Pop > > It seems this was forgotten when the release was made. > > Signed-off-by: Iustin Pop > --- > NEWS | 12 > 1 file

Re: [PATCH instance-debootstrap 2/2] Version bump and NEWS file update for 0.16 release

ure.ac b/configure.ac > index 6c09749..040efb3 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -1,5 +1,5 @@ > AC_PREREQ(2.59) > -AC_INIT(ganeti-instance-debootstrap, 0.15, gan...@googlegroups.com) > +AC_INIT(ganeti-instance-debootstrap, 0.16, gan...@googlegroups.com) > > AC_CONFIG_AUX_DIR(autotools) > AC_CONFIG_SRCDIR(configure) > -- > 2.8.1 > >

Re: [PATCH stable-2.16 1/3] Fix optimisation: Correctly extract secondary node.

On Mon, May 30, 2016 at 03:47:40PM +0100, Brian Foley wrote: > On Mon, May 30, 2016 at 03:33:50PM +0200, Iustin Pop wrote: > > On Mon, May 30, 2016 at 02:10:14PM +0100, Ganeti Development List wrote: > > > It is possible for either nodeA or nodeB in a DRBD disk to be

Re: [PATCH stable-2.16 1/3] Fix optimisation: Correctly extract secondary node.

On Mon, May 30, 2016 at 03:33:50PM +0200, Iustin Pop wrote: > On Mon, May 30, 2016 at 02:10:14PM +0100, Ganeti Development List wrote: > > It is possible for either nodeA or nodeB in a DRBD disk to be the > > primary or the secondary. Patch 9825767ad incorrectly assumed nodeB > &

Re: [PATCH stable-2.16 1/3] Fix optimisation: Correctly extract secondary node.

On Mon, May 30, 2016 at 02:10:14PM +0100, Ganeti Development List wrote: > It is possible for either nodeA or nodeB in a DRBD disk to be the > primary or the secondary. Patch 9825767ad incorrectly assumed nodeB > was the secondary. Correct this by passing the actua

[PATCH master 1/2] Update build_chroot GHC packages

* Use deb8 (jessie) binaries and xz tarballs where available. These are substantially smaller downloads. * Upgrade from ghc 7.10.2 to 7.10.3. This fixes a number of bugs and a compile time speed regression. https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-7.10.3 Signed-off-by: Brian Foley

Re: [PATCH master] Add profiling for precise, trusty, xenial, jessie

LGTM On Saturday, May 28, 2016 at 1:54:41 PM UTC+1, Brian Foley wrote: > > Signed-off-by: Brian Foley > --- > devel/build_chroot | 37 ++--- > 1 file changed, 34 insertions(+), 3 deletions(-) > > diff --git a/devel/build_chroot

[PATCH stable-2.16 1/3] Fix optimisation: Correctly extract secondary node.

com> --- src/Ganeti/Config.hs | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/Ganeti/Config.hs b/src/Ganeti/Config.hs index 5b40694..5ada94b 100644 --- a/src/Ganeti/Config.hs +++ b/src/Ganeti/Config.hs @@ -159,10 +159,10 @@ instNodes cfg inst = maybe id S.

Re: UUID references (was: Tune getNodeInstances DRBD secondary) computation

On Mon, May 30, 2016 at 01:56:15PM +0200, Iustin Pop wrote: [snip] > > That's a huge improvement, and it would be great if you could get it to > > work. The problem I had is that there's a lot of places that use > > getItem'/getItem, and these currently require a String arg (because they > > can

Re: UUID references (was: Tune getNodeInstances DRBD secondary) computation

On Mon, May 30, 2016 at 12:18:37PM +0100, Brian Foley wrote: > On Mon, May 30, 2016 at 12:40:49PM +0200, Iustin Pop wrote: > > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > > > This is exercised by the luxi QueryInstances call when the sinst_cnt a

Re: UUID references (was: Tune getNodeInstances DRBD secondary) computation

On Mon, May 30, 2016 at 12:40:49PM +0200, Iustin Pop wrote: > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > > This is exercised by the luxi QueryInstances call when the sinst_cnt and > > sint_list fields are used. This uses a lot of CPU and does a l

Re: UUID references (was: Tune getNodeInstances DRBD secondary) computation

nvasive it gets and I'll send it against stable-2.16 (if not too invasive) or master otherwise. iustin > On Mon, May 30, 2016 at 11:40 AM, 'Iustin Pop' via ganeti-devel < > ganeti-devel@googlegroups.com> wrote: > > > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development Li

Re: UUID references (was: Tune getNodeInstances DRBD secondary) computation

reduce allocations as well, that would be awesome. Thanks again, Viktor On Mon, May 30, 2016 at 11:40 AM, 'Iustin Pop' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > > This is exercised by the luxi

UUID references (was: Tune getNodeInstances DRBD secondary) computation

On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > This is exercised by the luxi QueryInstances call when the sinst_cnt and > sint_list fields are used. This uses a lot of CPU and does a lot of > short-lived heap allocation on clusters with many

Re: [PATCH stable-2.16 1/2] Tune getNodeInstances DRBD secondary computation

2016-05-30 11:34 GMT+02:00 Brian Foley <bpfo...@google.com>: > On Sun, May 29, 2016 at 12:01:39PM +0200, Iustin Pop wrote: > > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > > > This is exercised by the luxi QueryInstances call when the sinst_c

Re: [PATCH instance-debootstrap 0/4] Cleanups and simple tests

2016-05-30 11:51 GMT+02:00 Brian Foley <bpfo...@google.com>: > On Sun, May 29, 2016 at 02:38:22AM +0200, Iustin Pop wrote: > > From: Iustin Pop <ius...@google.com> > > > > A few cleanups, and adding a test suite to be run without Ganeti, i.e. on > > stand

Re: [PATCH instance-debootstrap 0/4] Cleanups and simple tests

On Sun, May 29, 2016 at 02:38:22AM +0200, Iustin Pop wrote: > From: Iustin Pop <ius...@google.com> > > A few cleanups, and adding a test suite to be run without Ganeti, i.e. on > standalone systems. However it has downsides, see that patch's description. > > Note: I h

Re: [PATCH instance-debootstrap 2/4] Check for required programs at configure time

On Sun, May 29, 2016 at 02:38:24AM +0200, Iustin Pop wrote: > From: Iustin Pop > > This is not terribly helpful, especially when instance-deboostrap is > installed from packages and not from source, however it helps preventing > "obvious" missing dependencies for from-source

Re: [PATCH stable-2.16 1/2] Tune getNodeInstances DRBD secondary computation

On Sun, May 29, 2016 at 12:01:39PM +0200, Iustin Pop wrote: > On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > > This is exercised by the luxi QueryInstances call when the sinst_cnt and > > sint_list fields are used. This uses a lot of CPU and does a l

Re: [PATCH stable-2.16 1/2] Tune getNodeInstances DRBD secondary computation

On Thu, May 26, 2016 at 03:56:10PM +0100, Ganeti Development List wrote: > This is exercised by the luxi QueryInstances call when the sinst_cnt and > sint_list fields are used. This uses a lot of CPU and does a lot of > short-lived heap allocation on clusters with many

[PATCH master] Add profiling for precise, trusty, xenial, jessie

Signed-off-by: Brian Foley --- devel/build_chroot | 37 ++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/devel/build_chroot b/devel/build_chroot index 5694a27..6a7cc64 100755 --- a/devel/build_chroot +++ b/devel/build_chroot

Re: [PATCH stable-2.16 1/2] Tune getNodeInstances DRBD secondary computation

DRBD instances on 80 nodes. > > Signed-off-by: Brian Foley <bpfo...@google.com> > --- > src/Ganeti/Config.hs | 37 ++--- > 1 file changed, 26 insertions(+), 11 deletions(-) > > diff --git a/src/Ganeti/Config.hs b/src/Ganeti/Config.hs > in

[PATCH stable-2.16 1/2] Tune getNodeInstances DRBD secondary computation

. The net effect is to reduce heap use from ~3.2GB to ~1.5GB, and CPU use from ~1200ms to ~770ms in a test harness using a config with 1000 DRBD instances on 80 nodes. Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/Config.hs | 37 ++--- 1 file c

Re: [PATCH instance-debootstrap] Fix sfdisk invocation for util-linux 2.26+

2016-05-26 12:37 GMT+02:00 Brian Foley <bpfo...@google.com>: > On Thu, May 26, 2016 at 12:26:28PM +0200, 'Iustin Pop' via ganeti-devel > wrote: > > sfdisk changed in 2.26; it stopped supporting the geometry options (-C, > -H, > > -S), deprecated -L and -u.

Re: [PATCH instance-debootstrap] Fix sfdisk invocation for util-linux 2.26+

On Thu, May 26, 2016 at 12:26:28PM +0200, 'Iustin Pop' via ganeti-devel wrote: > sfdisk changed in 2.26; it stopped supporting the geometry options (-C, -H, > -S), deprecated -L and -u. However, we can't just drop these options, as > pre-2.26 needs them for devices that don't report geom

[PATCH instance-debootstrap] Fix sfdisk invocation for util-linux 2.26+

' output contains, as parsing the version number seems a bit more involved. Tested manually (not from Ganeti) with both 2.20, 2.25 and 2.28. The resulting partitioned disks have similarly-sized partitions, but the geometry information is different (as expected). This was reported in, and will fix

Re: [PATCH master] Build jessie-ghc710 cabal pkgs with profiling

LGTM On Tue, May 24, 2016 at 2:04 PM, 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > Signed-off-by: Brian Foley <bpfo...@google.com> > --- > devel/build_chroot | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --gi

[PATCH master] Build jessie-ghc710 cabal pkgs with profiling

Signed-off-by: Brian Foley --- devel/build_chroot | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/devel/build_chroot b/devel/build_chroot index 1cc640d..5694a27 100755 --- a/devel/build_chroot +++ b/devel/build_chroot @@ -473,7 +473,7 @@ case

Re: [PATCH stable-2.16 2/2] Get haskell daemons to only compress files > 4kB

LGTM On Fri, May 20, 2016 at 5:42 PM, 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > This applies to file data sent to noded in "upload_file_single" and > "jobqueue_upde", ie job files, the job serial number, and config.data > being r

Re: [PATCH stable-2.16 1/2] Use zlib compression level 3 in Haskell RPC code

LGTM On Fri, May 20, 2016 at 5:42 PM, 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > This matches the behaviour of the python code and is faster but still > fairly efficient. For a large config.data, gzip -3 takes 25ms vs 40ms > for gzip -6 (ie ~60%

[PATCH stable-2.16 0/2] Tweak zlib compression of haskell->noded comms

This brings is closer to the behaviour of the old Python code, and will be faster both in the cases of replicating small job files, and large config.data updates. Brian Foley (2): Use zlib compression level 3 in Haskell RPC code Get haskell daemons to only compress files > 4kB src/Gan

[PATCH stable-2.16 1/2] Use zlib compression level 3 in Haskell RPC code

time it's updated on the master. Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/Codec.hs | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/Ganeti/Codec.hs b/src/Ganeti/Codec.hs index 85ce266..404c70b 100644 --- a/src/Ganeti/Codec.hs +++ b/src/

[PATCH stable-2.16 2/2] Get haskell daemons to only compress files > 4kB

ilar to the existing Python behaviour in _Compress() and should avoid zlib & base64 overhead for small files. A threshold of 4kB is chosen to speed up the common case of job files (which are typically ~2.5kB) Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/Rpc.hs | 13 +++

Issue 1181 in ganeti: `gnt-instance migrate` applies "--paused" option from `gnt-instance start`

Status: New Owner: New issue 1181 by johannes...@gmail.com: `gnt-instance migrate` applies "--paused" option from `gnt-instance start` https://code.google.com/p/ganeti/issues/detail?id=1181 What software version are you running? Please provide the output of "gnt- cl

Issue 1180 in ganeti: Ceph/RBD support is broken with Ceph 'Jewel' due to incorrect showmapped command

Status: New Owner: New issue 1180 by keenan.t...@gmail.com: Ceph/RBD support is broken with Ceph 'Jewel' due to incorrect showmapped command https://code.google.com/p/ganeti/issues/detail?id=1180 gnt-cluster (ganeti 2.15.2-3) 2.15.2 Software version: 2.15.2 Internode protocol: 215

Re: [MERGE] stable-2.17 to master

LGTM On Tuesday, May 17, 2016 at 2:11:21 PM UTC+1, Brian Foley wrote: > > commit 314d7a116027a1aaa925abe0290174a368ac2179 > Merge: 99c2b44 20c24a8 > Author: Brian Foley > Date: Tue May 17 13:45:23 2016 +0100 > > Merge branch 'stable-2.17' > > * stable-2.17

[MERGE] stable-2.17 to master

commit 314d7a116027a1aaa925abe0290174a368ac2179 Merge: 99c2b44 20c24a8 Author: Brian Foley Date: Tue May 17 13:45:23 2016 +0100 Merge branch 'stable-2.17' * stable-2.17 * stable-2.16 Fix LogicalVolume Attach failure on missing path

Re: [MERGE] stable-2.16 to stable 2.17

LGTM On Tuesday, May 17, 2016 at 1:41:54 PM UTC+1, Brian Foley wrote: > > commit 10e851b0af6d2f1c3bca6fccde9b8f3ce8c5dd92 > Merge: 0b57a03 8bcf605 > Author: Brian Foley > Date: Tue May 17 13:35:34 2016 +0100 > > Merge branch 'stable-2.16' into stable-2.17 > >

[MERGE] stable-2.16 to stable 2.17

commit 10e851b0af6d2f1c3bca6fccde9b8f3ce8c5dd92 Merge: 0b57a03 8bcf605 Author: Brian Foley Date: Tue May 17 13:35:34 2016 +0100 Merge branch 'stable-2.16' into stable-2.17 * stable-2.16 Fix LogicalVolume Attach failure on missing path Set

Re: [MERGE] stable-2.15 to stable-2.16

> > diff --cc test/py/ganeti.mcpu_unittest.py > index 54bd640,796c2ba..42446c7 > --- a/test/py/ganeti.mcpu_unittest.py > +++ b/test/py/ganeti.mcpu_unittest.py > @@@ -33,14 -33,16 +33,17 @@@ > > import unittest > import itertools > + import mocks > + from cmd

[MERGE] stable-2.15 to stable-2.16

33,14 -33,16 +33,17 @@@ import unittest import itertools + import mocks + from cmdlib.testsupport.rpc_runner_mock import CreateRpcRunnerMock from ganeti import compat +from ganeti import errors from ganeti import mcpu from ganeti import opcodes from ganeti import cmdlib from g

Re: [PATCH stable-2.15] mcpu: Raise caught OpPrereqErrors with too few args

LGTM, thanks for the extensive test! On Tue, May 17, 2016 at 12:15 PM, 'Brian Foley' via ganeti-devel < ganeti-devel@googlegroups.com> wrote: > Improve mocks and add a unit test to test this case. > > Signed-off-by: Brian Foley <bpfo...@google.com> > --- > lib/mcpu.

[PATCH stable-2.15] mcpu: Raise caught OpPrereqErrors with too few args

_runner_mock import CreateRpcRunnerMock from ganeti import compat from ganeti import mcpu from ganeti import opcodes from ganeti import cmdlib from ganeti import locking +from ganeti import ht +from ganeti import errors from ganeti import constants from ganeti.constants import \ LOCK_ATTEMPTS_TIMEOUT,

Re: [PATCH stable-2.16] Fix LogicalVolume Attach failure on missing path

On Mon, May 16, 2016 at 12:29:56PM +0100, 'Federico Morg Pareschi' via ganeti-devel wrote: > This commit fixes a bug where a LogicalVolume would try to attach > to a non-existing device and then raise an exception, rather than > gracefully returning a negative result. > > As a co

[PATCH stable-2.16] Fix LogicalVolume Attach failure on missing path

d.DRBD8Dev.Attach()""" +drbd_mock.return_value = [1] +dev = drbd.DRBD8Dev.__new__(drbd.DRBD8Dev) +dev._aminor = 1 + +self.assertEqual(dev.Attach(), True) + + @testutils.patch_object(drbd.DRBD8, "GetUsedDevs") + def testAttach(self, drbd_mock): +"""Test for drbd.DRB

Re: [PATCH stable-2.16] Correctly set USE_VERSION_FULL=no when --enable-versionfull=no

LGTM On Thursday, May 12, 2016 at 5:03:53 PM UTC+1, Brian Foley wrote: > > We happened to be lucky, and the configure script only checks if > USE_VERSION_FULL == yes, so we don't see any side effect from this. > > Thanks to Tobias for reporting. > > Signed-off-by: Brian Foley

Re: [PATCH stable-2.16 1/2] Make the TH fieldsDictsKeys more efficient

For a test application that just parses ConfigData, this trims about > 1.5MB off the stripped -O2 optimised binary size (from 37.3MiB to > 35.8MiB). It also shaves a total of ~5MB of temporarily allocated heap > objects for the first parse of these objects. > > More importantly

[PATCH stable-2.16] Correctly set USE_VERSION_FULL=no when --enable-versionfull=no

We happened to be lucky, and the configure script only checks if USE_VERSION_FULL == yes, so we don't see any side effect from this. Thanks to Tobias for reporting. Signed-off-by: Brian Foley --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH stable-2.16 2/2] Generate concatenated error strings in parser

..instead of generating a splice thats contains several ++ operators. Surprisingly, this isn't optimised away by GHC and this microoptimisation shaves a few 10s of kB off the binary sizes. Signed-off-by: Brian Foley <bpfo...@google.com> --- src/Ganeti/THH.hs | 10 +- 1 file chan

[PATCH stable-2.16 0/2] A pair of simple Template Haskell codegen optimisations

These don't have a huge effect, but they slightly reduce binary size, heap utilisation, and make the splices easier to debug. Brian Foley (2): Make the TH fieldsDictsKeys more efficient Generate concatenated error strings in parser src/Ganeti/THH.hs | 30 +++--- 1

[PATCH stable-2.16 1/2] Make the TH fieldsDictsKeys more efficient

$(buildObject ...). For a test application that just parses ConfigData, this trims about 1.5MB off the stripped -O2 optimised binary size (from 37.3MiB to 35.8MiB). It also shaves a total of ~5MB of temporarily allocated heap objects for the first parse of these objects. More importantly it makes the sp

Re: Issue 1178 in ganeti: Live migration fails with permission error on /var/run/ganeti/kvm-hypervisor

Comment #2 on issue 1178 by bcandler...@googlemail.com: Live migration fails with permission error on /var/run/ganeti/kvm-hypervisor https://code.google.com/p/ganeti/issues/detail?id=1178 Only difference is that gnt-metad is running as root: root@nuc1:~# grep '_USER =' /etc/ganeti/share

Re: [PATCH stable-2.16] Don't verify disks when all disk templates are ext

On Tue, May 10, 2016 at 04:23:17PM +0100, 'Federico Morg Pareschi' via ganeti-devel wrote: > Verify disks should not run on clusters or nodegroups with only ext > storage, as it requires an unnecessary amount of locks for its jobs. > This commit performs additional checks when submitt

<    2   3   4   5   6   7   8   9   10   11   >