Re: [MERGE] stable-2.16 to stable 2.17

2016-12-16 Thread 'Federico Pareschi' via ganeti-devel
LGTM

On Friday, December 16, 2016 at 5:36:15 PM UTC, Brian Foley wrote:
>
> commit ac63745b0ee268fa267b1f29cea0aaa79587c7e6 
> Merge: f52c6e1 d287130 
> Author: Brian Foley <bpfo...@google.com> 
> Date:   Fri Dec 16 17:23:46 2016 + 
>
>     Merge branch 'stable-2.16' into stable-2.17 
> 
> * stable-2.15 
>   Fix gnt-instance console instance unpausing for xl toolstack 
>   Disable pylint too-many-nested-blocks in _RunCmdPipe 
>   Reduce nesting in import-export ProcessChildIO 
>   Reduce nesting in LUOobCommand.Exec 
>   Reduce nesting in LUInstanceCreate.RunOsScripts 
>   Reduce nesting in RemoveNodeSshKeyBulk key calculation 
>   Reduce nesting in RemoveNodeSshKeyBulk ssh logic 
>   Reduce nesting in gnt-cluster VerifyDisks missing disk loop 
>   Reduce nesting in _CheckVLANArguments 
>   Reduce nesting in StartDaemon 
>   Disable pylint bad-continuation warning 
>   Disable pylint superfluous-parens warning 
>   Disable pylint redefined-variable-type warning 
>   Disable pylint too-many-branches warnings 
>   Disable pylint broad-except warnings 
>   Disable incorrect pylint assigning-non-slot warning 
>   Quell pylint unbalanced-tuple-unpacking warning 
>   Cleanup: Use new-style classes everywhere 
>   Quell pylint socket.timeout warning 
>   Quell the pylint wrong-import-order warnings 
>   Quell cell-var-from-loop warning 
>   Use default value lambda param to avoid cell-var-from-loop 
>   Quell too-many-boolean-expressions 
>   Remove pylint tests removed in pylint 2.0 
>   Quell trailing newline 
>   Quell bad-whitespace warning 
>   Quell consider-using-enumerate warning 
>   Disable pylint unsubscriptable-object warning 
>   Disable pylint bare-except warning 
>   Disable unwanted pylint wrong-import-position warnings 
>   Disable pylint unused-wildcard-import warning 
>   Disable incorrect pylint not-callable warning 
>   Disable pylint unpacking-non-sequence warning 
>   Disable pylint misplaced-comparison-constant warning 
>   Disable incorect pylint simplify-if-statement warning 
>   Disable pylint eval-used warning 
>   Disable pylint invalid-name warning 
>   Disable pylint import-self warning 
>   Disable some pylint unused-import warnings 
>   Replace deprecated pylint >=0.27 pragma with new form 
>   Delete old warning disables removed from pylint 1.6 
>   Fix pylint >1.4 pycurl no-member warnings 
>   Cleanup: Remove unused/duplicate module/fn import 
>   Cleanup: Fix unidiomatic-typecheck 
>   Cleanup: Remove some unneeded pylint disables 
>   Cleanup: Iterate dict rather than key list 
>   Cleanup: Remove unused format key 
>   Cleanup: StartInstance and RebootInstance return None 
>   Cleanup: Fix for/else with no break in AddAuthorizedKeys 
>   Cleanup: Replace map/filters with list comprehensions 
>   Cleanup: del is a statement not a function 
>   Cleanup: Use FOO not in BAR instead of not FOO in BAR 
>   Cleanup: Simplify boolean assignment 
>   Cleanup: Remove some unnecessary if (...) parens 
>   Fix invalid variable error for file-based disks 
>   FIX: Refactor DiagnoseOS to use a loop, not an inner fn 
>   FIX: Set INSTANCE_NICn_NETWORK_NAME only if net is defined 
>   StartInstance restores instance state if running 
>   Allow migrate --cleanup to adopt an instance 
>   Make migrate --cleanup more robust 
>   Make finalize_migration_{src,dst} a single op 
>   Make FinalizeMigration{Src,Dst} more robust 
>   Fix instance state detection in _Shutdowninstance 
>   Code cleanup in hypervisor backend 
>   Fix for incorrect parsing of DRBD versions 
>   Fix for instance reinstall not updating config 
>   Change a few errors to report names, not UUIDs 
>   Give atomicWriteFile temp filenames a more distinct pattern 
>   LV check failure should print instance name 
>   Add ganeti-noded and ganeti-rapi --max-clients options 
>   Disable logging CallRPCMethod timings in non-debug configs 
>   568 Update hv_kvm to handle output from qemu >= 1.6.0 
>   Improve cluster verify ssh key errors 
>   Fix inconsistent spaces vs tabs indent in makefile 
> 
> * stable-2.13 
>   Bugfix: migrate needs HypervisorClass, not an instance 
> 
> Fix easy merge conflict in lib/backend.py -- dead code removed in 2.15 
> 
> Signed-off-by: Brian Foley <bpfo...@google.com> 
>
> diff --cc lib/backend.py 
> index 58c8b3a,1d4d2e4..5b83290 
> --- a/lib/backend.py 
>

[MERGE] stable-2.16 to stable 2.17

2016-12-16 Thread 'Brian Foley' via ganeti-devel
commit ac63745b0ee268fa267b1f29cea0aaa79587c7e6
Merge: f52c6e1 d287130
Author: Brian Foley <bpfo...@google.com>
Date:   Fri Dec 16 17:23:46 2016 +

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.15
  Fix gnt-instance console instance unpausing for xl toolstack
  Disable pylint too-many-nested-blocks in _RunCmdPipe
  Reduce nesting in import-export ProcessChildIO
  Reduce nesting in LUOobCommand.Exec
  Reduce nesting in LUInstanceCreate.RunOsScripts
  Reduce nesting in RemoveNodeSshKeyBulk key calculation
  Reduce nesting in RemoveNodeSshKeyBulk ssh logic
  Reduce nesting in gnt-cluster VerifyDisks missing disk loop
  Reduce nesting in _CheckVLANArguments
  Reduce nesting in StartDaemon
  Disable pylint bad-continuation warning
  Disable pylint superfluous-parens warning
  Disable pylint redefined-variable-type warning
  Disable pylint too-many-branches warnings
  Disable pylint broad-except warnings
  Disable incorrect pylint assigning-non-slot warning
  Quell pylint unbalanced-tuple-unpacking warning
  Cleanup: Use new-style classes everywhere
  Quell pylint socket.timeout warning
  Quell the pylint wrong-import-order warnings
  Quell cell-var-from-loop warning
  Use default value lambda param to avoid cell-var-from-loop
  Quell too-many-boolean-expressions
  Remove pylint tests removed in pylint 2.0
  Quell trailing newline
  Quell bad-whitespace warning
  Quell consider-using-enumerate warning
  Disable pylint unsubscriptable-object warning
  Disable pylint bare-except warning
  Disable unwanted pylint wrong-import-position warnings
  Disable pylint unused-wildcard-import warning
  Disable incorrect pylint not-callable warning
  Disable pylint unpacking-non-sequence warning
  Disable pylint misplaced-comparison-constant warning
  Disable incorect pylint simplify-if-statement warning
  Disable pylint eval-used warning
  Disable pylint invalid-name warning
  Disable pylint import-self warning
  Disable some pylint unused-import warnings
  Replace deprecated pylint >=0.27 pragma with new form
  Delete old warning disables removed from pylint 1.6
  Fix pylint >1.4 pycurl no-member warnings
  Cleanup: Remove unused/duplicate module/fn import
  Cleanup: Fix unidiomatic-typecheck
  Cleanup: Remove some unneeded pylint disables
  Cleanup: Iterate dict rather than key list
  Cleanup: Remove unused format key
  Cleanup: StartInstance and RebootInstance return None
  Cleanup: Fix for/else with no break in AddAuthorizedKeys
  Cleanup: Replace map/filters with list comprehensions
  Cleanup: del is a statement not a function
  Cleanup: Use FOO not in BAR instead of not FOO in BAR
  Cleanup: Simplify boolean assignment
  Cleanup: Remove some unnecessary if (...) parens
  Fix invalid variable error for file-based disks
  FIX: Refactor DiagnoseOS to use a loop, not an inner fn
  FIX: Set INSTANCE_NICn_NETWORK_NAME only if net is defined
  StartInstance restores instance state if running
  Allow migrate --cleanup to adopt an instance
  Make migrate --cleanup more robust
  Make finalize_migration_{src,dst} a single op
  Make FinalizeMigration{Src,Dst} more robust
  Fix instance state detection in _Shutdowninstance
  Code cleanup in hypervisor backend
  Fix for incorrect parsing of DRBD versions
  Fix for instance reinstall not updating config
  Change a few errors to report names, not UUIDs
  Give atomicWriteFile temp filenames a more distinct pattern
  LV check failure should print instance name
  Add ganeti-noded and ganeti-rapi --max-clients options
  Disable logging CallRPCMethod timings in non-debug configs
  568 Update hv_kvm to handle output from qemu >= 1.6.0
  Improve cluster verify ssh key errors
  Fix inconsistent spaces vs tabs indent in makefile

* stable-2.13
  Bugfix: migrate needs HypervisorClass, not an instance

Fix easy merge conflict in lib/backend.py -- dead code removed in 2.15

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc lib/backend.py
index 58c8b3a,1d4d2e4..5b83290
--- a/lib/backend.py
+++ b/lib/backend.py
@@@ -1914,17 -1903,21 +1919,21 @@@ def RemoveNodeSshKeyBulk(node_list
  error_msg_final = ("When removing the key of node '%s', updating the"
 " SSH key files of node '%s' failed. Last error"
 " was: %s.")
- if node in potential_master_candidates:
-   logging.debug("Updating key setup of potential master candidate 
node"
- " %s.", node)
+ 
+ if node in potential_master_candidates or from_authorized_keys:
+   if node in potential_master_candidates:

Re: [MERGE] stable-2.16 to stable 2.17

2016-08-17 Thread 'Viktor Bachraty' via ganeti-devel
LGTM, thanks.

On Wed, Aug 17, 2016 at 11:03 AM, 'Brian Foley' via ganeti-devel <
ganeti-devel@googlegroups.com> wrote:

> commit cf4942e3e8f28688437693f6afedc3f23035855c
> Merge: 3b5c416 84cc5fc
> Author: Brian Foley <bpfo...@google.com>
> Date:   Wed Aug 17 11:02:01 2016 +0100
>
> Merge branch 'stable-2.16' into stable-2.17
>
> * stable-2.16
>   Fix some typos/poor phrasing in gnt-node man page
>   Add missing cluster modify --modify-etc-hosts to man
>   Fix/quell hlint warnings
>
> Manually fix merge conflict in src/Ganeti/Utils.hs
>
> Signed-off-by: Brian Foley <bpfo...@google.com>
>
> diff --cc src/Ganeti/Utils.hs
> index 0c2a0ac,8a586d0..d061ec9
> --- a/src/Ganeti/Utils.hs
> +++ b/src/Ganeti/Utils.hs
> @@@ -849,45 -820,7 +849,46 @@@ isSubsequenceOf _ [
>   isSubsequenceOf a@(x:a') (y:b) | x == y= isSubsequenceOf a' b
>  | otherwise = isSubsequenceOf a b
>
>  +-- | Compute the maximum of two elements by a given order.
>  +-- As opposed to using `maximumBy`, is function is guaranteed
>  +-- to be total, as the signature enforces a non-empty list of
>  +-- arguments.
>  +maxBy :: (a -> a -> Ordering) -> a -> a -> a
>  +maxBy ord a b = maximumBy ord [a, b]
>  +
>  +-- | Given a predicate that is monotone on a list, find the
>  +-- first list entry where it holds, if any. Use the monotonicity
>  +-- property to evaluate the property at as few places as possible,
>  +-- guided by the heuristics provided.
>  +monotoneFind :: ([a] -> Int) -> (a -> Bool) -> [a] -> Maybe a
>  +monotoneFind heuristics p xs =
>  +  let count = heuristics xs
>  +  in case () of
>  +_ | x:xs' <- drop count xs
>  +-> if p x
>  + then (`mplus` Just x) . monotoneFind heuristics p
>  +  $ take count xs
>  + else monotoneFind heuristics p xs'
>  +_ | x:xs' <- xs
>  +-> if p x
>  + then Just x
>  + else monotoneFind heuristics p xs'
>  +_ -> Nothing
>  +
>  +-- | Iterate a function as long as it returns Just values, collecting
>  +-- all the Justs that where obtained.
>  +iterateJust :: (a -> Maybe a) -> a -> [a]
>  +iterateJust f a = a : maybe [] (iterateJust f) (f a)
>  +
>  +-- | A version of partition with a monadic predicate
>  +-- Implementation taken from David Fox's Extras package.
>  +partitionM :: (Monad m) => (a -> m Bool) -> [a] -> m ([a], [a])
>  +partitionM p xs = foldM f ([], []) xs
>  +  where f (a, b) x = do
>  +pv <- p x
>  +return $ if pv then (x : a, b) else (a, x : b)
>  +
> + {-# ANN frequency "HLint: ignore Use alternative" #-}
>   -- | Returns a list of tuples of elements and the number of times they
> occur
>   -- in a list
>   frequency :: Ord t => [t] -> [(Int, t)]
>


[MERGE] stable-2.16 to stable 2.17

2016-08-17 Thread 'Brian Foley' via ganeti-devel
commit cf4942e3e8f28688437693f6afedc3f23035855c
Merge: 3b5c416 84cc5fc
Author: Brian Foley <bpfo...@google.com>
Date:   Wed Aug 17 11:02:01 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Fix some typos/poor phrasing in gnt-node man page
  Add missing cluster modify --modify-etc-hosts to man
  Fix/quell hlint warnings

Manually fix merge conflict in src/Ganeti/Utils.hs

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc src/Ganeti/Utils.hs
index 0c2a0ac,8a586d0..d061ec9
--- a/src/Ganeti/Utils.hs
+++ b/src/Ganeti/Utils.hs
@@@ -849,45 -820,7 +849,46 @@@ isSubsequenceOf _ [
  isSubsequenceOf a@(x:a') (y:b) | x == y= isSubsequenceOf a' b
 | otherwise = isSubsequenceOf a b
  
 +-- | Compute the maximum of two elements by a given order.
 +-- As opposed to using `maximumBy`, is function is guaranteed
 +-- to be total, as the signature enforces a non-empty list of
 +-- arguments.
 +maxBy :: (a -> a -> Ordering) -> a -> a -> a
 +maxBy ord a b = maximumBy ord [a, b]
 +
 +-- | Given a predicate that is monotone on a list, find the
 +-- first list entry where it holds, if any. Use the monotonicity
 +-- property to evaluate the property at as few places as possible,
 +-- guided by the heuristics provided.
 +monotoneFind :: ([a] -> Int) -> (a -> Bool) -> [a] -> Maybe a
 +monotoneFind heuristics p xs =
 +  let count = heuristics xs
 +  in case () of
 +_ | x:xs' <- drop count xs
 +-> if p x
 + then (`mplus` Just x) . monotoneFind heuristics p
 +  $ take count xs
 + else monotoneFind heuristics p xs'
 +_ | x:xs' <- xs
 +-> if p x
 + then Just x
 + else monotoneFind heuristics p xs'
 +_ -> Nothing
 +
 +-- | Iterate a function as long as it returns Just values, collecting
 +-- all the Justs that where obtained.
 +iterateJust :: (a -> Maybe a) -> a -> [a]
 +iterateJust f a = a : maybe [] (iterateJust f) (f a)
 +
 +-- | A version of partition with a monadic predicate
 +-- Implementation taken from David Fox's Extras package.
 +partitionM :: (Monad m) => (a -> m Bool) -> [a] -> m ([a], [a])
 +partitionM p xs = foldM f ([], []) xs
 +  where f (a, b) x = do
 +pv <- p x
 +return $ if pv then (x : a, b) else (a, x : b)
 +
+ {-# ANN frequency "HLint: ignore Use alternative" #-}
  -- | Returns a list of tuples of elements and the number of times they occur
  -- in a list
  frequency :: Ord t => [t] -> [(Int, t)]


Re: [MERGE] stable-2.16 to stable-2.17

2016-08-16 Thread 'Viktor Bachraty' via ganeti-devel
LGTM, thanks!

On Tuesday, August 16, 2016 at 1:15:38 PM UTC+1, Brian Foley wrote:
>
> commit d55c7e85ac87be0529d2d84f8ed0f72c18f31e8f 
> Merge: 3a366d6 6a2114c 
> Author: Brian Foley <bpfo...@google.com> 
> Date:   Tue Aug 16 13:14:32 2016 +0100 
>
>     Merge branch 'stable-2.16' into stable-2.17 
> 
> * stable-2.16 
>   Improve error reporting in _VerifyClientCertificates 
>   Simplify some inscrutable map/map/ifilter/zip code 
>   Avoid overuse of operator in watcher *.py 
>   Sprinkle some more list comprehensions 
>   Replace map/partial with list comprehension 
>   Replace uses of map/lambda with more Pythonic code 
>   Replace map(operator.attrgetter, ...) uses 
>   Fix typos in gnt-cluster man page 
>   Hide errors for expected inotify failures in unittest 
>   Add gnt-instance rename --force option 
>   Improve documentation for gnt-instance failover 
>   Allow master failover to ignore offline nodes 
>   Fix LogicalVolume code to work with older /sbin/lvs 
>   Shorten verifyMasterVote failure message 
>   Adding a confirmation before gnt-node --offline no 
>   Removed unnecessary dependency from rpc in cli 
>   Refactor cli exception to its appropriate module 
>   Clean-up of code and fix of pylint warnings 
>   Use fork instead of spawnv in the watcher 
>   Make 'make pep8' happy 
> 
> Manually fix merge conflicts in src/Ganeti/Utils.py 
> 
> Signed-off-by: Brian Foley <bpfo...@google.com> 
>
> diff --cc src/Ganeti/Utils.hs 
> index 42a8db9,47f65c1..0c2a0ac 
> --- a/src/Ganeti/Utils.hs 
> +++ b/src/Ganeti/Utils.hs 
> @@@ -97,16 -96,10 +97,17 @@@ module Ganeti.Util 
> , ensurePermissions 
> , ordNub 
> , isSubsequenceOf 
>  +  , maxBy 
>  +  , threadDelaySeconds 
>  +  , monotoneFind 
>  +  , iterateJust 
>  +  , partitionM 
> +   , frequency 
> ) where 
>   
>  -import Control.Applicative 
>  +import Prelude () 
>  +import Ganeti.Prelude 
>  + 
>   import Control.Concurrent 
>   import Control.Exception (try, bracket) 
>   import Control.Monad 
> @@@ -116,17 -110,7 +117,19 @@@ import Data.Char (toUpper, isAlphaNum, 
>   import qualified Data.Either as E 
>   import Data.Function (on) 
>   import Data.IORef 
>  -import Data.List 
>  +#if MIN_VERSION_base(4,8,0) 
>  +import Data.List hiding (isSubsequenceOf) 
>  +#else 
>  +import Data.List ( intercalate 
>  + , find 
>  + , foldl' 
> ++ , group 
>  + , transpose 
> ++ , sort 
>  + , sortBy 
>  + , isPrefixOf 
>  + , maximumBy) 
>  +#endif 
>   import qualified Data.Map as M 
>   import Data.Maybe (fromMaybe) 
>   import qualified Data.Set as S 
> @@@ -846,41 -820,7 +849,46 @@@ isSubsequenceOf _ [ 
>   isSubsequenceOf a@(x:a') (y:b) | x == y= isSubsequenceOf a' b 
>  | otherwise = isSubsequenceOf a b 
>   
>  +-- | Compute the maximum of two elements by a given order. 
>  +-- As opposed to using `maximumBy`, is function is guaranteed 
>  +-- to be total, as the signature enforces a non-empty list of 
>  +-- arguments. 
>  +maxBy :: (a -> a -> Ordering) -> a -> a -> a 
>  +maxBy ord a b = maximumBy ord [a, b] 
>  + 
>  +-- | Given a predicate that is monotone on a list, find the 
>  +-- first list entry where it holds, if any. Use the monotonicity 
>  +-- property to evaluate the property at as few places as possible, 
>  +-- guided by the heuristics provided. 
>  +monotoneFind :: ([a] -> Int) -> (a -> Bool) -> [a] -> Maybe a 
>  +monotoneFind heuristics p xs = 
>  +  let count = heuristics xs 
>  +  in case () of 
>  +_ | x:xs' <- drop count xs 
>  +-> if p x 
>  + then (`mplus` Just x) . monotoneFind heuristics p 
>  +  $ take count xs 
>  + else monotoneFind heuristics p xs' 
>  +_ | x:xs' <- xs 
>  +-> if p x 
>  + then Just x 
>  + else monotoneFind heuristics p xs' 
>  +_ -> Nothing 
>  + 
>  +-- | Iterate a function as long as it returns Just values, collecting 
>  +-- all the Justs that where obtained. 
>  +iterateJust :: (a -> Maybe a) -> a -> [a] 
>  +iterateJust f a = a : maybe [] (iterateJust f) (f a) 
>  + 
>  +-- | A version of partition with a monadic predicate 
>  +-- Implementation taken from David Fox's Extras package. 
>  +partitionM :: (Monad m) => (a -> m Bool) -> [a] -> m ([a], [a]) 
>  +partitionM p xs = foldM f ([], []) xs 
>  +  where f (a, b) x = do 
>  +pv <- p x 
>  +return $ if pv then (x : a, b) else (a, x : b) 
> ++ 
> + -- | Returns a list of tuples of elements and the number of times they 
> occur 
> + -- in a list 
> + frequency :: Ord t => [t] -> [(Int, t)] 
> + frequency xs = map (\x -> (length x, head x)) . group . sort $ xs 
>


[MERGE] stable-2.16 to stable-2.17

2016-08-16 Thread 'Brian Foley' via ganeti-devel
commit d55c7e85ac87be0529d2d84f8ed0f72c18f31e8f
Merge: 3a366d6 6a2114c
Author: Brian Foley <bpfo...@google.com>
Date:   Tue Aug 16 13:14:32 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Improve error reporting in _VerifyClientCertificates
  Simplify some inscrutable map/map/ifilter/zip code
  Avoid overuse of operator in watcher *.py
  Sprinkle some more list comprehensions
  Replace map/partial with list comprehension
  Replace uses of map/lambda with more Pythonic code
  Replace map(operator.attrgetter, ...) uses
  Fix typos in gnt-cluster man page
  Hide errors for expected inotify failures in unittest
  Add gnt-instance rename --force option
  Improve documentation for gnt-instance failover
  Allow master failover to ignore offline nodes
  Fix LogicalVolume code to work with older /sbin/lvs
  Shorten verifyMasterVote failure message
  Adding a confirmation before gnt-node --offline no
  Removed unnecessary dependency from rpc in cli
  Refactor cli exception to its appropriate module
  Clean-up of code and fix of pylint warnings
  Use fork instead of spawnv in the watcher
  Make 'make pep8' happy

Manually fix merge conflicts in src/Ganeti/Utils.py

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc src/Ganeti/Utils.hs
index 42a8db9,47f65c1..0c2a0ac
--- a/src/Ganeti/Utils.hs
+++ b/src/Ganeti/Utils.hs
@@@ -97,16 -96,10 +97,17 @@@ module Ganeti.Util
, ensurePermissions
, ordNub
, isSubsequenceOf
 +  , maxBy
 +  , threadDelaySeconds
 +  , monotoneFind
 +  , iterateJust
 +  , partitionM
+   , frequency
) where
  
 -import Control.Applicative
 +import Prelude ()
 +import Ganeti.Prelude
 +
  import Control.Concurrent
  import Control.Exception (try, bracket)
  import Control.Monad
@@@ -116,17 -110,7 +117,19 @@@ import Data.Char (toUpper, isAlphaNum, 
  import qualified Data.Either as E
  import Data.Function (on)
  import Data.IORef
 -import Data.List
 +#if MIN_VERSION_base(4,8,0)
 +import Data.List hiding (isSubsequenceOf)
 +#else
 +import Data.List ( intercalate
 + , find
 + , foldl'
++ , group
 + , transpose
++ , sort
 + , sortBy
 + , isPrefixOf
 + , maximumBy)
 +#endif
  import qualified Data.Map as M
  import Data.Maybe (fromMaybe)
  import qualified Data.Set as S
@@@ -846,41 -820,7 +849,46 @@@ isSubsequenceOf _ [
  isSubsequenceOf a@(x:a') (y:b) | x == y= isSubsequenceOf a' b
 | otherwise = isSubsequenceOf a b
  
 +-- | Compute the maximum of two elements by a given order.
 +-- As opposed to using `maximumBy`, is function is guaranteed
 +-- to be total, as the signature enforces a non-empty list of
 +-- arguments.
 +maxBy :: (a -> a -> Ordering) -> a -> a -> a
 +maxBy ord a b = maximumBy ord [a, b]
 +
 +-- | Given a predicate that is monotone on a list, find the
 +-- first list entry where it holds, if any. Use the monotonicity
 +-- property to evaluate the property at as few places as possible,
 +-- guided by the heuristics provided.
 +monotoneFind :: ([a] -> Int) -> (a -> Bool) -> [a] -> Maybe a
 +monotoneFind heuristics p xs =
 +  let count = heuristics xs
 +  in case () of
 +_ | x:xs' <- drop count xs
 +-> if p x
 + then (`mplus` Just x) . monotoneFind heuristics p
 +  $ take count xs
 + else monotoneFind heuristics p xs'
 +_ | x:xs' <- xs
 +-> if p x
 + then Just x
 + else monotoneFind heuristics p xs'
 +_ -> Nothing
 +
 +-- | Iterate a function as long as it returns Just values, collecting
 +-- all the Justs that where obtained.
 +iterateJust :: (a -> Maybe a) -> a -> [a]
 +iterateJust f a = a : maybe [] (iterateJust f) (f a)
 +
 +-- | A version of partition with a monadic predicate
 +-- Implementation taken from David Fox's Extras package.
 +partitionM :: (Monad m) => (a -> m Bool) -> [a] -> m ([a], [a])
 +partitionM p xs = foldM f ([], []) xs
 +  where f (a, b) x = do
 +pv <- p x
 +return $ if pv then (x : a, b) else (a, x : b)
++
+ -- | Returns a list of tuples of elements and the number of times they occur
+ -- in a list
+ frequency :: Ord t => [t] -> [(Int, t)]
+ frequency xs = map (\x -> (length x, head x)) . group . sort $ xs


Re: [MERGE] stable-2.16 to stable-2.17

2016-07-07 Thread 'Viktor Bachraty' via ganeti-devel
LGTM, thanks!

On Thu, Jul 7, 2016 at 12:32 PM, 'Brian Foley' via ganeti-devel <
ganeti-devel@googlegroups.com> wrote:

> commit 588fa31dd22a1e9320a9bf355c66ca1b837ff586
> Merge: fd55874 20c2aec
> Author: Brian Foley <bpfo...@google.com>
> Date:   Thu Jul 7 12:12:35 2016 +0100
>
> Merge branch 'stable-2.16' into stable-2.17
>
> * stable-2.16
>   Fix some trivial pep8/pylint errors
>   Make executeRpcCall only compute rpcCallData once
>   Remove storage unit selection from rpcCallData
>   Only import needed functions from Ganeti.JSON
>   Special case WaitForJobChange to reduce heap use
>
> * stable-2.15
>   Cleanup more pylint/pep8/apidoc errors
>   KVM: handle gracefully too old/too new psutil versions
>   Fix small typo in opcode tests
>   Fix line-too long errors
>
> Manually update import lists to fix compile errors, and fix trivial
> pep8 whitespace warning.
>
> Signed-off-by: Brian Foley <bpfo...@google.com>
>
> diff --cc src/Ganeti/Config.hs
> index f67aff1,4d0e5a0..5687b54
> --- a/src/Ganeti/Config.hs
> +++ b/src/Ganeti/Config.hs
> @@@ -101,9 -99,8 +101,9 @@@ import System.I
>
>   import Ganeti.BasicTypes
>   import qualified Ganeti.Constants as C
>  +import qualified Ganeti.ConstantUtils as CU
>   import Ganeti.Errors
> - import Ganeti.JSON
> + import Ganeti.JSON (fromJResult, fromContainer, GenericContainer(..))
>   import Ganeti.Objects
>   import Ganeti.Types
>   import qualified Ganeti.Utils.MultiMap as MM
> diff --cc src/Ganeti/HTools/Backend/IAlloc.hs
> index e431947,3a67c2d..e40c3d7
> --- a/src/Ganeti/HTools/Backend/IAlloc.hs
> +++ b/src/Ganeti/HTools/Backend/IAlloc.hs
> @@@ -65,7 -65,7 +65,7 @@@ import Ganeti.HTools.AlgorithmParams (A
>   import Ganeti.HTools.CLI
>   import Ganeti.HTools.Loader
>   import Ganeti.HTools.Types
> - import Ganeti.JSON
>  -import Ganeti.JSON (maybeFromObj, JSRecord, tryFromObj, toArray,
> asObjectList, readEitherString, fromJResult, fromObj, fromObjWithDefault,
> asJSObject)
> ++import Ganeti.JSON (maybeFromObj, JSRecord, tryFromObj, toArray,
> asObjectList, readEitherString, fromJResult, fromObj, fromObjWithDefault,
> asJSObject, emptyContainer)
>   import Ganeti.Types ( EvacMode(ChangePrimary, ChangeSecondary)
>   , adminStateFromRaw, AdminState(..))
>   import Ganeti.Utils
> diff --cc src/Ganeti/HTools/Backend/Luxi.hs
> index be81a3c,53b0794..639d74d
> --- a/src/Ganeti/HTools/Backend/Luxi.hs
> +++ b/src/Ganeti/HTools/Backend/Luxi.hs
> @@@ -51,8 -51,7 +51,9 @@@ import Ganeti.HTools.Type
>   import qualified Ganeti.HTools.Group as Group
>   import qualified Ganeti.HTools.Node as Node
>   import qualified Ganeti.HTools.Instance as Instance
> - import Ganeti.JSON
> - import Ganeti.Objects as O
>  -import Ganeti.JSON (fromObj, fromJVal, tryFromObj, arrayMaybeFromJVal)
> ++import Ganeti.JSON (fromJVal, tryFromObj, arrayMaybeFromJVal,
> ++getKeysFromContainer, Container)
> ++import Ganeti.Objects (PartialNicParams)
>
>   {-# ANN module "HLint: ignore Eta reduce" #-}
>
> diff --cc src/Ganeti/HTools/Backend/Rapi.hs
> index 03d158c,7d76751..218411c
> --- a/src/Ganeti/HTools/Backend/Rapi.hs
> +++ b/src/Ganeti/HTools/Backend/Rapi.hs
> @@@ -53,7 -53,7 +53,7 @@@ import System.FilePat
>   import Ganeti.BasicTypes
>   import Ganeti.HTools.Loader
>   import Ganeti.HTools.Types
> - import Ganeti.JSON
>  -import Ganeti.JSON (loadJSArray, JSRecord, tryFromObj, fromJVal,
> maybeFromObj, fromJResult, tryArrayMaybeFromObj, readEitherString,
> fromObjWithDefault, asJSObject)
> ++import Ganeti.JSON (loadJSArray, JSRecord, tryFromObj, fromJVal,
> maybeFromObj, fromJResult, tryArrayMaybeFromObj, readEitherString,
> fromObjWithDefault, asJSObject, emptyContainer)
>   import qualified Ganeti.HTools.Group as Group
>   import qualified Ganeti.HTools.Node as Node
>   import qualified Ganeti.HTools.Instance as Instance
> diff --cc src/Ganeti/Luxi.hs
> index 4439cef,f763eee..831e859
> --- a/src/Ganeti/Luxi.hs
> +++ b/src/Ganeti/Luxi.hs
> @@@ -73,7 -71,7 +73,7 @@@ import Text.JSON.Type
>   import Ganeti.BasicTypes
>   import Ganeti.Constants
>   import Ganeti.Errors
> - import Ganeti.JSON
>  -import Ganeti.JSON (fromJResult, fromJVal, Tuple5(..), MaybeForJSON(..),
> TimeAsDoubleJSON(..))
> ++import Ganeti.JSON (fromJResult, fromJVal, fromObj, Tuple5(..),
> MaybeForJSON(..), TimeAsDoubleJSON(..))
>   import Ganeti.UDSServer
>   import Ganeti.Objects
>   import Ganeti.OpParams (pTagsObject)
> diff --cc src/Ganeti/Objects.hs
> index 4b92561,59abc5c..065aaa8
> --- a/src/Ganeti/Objects.hs
> +++ b/src/Ganeti/Objects.hs
> @@@ -130,1

[MERGE] stable-2.16 to stable-2.17

2016-07-07 Thread 'Brian Foley' via ganeti-devel
commit 588fa31dd22a1e9320a9bf355c66ca1b837ff586
Merge: fd55874 20c2aec
Author: Brian Foley <bpfo...@google.com>
Date:   Thu Jul 7 12:12:35 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Fix some trivial pep8/pylint errors
  Make executeRpcCall only compute rpcCallData once
  Remove storage unit selection from rpcCallData
  Only import needed functions from Ganeti.JSON
  Special case WaitForJobChange to reduce heap use

* stable-2.15
  Cleanup more pylint/pep8/apidoc errors
  KVM: handle gracefully too old/too new psutil versions
  Fix small typo in opcode tests
  Fix line-too long errors

Manually update import lists to fix compile errors, and fix trivial
pep8 whitespace warning.

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc src/Ganeti/Config.hs
index f67aff1,4d0e5a0..5687b54
--- a/src/Ganeti/Config.hs
+++ b/src/Ganeti/Config.hs
@@@ -101,9 -99,8 +101,9 @@@ import System.I
  
  import Ganeti.BasicTypes
  import qualified Ganeti.Constants as C
 +import qualified Ganeti.ConstantUtils as CU
  import Ganeti.Errors
- import Ganeti.JSON
+ import Ganeti.JSON (fromJResult, fromContainer, GenericContainer(..))
  import Ganeti.Objects
  import Ganeti.Types
  import qualified Ganeti.Utils.MultiMap as MM
diff --cc src/Ganeti/HTools/Backend/IAlloc.hs
index e431947,3a67c2d..e40c3d7
--- a/src/Ganeti/HTools/Backend/IAlloc.hs
+++ b/src/Ganeti/HTools/Backend/IAlloc.hs
@@@ -65,7 -65,7 +65,7 @@@ import Ganeti.HTools.AlgorithmParams (A
  import Ganeti.HTools.CLI
  import Ganeti.HTools.Loader
  import Ganeti.HTools.Types
- import Ganeti.JSON
 -import Ganeti.JSON (maybeFromObj, JSRecord, tryFromObj, toArray, 
asObjectList, readEitherString, fromJResult, fromObj, fromObjWithDefault, 
asJSObject)
++import Ganeti.JSON (maybeFromObj, JSRecord, tryFromObj, toArray, 
asObjectList, readEitherString, fromJResult, fromObj, fromObjWithDefault, 
asJSObject, emptyContainer)
  import Ganeti.Types ( EvacMode(ChangePrimary, ChangeSecondary)
  , adminStateFromRaw, AdminState(..))
  import Ganeti.Utils
diff --cc src/Ganeti/HTools/Backend/Luxi.hs
index be81a3c,53b0794..639d74d
--- a/src/Ganeti/HTools/Backend/Luxi.hs
+++ b/src/Ganeti/HTools/Backend/Luxi.hs
@@@ -51,8 -51,7 +51,9 @@@ import Ganeti.HTools.Type
  import qualified Ganeti.HTools.Group as Group
  import qualified Ganeti.HTools.Node as Node
  import qualified Ganeti.HTools.Instance as Instance
- import Ganeti.JSON
- import Ganeti.Objects as O
 -import Ganeti.JSON (fromObj, fromJVal, tryFromObj, arrayMaybeFromJVal)
++import Ganeti.JSON (fromJVal, tryFromObj, arrayMaybeFromJVal,
++getKeysFromContainer, Container)
++import Ganeti.Objects (PartialNicParams)
  
  {-# ANN module "HLint: ignore Eta reduce" #-}
  
diff --cc src/Ganeti/HTools/Backend/Rapi.hs
index 03d158c,7d76751..218411c
--- a/src/Ganeti/HTools/Backend/Rapi.hs
+++ b/src/Ganeti/HTools/Backend/Rapi.hs
@@@ -53,7 -53,7 +53,7 @@@ import System.FilePat
  import Ganeti.BasicTypes
  import Ganeti.HTools.Loader
  import Ganeti.HTools.Types
- import Ganeti.JSON
 -import Ganeti.JSON (loadJSArray, JSRecord, tryFromObj, fromJVal, 
maybeFromObj, fromJResult, tryArrayMaybeFromObj, readEitherString, 
fromObjWithDefault, asJSObject)
++import Ganeti.JSON (loadJSArray, JSRecord, tryFromObj, fromJVal, 
maybeFromObj, fromJResult, tryArrayMaybeFromObj, readEitherString, 
fromObjWithDefault, asJSObject, emptyContainer)
  import qualified Ganeti.HTools.Group as Group
  import qualified Ganeti.HTools.Node as Node
  import qualified Ganeti.HTools.Instance as Instance
diff --cc src/Ganeti/Luxi.hs
index 4439cef,f763eee..831e859
--- a/src/Ganeti/Luxi.hs
+++ b/src/Ganeti/Luxi.hs
@@@ -73,7 -71,7 +73,7 @@@ import Text.JSON.Type
  import Ganeti.BasicTypes
  import Ganeti.Constants
  import Ganeti.Errors
- import Ganeti.JSON
 -import Ganeti.JSON (fromJResult, fromJVal, Tuple5(..), MaybeForJSON(..), 
TimeAsDoubleJSON(..))
++import Ganeti.JSON (fromJResult, fromJVal, fromObj, Tuple5(..), 
MaybeForJSON(..), TimeAsDoubleJSON(..))
  import Ganeti.UDSServer
  import Ganeti.Objects
  import Ganeti.OpParams (pTagsObject)
diff --cc src/Ganeti/Objects.hs
index 4b92561,59abc5c..065aaa8
--- a/src/Ganeti/Objects.hs
+++ b/src/Ganeti/Objects.hs
@@@ -130,13 -124,11 +130,13 @@@ import qualified Text.JSON as 
  import qualified AutoConf
  import qualified Ganeti.Constants as C
  import qualified Ganeti.ConstantUtils as ConstantUtils
- import Ganeti.JSON
+ import Ganeti.JSON (DictObject(..), Container, emptyContainer, 
GenericContainer)
  import Ganeti.Objects.BitArray (BitArray)
  import Ganeti.Objects.Disk
 +import Ganeti.Objects.Maintenance
  import Ganeti.Objects.Nic
  import Ganeti.Objects.Instance
 +import Ganeti.Objects.HvState
  import Ganeti.Query.Language
  import Ganeti.PartialParams
  import Ganeti.Types
diff --cc src/Ganeti/Query/Node.hs
index 698e209,9d36c74..f431ade
--- a/src/Ganeti/Query/No

Re: [MERGE] stable-2.16 to stable-2.17

2016-06-10 Thread 'Iustin Pop' via ganeti-devel
2016-06-10 15:40 GMT+02:00 'Brian Foley' via ganeti-devel <
ganeti-devel@googlegroups.com>:

> commit b462d8c77bff0789e8a951288dea34226ab8b6d7
> Merge: 20c24a8 90281b4
> Author: Brian Foley <bpfo...@google.com>
> Date:   Fri Jun 10 14:35:13 2016 +0100
>
> Merge branch 'stable-2.16' into stable-2.17
>
> * stable-2.16
>   Fix optimisation: Correctly extract secondary node
>   Tune getNodeInstances DRBD secondary computation
>   Get haskell daemons to only compress files > 4kB
>   Use zlib compression level 3 in Haskell RPC code
>
> * stable-2.15
>   Fixup compatibility with GHC 7.4/base 4.5
>
> Manually resolve import conflicts.
>
> Signed-off-by: Brian Foley <bpfo...@google.com>
>

LGTM, although I can't check the merge, but the diffs look sane.

diff --cc src/Ganeti/Codec.hs
> index 9a41499,404c70b..6f54c0d
> --- a/src/Ganeti/Codec.hs
> +++ b/src/Ganeti/Codec.hs
> @@@ -37,21 -37,18 +37,23 @@@ module Ganeti.Code
> , decompressZlib
> ) where
>
>  +import Prelude ()
>  +import Ganeti.Prelude
>

Oh, this is interesting, I was wondering if there is such a module, and it
seems there is :)

iustin


[MERGE] stable-2.16 to stable-2.17

2016-06-10 Thread 'Brian Foley' via ganeti-devel
commit b462d8c77bff0789e8a951288dea34226ab8b6d7
Merge: 20c24a8 90281b4
Author: Brian Foley <bpfo...@google.com>
Date:   Fri Jun 10 14:35:13 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Fix optimisation: Correctly extract secondary node
  Tune getNodeInstances DRBD secondary computation
  Get haskell daemons to only compress files > 4kB
  Use zlib compression level 3 in Haskell RPC code

* stable-2.15
  Fixup compatibility with GHC 7.4/base 4.5

Manually resolve import conflicts.

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc src/Ganeti/Codec.hs
index 9a41499,404c70b..6f54c0d
--- a/src/Ganeti/Codec.hs
+++ b/src/Ganeti/Codec.hs
@@@ -37,21 -37,18 +37,23 @@@ module Ganeti.Code
, decompressZlib
) where
  
 +import Prelude ()
 +import Ganeti.Prelude
 +
- import Codec.Compression.Zlib (compress)
+ import Codec.Compression.Zlib
  import qualified Codec.Compression.Zlib.Internal as I
 -import Control.Monad.Error
 +import Control.Monad (liftM)
 +import Control.Monad.Error.Class (MonadError(..))
  import qualified Data.ByteString.Lazy as BL
  import qualified Data.ByteString.Lazy.Internal as BL
 -import Data.Monoid (mempty)
 +
 +import Ganeti.BasicTypes
  
+ 
  -- | Compresses a lazy bytestring.
  compressZlib :: BL.ByteString -> BL.ByteString
- compressZlib = compress
+ compressZlib = compressWith $
+   defaultCompressParams { compressLevel = CompressionLevel 3 }
  
  -- | Decompresses a lazy bytestring, throwing decoding errors using
  -- 'throwError'.
diff --cc src/Ganeti/JQScheduler.hs
index a229b2f,df6fefc..4c594fa
--- a/src/Ganeti/JQScheduler.hs
+++ b/src/Ganeti/JQScheduler.hs
@@@ -55,22 -52,12 +55,22 @@@ import Control.Applicative (liftA2
  import Control.Arrow
  import Control.Concurrent
  import Control.Exception
 -import Control.Monad
 +import Control.Monad ( when
 + , mfilter
 + , liftM
 + , void
 + , unless
 + , forever
 + , forM_)
  import Control.Monad.IO.Class
  import Data.Function (on)
- import Data.IORef
 -import Data.Functor ((<$))
+ import Data.IORef (IORef, atomicModifyIORef, newIORef, readIORef)
 -import Data.List
 +import Data.List ( find
 + , deleteFirstsBy
 + , sortBy
 + , intercalate
 + , partition
 + , insertBy)
  import Data.Maybe
  import qualified Data.Map as Map
  import Data.Ord (comparing)


Re: [MERGE] stable-2.16 to stable 2.17

2016-05-17 Thread 'Viktor Bachraty' via ganeti-devel
LGTM

On Tuesday, May 17, 2016 at 1:41:54 PM UTC+1, Brian Foley wrote:
>
> commit 10e851b0af6d2f1c3bca6fccde9b8f3ce8c5dd92 
> Merge: 0b57a03 8bcf605 
> Author: Brian Foley <bpfo...@google.com> 
> Date:   Tue May 17 13:35:34 2016 +0100 
>
>     Merge branch 'stable-2.16' into stable-2.17 
> 
> * stable-2.16 
>   Fix LogicalVolume Attach failure on missing path 
>   Set USE_VERSION_FULL=no if --enable-versionfull=no 
>   Generate concatenated error strings in parser 
>   Make the TH fieldsDictsKeys more efficient 
>   Don't verify disks when all disk templates are ext 
>   Improve test coverage for block device creation 
>   Docstring for LUClusterVerifyGroup 
>   Fix broken parameters in storage create methods 
>   Implement localized cache for lvs commands 
> 
> * stable-2.15 
>   mcpu: Raise caught OpPrereqErrors with too few args 
>   Make EnsureDirs print chmod mode on errors 
>   Fix CLI option typo in NEWS file 
> 
> Signed-off-by: Brian Foley <bpfo...@google.com> 
>
>

[MERGE] stable-2.16 to stable 2.17

2016-05-17 Thread 'Brian Foley' via ganeti-devel
commit 10e851b0af6d2f1c3bca6fccde9b8f3ce8c5dd92
Merge: 0b57a03 8bcf605
Author: Brian Foley <bpfo...@google.com>
Date:   Tue May 17 13:35:34 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Fix LogicalVolume Attach failure on missing path
  Set USE_VERSION_FULL=no if --enable-versionfull=no
  Generate concatenated error strings in parser
  Make the TH fieldsDictsKeys more efficient
  Don't verify disks when all disk templates are ext
  Improve test coverage for block device creation
  Docstring for LUClusterVerifyGroup
  Fix broken parameters in storage create methods
  Implement localized cache for lvs commands

* stable-2.15
  mcpu: Raise caught OpPrereqErrors with too few args
  Make EnsureDirs print chmod mode on errors
  Fix CLI option typo in NEWS file

Signed-off-by: Brian Foley <bpfo...@google.com>



Re: [MERGE] stable-2.16 to stable-2.17

2016-05-03 Thread 'Viktor Bachraty' via ganeti-devel
LGTM

On Tue, May 3, 2016 at 11:47 AM, 'Brian Foley' via ganeti-devel <
ganeti-devel@googlegroups.com> wrote:

> commit 394c8ffa24f93223049d11c6e7f676d04d5c4754
> Merge: b9de363 cf10806
> Author: Brian Foley <bpfo...@google.com>
> Date:   Tue May 3 11:32:16 2016 +0100
>
> Merge branch 'stable-2.16' into stable-2.17
>
> * stable-2.15
>   Use socat method string compatible with <1.73 & >=1.73
>   Reduce heap when parsing & storing ConfigData 10%
>   Cancel RAPI job if the client drops the connection
>   Make JQScheduler queues more strict to avoid leaks
>
> * stable-2.16
>   
>
> Manually resolve Constants.hs merge conflict. Both branches appended.
>
> Signed-off-by: Brian Foley <bpfo...@google.com>
>
> diff --cc src/Ganeti/Constants.hs
> index 7250372,420ccb6..13bff2e
> --- a/src/Ganeti/Constants.hs
> +++ b/src/Ganeti/Constants.hs
> @@@ -5534,13 -5496,10 +5534,22 @@@ dataCollectorDiagnoseDirectory = syscon
>   exTagsPrefix :: String
>   exTagsPrefix = Tags.exTagsPrefix
>
>  +-- * MaintD tag prefixes
>  +
>  +maintdPrefix :: String
>  +maintdPrefix = "maintd:"
>  +
>  +maintdSuccessTagPrefix :: String
>  +maintdSuccessTagPrefix = maintdPrefix ++ "repairready:"
>  +
>  +maintdFailureTagPrefix :: String
>  +maintdFailureTagPrefix = maintdPrefix ++ "repairfailed:"
> ++
> + -- | The polling frequency to wait for a job status change
> + cliWfjcFrequency :: Int
> + cliWfjcFrequency = 20
> +
> + -- | Default 'WaitForJobChange' timeout in seconds
> + defaultWfjcTimeout :: Int
> + defaultWfjcTimeout = 60
> ++
> diff --cc src/Ganeti/THH.hs
> index 6dd552f,6a3cb8f..1133490
> --- a/src/Ganeti/THH.hs
> +++ b/src/Ganeti/THH.hs
> @@@ -93,10 -90,12 +93,11 @@@ import Data.Attoparsec.Text (
> -- See issue #683 and https://ghc.haskell.org/trac/ghc/ticket/4899
>   import Data.Char
>   import Data.Function (on)
>  -import Data.List
>  +import Data.List (intercalate, groupBy, stripPrefix, sort, nub)
>   import Data.Maybe
>   import qualified Data.Map as M
>  -import Data.Monoid
>   import qualified Data.Set as S
> + import qualified Data.Text as T
>   import Language.Haskell.TH
>   import Language.Haskell.TH.Syntax (lift)
>
>


[MERGE] stable-2.16 to stable-2.17

2016-05-03 Thread 'Brian Foley' via ganeti-devel
commit 394c8ffa24f93223049d11c6e7f676d04d5c4754
Merge: b9de363 cf10806
Author: Brian Foley <bpfo...@google.com>
Date:   Tue May 3 11:32:16 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.15
  Use socat method string compatible with <1.73 & >=1.73
  Reduce heap when parsing & storing ConfigData 10%
  Cancel RAPI job if the client drops the connection
  Make JQScheduler queues more strict to avoid leaks

* stable-2.16
  

Manually resolve Constants.hs merge conflict. Both branches appended.

Signed-off-by: Brian Foley <bpfo...@google.com>

diff --cc src/Ganeti/Constants.hs
index 7250372,420ccb6..13bff2e
--- a/src/Ganeti/Constants.hs
+++ b/src/Ganeti/Constants.hs
@@@ -5534,13 -5496,10 +5534,22 @@@ dataCollectorDiagnoseDirectory = syscon
  exTagsPrefix :: String
  exTagsPrefix = Tags.exTagsPrefix
  
 +-- * MaintD tag prefixes
 +
 +maintdPrefix :: String
 +maintdPrefix = "maintd:"
 +
 +maintdSuccessTagPrefix :: String
 +maintdSuccessTagPrefix = maintdPrefix ++ "repairready:"
 +
 +maintdFailureTagPrefix :: String
 +maintdFailureTagPrefix = maintdPrefix ++ "repairfailed:"
++
+ -- | The polling frequency to wait for a job status change
+ cliWfjcFrequency :: Int
+ cliWfjcFrequency = 20
+ 
+ -- | Default 'WaitForJobChange' timeout in seconds
+ defaultWfjcTimeout :: Int
+ defaultWfjcTimeout = 60
++
diff --cc src/Ganeti/THH.hs
index 6dd552f,6a3cb8f..1133490
--- a/src/Ganeti/THH.hs
+++ b/src/Ganeti/THH.hs
@@@ -93,10 -90,12 +93,11 @@@ import Data.Attoparsec.Text (
-- See issue #683 and https://ghc.haskell.org/trac/ghc/ticket/4899
  import Data.Char
  import Data.Function (on)
 -import Data.List
 +import Data.List (intercalate, groupBy, stripPrefix, sort, nub)
  import Data.Maybe
  import qualified Data.Map as M
 -import Data.Monoid
  import qualified Data.Set as S
+ import qualified Data.Text as T
  import Language.Haskell.TH
  import Language.Haskell.TH.Syntax (lift)
  


Re: [MERGE] stable-2.16 to stable-2.17

2016-04-22 Thread 'Brian Foley' via ganeti-devel
On Fri, Apr 22, 2016 at 03:06:14PM +0100, 'Viktor Bachraty' via ganeti-devel 
wrote:
> commit 2f37deebd9be468336cb52e5db098e0a98f12a51
> Merge: 461c9c8 eb3f956
> Author: Viktor Bachraty <vbachr...@google.com>
> Date:   Fri Apr 22 15:01:51 2016 +0100
> 
>     Merge branch 'stable-2.16' into stable-2.17
> 
> * stable-2.16
>   Make debug http server more verbose
>   Reduce load in NV_NODENETTEST and NV_MASTERIP
>   Re-fix local port/ip logging in HttpClientRequest
> 
> * stable-2.15
>   Allow newer cabal deps for latest Debian/Ubuntu
> 
> Manually fixup merge conflicts.

LGTM.


[MERGE] stable-2.16 to stable-2.17

2016-04-22 Thread 'Viktor Bachraty' via ganeti-devel
commit 2f37deebd9be468336cb52e5db098e0a98f12a51
Merge: 461c9c8 eb3f956
Author: Viktor Bachraty <vbachr...@google.com>
Date:   Fri Apr 22 15:01:51 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Make debug http server more verbose
  Reduce load in NV_NODENETTEST and NV_MASTERIP
  Re-fix local port/ip logging in HttpClientRequest

* stable-2.15
  Allow newer cabal deps for latest Debian/Ubuntu

Manually fixup merge conflicts.

Signed-off-by: Viktor Bachraty <vbachr...@google.com>



Merge 'stable-2.16' to 'stable-2.17'

2016-01-05 Thread 'Helga Velroyen' via ganeti-devel
commit 0d10936ea1741b381583dc1a03aac4816f3b6014
Merge: cd0db80 23503ae
Author: Helga Velroyen <hel...@google.com>
Date:   Tue Jan 5 11:13:22 2016 +0100

    Merge branch 'stable-2.16' into stable-2.17

* stable-2.16
  Fix typo 'option' instead of 'options'
  Fix error message in attachInstanceDiskChecks
  Update documentation of harep
  Document harep --dry-run in the man page
  Support --dry-run in harep
  Add a --dry-run option to htools

* stable-2.15
  Add more documentation to testutils_ssh.py
  renew-crypto: use bulk-removal of SSH keys
  Use bulk-removal of SSH keys for single keys
  Bulk-removing SSH keys of diverse set of nodes
  Bulk-removal of SSH keys of normal nodes
  Bulk-remove SSH keys of potential master candidates
  Bulk-removal of SSH keys
  testutils: add keys to own 'authorized_keys' file
  Make mock SSH file manager deal with lists
  Don't deepcopy the config if the old value is not needed
  Revision bump for 2.15.2
  Update NEWS file for 2.15.2
  Compute lock allocation strictly

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  src/Ganeti/HTools/Program/Harep.hs

Resolutions:
  Harep.hs: use detectBroken from Repair.hs

Signed-off-by: Helga Velroyen <hel...@google.com>

diff --cc src/Ganeti/HTools/CLI.hs
index d24392d,7ca25d9..110375e
--- a/src/Ganeti/HTools/CLI.hs
+++ b/src/Ganeti/HTools/CLI.hs
@@@ -55,14 -55,12 +55,15 @@@ module Ganeti.HTools.CL
-- * The options
, oDataFile
, oDiskMoves
 +  , oAvoidDiskMoves
, oDiskTemplate
+   , oDryRun
, oSpindleUse
, oDynuFile
 +  , oMemWeight
, oMonD
, oMonDDataFile
 +  , oMonDKvmRSS
, oMonDXen
, oEvacMode
, oMonDExitMissing
diff --cc src/Ganeti/HTools/Program/Harep.hs
index c81bef8,8ad7deb..3010bf8
--- a/src/Ganeti/HTools/Program/Harep.hs
+++ b/src/Ganeti/HTools/Program/Harep.hs
@@@ -42,9 -42,13 +42,10 @@@ module Ganeti.HTools.Program.Hare
  import Control.Exception (bracket)
  import Control.Lens (over)
  import Control.Monad
 -import Data.Function
 -import Data.List
  import Data.Maybe
 -import Data.Ord
  import System.Time
  import qualified Data.Map as Map
+ import qualified Text.JSON as J

  import Ganeti.BasicTypes
  import Ganeti.Common
diff --cc src/Ganeti/Locking/Allocation.hs
index a17efb7,d1caa2a..4a681b4
--- a/src/Ganeti/Locking/Allocation.hs
+++ b/src/Ganeti/Locking/Allocation.hs
@@@ -49,13 -50,11 +50,14 @@@ module Ganeti.Locking.Allocatio
, freeLocks
) where

 -import Control.Applicative (liftA2, (<$>), (<*>), pure)
 +import Prelude ()
 +import Ganeti.Prelude
 +
 +import Control.Applicative (liftA2)
  import Control.Arrow (second, (***))
 -import Control.Monad
 +import Control.Monad (unless, guard, foldM, when)
  import Data.Foldable (for_, find)
+ import Data.List (foldl')
  import qualified Data.Map as M
  import Data.Maybe (fromMaybe)
  import qualified Data.Set as S
-- 

Helga Velroyen
Software Engineer
hel...@google.com

Google Germany GmbH
Dienerstraße 12
80331 München

Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den Absender und
löschen Sie die E-Mail und alle Anhänge. Vielen Dank.

This e-mail is confidential. If you are not the right addressee please do
not forward it, please inform the sender, and please erase this e-mail
including any attachments. Thanks.


Re: Merge 'stable-2.16' to 'stable-2.17'

2016-01-05 Thread 'Hrvoje Ribicic' via ganeti-devel
LGTM, thanks

On Tue, Jan 5, 2016 at 11:51 AM, 'Helga Velroyen' via ganeti-devel <
ganeti-devel@googlegroups.com> wrote:

> commit 0d10936ea1741b381583dc1a03aac4816f3b6014
> Merge: cd0db80 23503ae
> Author: Helga Velroyen <hel...@google.com>
> Date:   Tue Jan 5 11:13:22 2016 +0100
>
> Merge branch 'stable-2.16' into stable-2.17
>
> * stable-2.16
>   Fix typo 'option' instead of 'options'
>   Fix error message in attachInstanceDiskChecks
>   Update documentation of harep
>   Document harep --dry-run in the man page
>   Support --dry-run in harep
>   Add a --dry-run option to htools
>
> * stable-2.15
>   Add more documentation to testutils_ssh.py
>   renew-crypto: use bulk-removal of SSH keys
>   Use bulk-removal of SSH keys for single keys
>   Bulk-removing SSH keys of diverse set of nodes
>   Bulk-removal of SSH keys of normal nodes
>   Bulk-remove SSH keys of potential master candidates
>   Bulk-removal of SSH keys
>   testutils: add keys to own 'authorized_keys' file
>   Make mock SSH file manager deal with lists
>   Don't deepcopy the config if the old value is not needed
>   Revision bump for 2.15.2
>   Update NEWS file for 2.15.2
>   Compute lock allocation strictly
>
> * stable-2.14
>   Revision bump for 2.14.2
>   Update NEWS file for 2.14.2
>   Fix lines with more than 80 characters
>   Add more detach/attach sequence tests
>   Allow disk attachment to diskless instances
>   Improve tests for attaching disks
>
> * stable-2.13
>   Revision bump for 2.13.3
>   Update NEWS file for 2.13.3
>
> * stable-2.12
>   Bump revision number for 2.12.6
>   Update NEWS file for 2.12.6
>   Restrict showing of DRBD secret using types
>   Calculate correct affected nodes set in InstanceChangeGroup
>
> * stable-2.11
>   Revision bump for 2.11.8
>   Update NEWS file for 2.11.8
>
> * stable-2.10
>   Version bump for 2.10.8
>   Update NEWS file for 2.10.8
>
> * stable-2.9
>   Bump revision number
>   Update NEWS file for 2.9.7 release
>   Improve RAPI section on security
>   QA: Ensure the DRBD secret is not retrievable via RAPI
>   Redact the DRBD secret in instance queries
>   Do not attempt to use the DRBD secret in gnt-instance info
>
> Conflicts:
>   src/Ganeti/HTools/Program/Harep.hs
>
> Resolutions:
>   Harep.hs: use detectBroken from Repair.hs
>
> Signed-off-by: Helga Velroyen <hel...@google.com>
>
> diff --cc src/Ganeti/HTools/CLI.hs
> index d24392d,7ca25d9..110375e
> --- a/src/Ganeti/HTools/CLI.hs
> +++ b/src/Ganeti/HTools/CLI.hs
> @@@ -55,14 -55,12 +55,15 @@@ module Ganeti.HTools.CL
> -- * The options
> , oDataFile
> , oDiskMoves
>  +  , oAvoidDiskMoves
> , oDiskTemplate
> +   , oDryRun
> , oSpindleUse
> , oDynuFile
>  +  , oMemWeight
> , oMonD
> , oMonDDataFile
>  +  , oMonDKvmRSS
> , oMonDXen
> , oEvacMode
> , oMonDExitMissing
> diff --cc src/Ganeti/HTools/Program/Harep.hs
> index c81bef8,8ad7deb..3010bf8
> --- a/src/Ganeti/HTools/Program/Harep.hs
> +++ b/src/Ganeti/HTools/Program/Harep.hs
> @@@ -42,9 -42,13 +42,10 @@@ module Ganeti.HTools.Program.Hare
>   import Control.Exception (bracket)
>   import Control.Lens (over)
>   import Control.Monad
>  -import Data.Function
>  -import Data.List
>   import Data.Maybe
>  -import Data.Ord
>   import System.Time
>   import qualified Data.Map as Map
> + import qualified Text.JSON as J
>
>   import Ganeti.BasicTypes
>   import Ganeti.Common
> diff --cc src/Ganeti/Locking/Allocation.hs
> index a17efb7,d1caa2a..4a681b4
> --- a/src/Ganeti/Locking/Allocation.hs
> +++ b/src/Ganeti/Locking/Allocation.hs
> @@@ -49,13 -50,11 +50,14 @@@ module Ganeti.Locking.Allocatio
> , freeLocks
> ) where
>
>  -import Control.Applicative (liftA2, (<$>), (<*>), pure)
>  +import Prelude ()
>  +import Ganeti.Prelude
>  +
>  +import Control.Applicative (liftA2)
>   import Control.Arrow (second, (***))
>  -import Control.Monad
>  +import Control.Monad (unless, guard, foldM, when)
>   import Data.Foldable (for_, find)
> + import Data.List (foldl')
>   import qualified Data.Map as M
>   import Data.Maybe (fromMaybe)
>   import qualified Data.Set as S
> --
>
> Helga Velroyen
> Software Engineer
> hel...@google.com
>
> Google Germany GmbH
> Dienerstraße 12
> 80331 München
>
> Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle
> Registergericht und -nummer: Hamburg, HR