Build failed in Jenkins: brooklyn-library-master #485

[Apache Jenkins Server]

See 

--
[...truncated 553.32 KB...]
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/maven-metadata.xml
 (471 B at 0.4 KB/sec)
Deploying the main artifact brooklyn-qa-1.0.0-SNAPSHOT-tests.jar
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-tests.jar
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-tests.jar
 (78 KB at 62.2 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
 (2 KB at 1.4 KB/sec)
Deploying the main artifact brooklyn-qa-1.0.0-SNAPSHOT-sources.jar
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-sources.jar
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-sources.jar
 (53 KB at 42.6 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
 (2 KB at 1.1 KB/sec)
Deploying the main artifact brooklyn-qa-1.0.0-SNAPSHOT-test-sources.jar
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-test-sources.jar
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/brooklyn-qa-1.0.0-20171108.040111-32-test-sources.jar
 (50 KB at 39.0 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-qa/1.0.0-SNAPSHOT/maven-metadata.xml
 (2 KB at 1.3 KB/sec)
[INFO] Deployment in 
https://repository.apache.org/content/repositories/snapshots 
(id=apache.snapshots.https,uniqueVersion=true)
Deploying the main artifact brooklyn-software-cm-1.0.0-SNAPSHOT.pom
Downloading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/maven-metadata.xml
Downloaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/maven-metadata.xml
 (618 B at 1.0 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/brooklyn-software-cm-1.0.0-20171108.040124-32.pom
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/brooklyn-software-cm-1.0.0-20171108.040124-32.pom
 (4 KB at 2.6 KB/sec)
Downloading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/maven-metadata.xml
Downloaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/maven-metadata.xml
 (400 B at 0.7 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/maven-metadata.xml
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/1.0.0-SNAPSHOT/maven-metadata.xml
 (618 B at 0.4 KB/sec)
Uploading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/maven-metadata.xml
Uploaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm/maven-metadata.xml
 (400 B at 0.3 KB/sec)
[INFO] Deployment in 
https://repository.apache.org/content/repositories/snapshots 
(id=apache.snapshots.https,uniqueVersion=true)
Deploying the main artifact brooklyn-software-cm-ansible-1.0.0-SNAPSHOT.jar
Downloading: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm-ansible/1.0.0-SNAPSHOT/maven-metadata.xml
Downloaded: 
https://repository.apache.org/content/repositories/snapshots/org/apache/brooklyn/brooklyn-software-cm-ansible/1.0.0-SNAPSHOT/maven-metadata.xml
 (2 KB at 1.8 KB/sec)
Uploading: 

Build failed in Jenkins: brooklyn-server-master #789

[Apache Jenkins Server]

See 


Changes:

[graeme.miller] Add brooklyn-cli as a feature to karaf

--
[...truncated 8.64 MB...]
2017-11-07 21:45:30,550 INFO  TESTNG PASSED: "Surefire test" - 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.testTargetEntityByIdNotFoundWithResolutionTimeout()
 finished in 26 ms
2017-11-07 21:45:30,550 INFO  TESTNG INVOKING CONFIGURATION: "Surefire test" - 
@AfterMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.tearDown()
2017-11-07 21:45:30,552 INFO  TESTNG PASSED CONFIGURATION: "Surefire test" - 
@AfterMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.tearDown() 
finished in 2 ms
2017-11-07 21:45:30,552 INFO  TESTNG INVOKING CONFIGURATION: "Surefire test" - 
@BeforeMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.setUp()
2017-11-07 21:45:30,553 INFO  Added external config supplier named 
'brooklyn-demo-sample': 
org.apache.brooklyn.core.config.external.InPlaceExternalConfigSupplier@14a1769d
2017-11-07 21:45:30,560 INFO  TESTNG PASSED CONFIGURATION: "Surefire test" - 
@BeforeMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.setUp() finished 
in 8 ms
2017-11-07 21:45:30,560 INFO  TESTNG INVOKING: "Surefire test" - 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.testTargetEntityByIdWithDelayedEntityCreation()
2017-11-07 21:45:30,853 INFO  TESTNG PASSED: "Surefire test" - 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.testTargetEntityByIdWithDelayedEntityCreation()
 finished in 293 ms
2017-11-07 21:45:30,854 INFO  TESTNG INVOKING CONFIGURATION: "Surefire test" - 
@AfterMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.tearDown()
2017-11-07 21:45:30,861 INFO  TESTNG PASSED CONFIGURATION: "Surefire test" - 
@AfterMethod 
org.apache.brooklyn.test.framework.TargetableTestComponentTest.tearDown() 
finished in 7 ms
2017-11-07 21:45:30,914 INFO  TESTNG 
===
Surefire test
Tests run: 235, Failures: 1, Skips: 0
===
Tests run: 235, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 16.581 sec 
<<< FAILURE! - in TestSuite
testOneChildWhichPasses(org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest)
  Time elapsed: 0.121 sec  <<< FAILURE!
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException:
 Error invoking start at Application[jlsz5ip6]
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: java.util.concurrent.ExecutionException: 
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: 
ProblemStartingChildrenException: 
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Error invoking 
start at LoopOverGroupMembersTestCaseImpl{id=i2pfgcf8av}: Test failed on group 
member(s)
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: 
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: java.lang.reflect.InvocationTargetException
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: 
org.apache.brooklyn.core.entity.AbstractApplication$ProblemStartingChildrenException:
 org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Error invoking 
start at LoopOverGroupMembersTestCaseImpl{id=i2pfgcf8av}: Test failed on group 
member(s)
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: 
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: java.util.concurrent.ExecutionException: 
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Error invoking 
start at LoopOverGroupMembersTestCaseImpl{id=i2pfgcf8av}: Test failed on group 
member(s)
at 
org.apache.brooklyn.test.framework.LoopOverGroupMembersTestCaseTest.testOneChildWhichPasses(LoopOverGroupMembersTestCaseTest.java:79)
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: 
Caused by: java.util.concurrent.ExecutionException: 
org.apache.brooklyn.core.mgmt.internal.EffectorUtils$EffectorCallPropagatedRuntimeException:
 Error invoking start at LoopOverGroupMembersTestCaseImpl{id=i2pfgcf8av}: Test 
failed on group member(s)
Caused by: 

[GitHub] brooklyn-server issue #884: Add brooklyn-cli as a feature to karaf

[tbouron]

Github user tbouron commented on the issue:

https://github.com/apache/brooklyn-server/pull/884
  
I think you are right @aledsage, I clearly didn't think this through with 
#882. I would remove this dependency and add those item into `brooklyn-library` 
or `brooklyn-core` instead


---

[GitHub] brooklyn-server issue #884: Add brooklyn-cli as a feature to karaf

[aledsage]

Github user aledsage commented on the issue:

https://github.com/apache/brooklyn-server/pull/884
  
Turns out this might only be working because of an error in 
https://github.com/apache/brooklyn-server/blob/master/server-cli/pom.xml#L118 - 
there's a missing comma, so this seems to behave the same as 
`!*`. The `org.apache.brooklyn.cli` OSGi bundle 
has no `Import-Package` in its manifest!

If it did generate that correctly, then it would include an import package 
dependency on `org.apache.brooklyn.camp.brooklyn`, which comes from 
`brooklyn-camp`. The order the features are declared means it might try to 
install `brooklyn-cli` before `brooklyn-camp`, so its dependencies wouldn't 
resolve. Or maybe it would all be fine, with karaf correctly figuring out all 
the dependencies at startup.

---
However, the fundamentally concern is that there is no reason to include 
the `brooklyn-cli` module in karaf. Its java code is pretty-much unusable 
(we've excluded the `airlift` package import because that is not in karaf).

If we're just doing this to get the contents of its catalog.bom, maybe we 
should move its catalog.bom somewhere else instead.

As was discussed on the mailing list previously, we agreed to delete from 
production-code the brooklyn classic mode (i.e. the `Main` class way of 
launching brooklyn that is in `brooklyn-cli`). The suggestion was to move it to 
tests-only so developers could still launch it from their IDE if they really 
wanted. Therefore it would be nice to consider the `brooklyn-cli` module as 
deprecated.

Thoughts @Graeme-Miller @ahgittin @tbouron?


---

[GitHub] brooklyn-server pull request #884: Add brooklyn-cli as a feature to karaf

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-server/pull/884


---

[GitHub] brooklyn-server issue #884: Add brooklyn-cli as a feature to karaf

[aledsage]

Github user aledsage commented on the issue:

https://github.com/apache/brooklyn-server/pull/884
  
LGTM; merging.


---

[GitHub] brooklyn-server pull request #884: Add brooklyn-cli as a feature to karaf

[Graeme-Miller]

GitHub user Graeme-Miller opened a pull request:

https://github.com/apache/brooklyn-server/pull/884

Add brooklyn-cli as a feature to karaf

https://github.com/apache/brooklyn-server/pull/882 added brooklyn-cli to a 
catalog.bom. This causes brooklyn to fail to start, as karaf can't find 
brooklyn-cli. This PR adds brooklyn-cli to karaf

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Graeme-Miller/brooklyn-server 
add_brooklyn_cli_feature

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-server/pull/884.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #884


commit a2570d8b0a462e94410e2b2f5eab29b56100a0c5
Author: graeme.miller 
Date:   2017-11-07T21:17:22Z

Add brooklyn-cli as a feature to karaf




---

[GitHub] brooklyn-docs pull request #232: Fix typo + syntax highlighting

[tbouron]

GitHub user tbouron opened a pull request:

https://github.com/apache/brooklyn-docs/pull/232

Fix typo + syntax highlighting

Fix typo in the first YAML example (`:` at the end of the location) + add 
syntax highlighting 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tbouron/brooklyn-docs patch-4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-docs/pull/232.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #232


commit 2271b873db9866867e399380bbff7f2aa902425b
Author: Thomas Bouron 
Date:   2017-11-07T17:57:06Z

Fix typo + syntax highlighting




---

[GitHub] brooklyn-server pull request #883: Deprecates use of Task in ConfigKey

[aledsage]

GitHub user aledsage opened a pull request:

https://github.com/apache/brooklyn-server/pull/883

Deprecates use of Task in ConfigKey

Also avoid `StackOverflowError` in `ExecutionContext.getImmediately(task)`
When a non-`BasicTask` is passed in.

---
The motivation for deprecating setting a `ConfigKey` as a `Task` is that we 
really shouldn't be doing it! It can get very messed up if one tries to 
`getImmediately` or get non-blocking (e.g. by a `Transformer` that is trying to 
evaluate its result when triggered). That can leave the task in a cancelled 
state, so all subsequent calls to `task.get()` (i.e. all calls to 
`config().get(mykey)`) will fail!

Instead, we should be using `TaskFactory` (which is how the DSL evaluation 
works).

It's good to deprecate this before a major release of 1.0.0.

It would be good to add an overloaded method for setting config to 
`set(ConfigKey, TaskFactory)` so that such usage doesn't involve ugly 
casting to work around the strongly typed generics. We can do that as a stage 
two, separate from this PR.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aledsage/brooklyn-server 
deprecate-configkey-tasks

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-server/pull/883.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #883


commit 2e4fc3e205df4f9da01751fc2aba4ea004ef96bf
Author: Aled Sage 
Date:   2017-11-07T16:51:54Z

Deprecates use of Task in ConfigKey

Also avoid StackOverflowError in ExecutionContext.getImmediately(task)
When a non-BasicTask is passed in.




---

[GitHub] brooklyn-server pull request #877: update maven-bundle-plugin to include hea...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-server/pull/877


---

[GitHub] brooklyn-server issue #877: update maven-bundle-plugin to include headers fo...

[aledsage]

Github user aledsage commented on the issue:

https://github.com/apache/brooklyn-server/pull/877
  
Agreed this should be safe, and agree it's the right default. Merging.

---
For the record, my reasoning is...

bundles that are shipped in brooklyn will be automatically updated when a 
new brooklyn release is shipped. The new distro version will contain the new 
bundle versions of everything in brooklyn. They are extremely likely to contain 
java code, rather than pure yaml .bom files (for pure-yaml, we can much better 
handle having multiple versions - we don't need to worry about java binary 
compatibility, for which version of Brooklyn it was compiled against). These 
bundles should be automatically upgraded.

We should be careful in Brooklyn about moving entities/policies around 
between bundles, or renaming bundles. If we do, then that bundle may need to 
override the defaults.

Blueprints are increasingly written by the community (e.g. see 
contributions to github.com/brooklyncentral). Their release cadence is 
different from that of core brooklyn. Moving forwards, the core of brooklyn is 
not where we should be writing and maintaining a lot of entity integrations.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149334438
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149335124
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149344781
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149326857
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
--- End diff --

Perhaps we should just delete support for propagating effectors. We can 
live without it; it's additive (i.e. backwards compatible to add support for it 
later); and would simplify this enricher a lot.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149317018
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
+
+A simple use case is where we have two children, call them North and 
South, and we wish for North to be primary.  If North fails, however, we want 
to promote and fail over to South.  This can be done by:
+
+* adding this policy at the parent
+* setting ` ha.primary.weight` on North
+* optionally defining `promote` on North and South (if action is required 
there to promote it)
+* observing the `primary` sensor to see which is primary
+* optionally setting `propagate.primary.sensors: [ main.uri ]` to publish 
`main.uri` from whichever of North or South is active
+* optionally setting `primary.selection.mode: best` to switch back to 
North if it comes back online
+
+The policy works by listening for service-up changes in the target pool 
(children or members) and listening for `ha.primary.weight` sensor values from 
those elements.  
+On any change, it invokes an effector to perform the primary election.  
+By default, the effector invoked is `electPrimary`, but this can be 
changed with the `primary.election.effector` config key.  If this effector does 
not exist, the policy will add a default behaviour using 
`ElectPrimaryEffector`.  Details of the election are described in that 
effector, but to summarize, it will find an appropriate primary from the target 
pool and publish a sensor indicating who the new primary is.
+If the effector is not defined this policy will add one with the standard 
election behaviour
+({@link ElectPrimaryEffector}).
+That effector will also invoke `promote` and `demote` on the relevant 
entities.
+
+All the `primary.*` parameters accepted by that effector can be defined on 
this policy 
+and will be passed 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149319256
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
+
+A simple use case is where we have two children, call them North and 
South, and we wish for North to be primary.  If North fails, however, we want 
to promote and fail over to South.  This can be done by:
+
+* adding this policy at the parent
+* setting ` ha.primary.weight` on North
+* optionally defining `promote` on North and South (if action is required 
there to promote it)
+* observing the `primary` sensor to see which is primary
+* optionally setting `propagate.primary.sensors: [ main.uri ]` to publish 
`main.uri` from whichever of North or South is active
+* optionally setting `primary.selection.mode: best` to switch back to 
North if it comes back online
+
+The policy works by listening for service-up changes in the target pool 
(children or members) and listening for `ha.primary.weight` sensor values from 
those elements.  
+On any change, it invokes an effector to perform the primary election.  
+By default, the effector invoked is `electPrimary`, but this can be 
changed with the `primary.election.effector` config key.  If this effector does 
not exist, the policy will add a default behaviour using 
`ElectPrimaryEffector`.  Details of the election are described in that 
effector, but to summarize, it will find an appropriate primary from the target 
pool and publish a sensor indicating who the new primary is.
+If the effector is not defined this policy will add one with the standard 
election behaviour
+({@link ElectPrimaryEffector}).
+That effector will also invoke `promote` and `demote` on the relevant 
entities.
+
+All the `primary.*` parameters accepted by that effector can be defined on 
this policy 
+and will be passed 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149317795
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
+
+A simple use case is where we have two children, call them North and 
South, and we wish for North to be primary.  If North fails, however, we want 
to promote and fail over to South.  This can be done by:
+
+* adding this policy at the parent
+* setting ` ha.primary.weight` on North
+* optionally defining `promote` on North and South (if action is required 
there to promote it)
+* observing the `primary` sensor to see which is primary
+* optionally setting `propagate.primary.sensors: [ main.uri ]` to publish 
`main.uri` from whichever of North or South is active
+* optionally setting `primary.selection.mode: best` to switch back to 
North if it comes back online
+
+The policy works by listening for service-up changes in the target pool 
(children or members) and listening for `ha.primary.weight` sensor values from 
those elements.  
--- End diff --

Perhaps worth making the trigger sensor (defaulting to `service-up`) 
configurable. I could see that someone would want stabilisation delays etc.

However, I see that is additive (i.e. does not break backwards 
compatibility if we add it later) so is fine to defer.

(With hindsight, the way we've done stabilisation delays in 
service-restarter, service-replacer etc doesn't feel quite right. We should 
probably have just had a simple enricher that does a stabilisation delay, 
setting a different sensor.)


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149332837
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149353377
  
--- Diff: 
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/policies/ha/brooklyn/ElectPrimaryTest.java
 ---
@@ -0,0 +1,379 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.camp.policies.ha.brooklyn;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityLocal;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.camp.brooklyn.AbstractYamlTest;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.effector.AddEffector;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Dumper;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityAsserts;
+import org.apache.brooklyn.core.entity.EntityPredicates;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.trait.Startable;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.core.test.entity.TestEntity;
+import org.apache.brooklyn.policy.ha.ElectPrimaryConfig;
+import 
org.apache.brooklyn.policy.ha.ElectPrimaryConfig.PrimaryDefaultSensorsAndEffectors;
+import org.apache.brooklyn.policy.ha.ElectPrimaryConfig.SelectionMode;
+import org.apache.brooklyn.policy.ha.ElectPrimaryEffector;
+import org.apache.brooklyn.test.Asserts;
+import org.apache.brooklyn.test.support.LoggingVerboseReporter;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.text.Strings;
+import org.apache.brooklyn.util.time.Duration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testng.Assert;
+import org.testng.ITestNGListener;
+import org.testng.TestNG;
+import org.testng.annotations.Test;
+import org.testng.reporters.FailedReporter;
+
+import com.google.common.base.Predicates;
+import com.google.common.base.Stopwatch;
+
+// TODO this test is in the CAMP package because YAML not available in 
policies package
+public class ElectPrimaryTest extends AbstractYamlTest {
--- End diff --

There are no rebind tests - we need those as well.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149327198
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
--- End diff --

If we *really* want to propagate effectors, then perhaps the best pattern 
would be a dedicated `EffectorPropagator` that feels like the existing sensor 
`Propagator`. That would be a nice separation of concerns.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149349474
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149329523
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
+
blacklistedSensors.addAll(((EntityInternal)entity).getMutableEntityType().getSensors().keySet());
+for (Sensor s: Propagator.SENSORS_NOT_USUALLY_PROPAGATED) {
+blacklistedSensors.add(s.getName());
+}
+
blacklistedSensors.addAll(MutableSet.of(getConfig(PRIMARY_SENSOR_NAME), 
getConfig(ElectPrimaryConfig.PRIMARY_WEIGHT_NAME)));
+
+subscriptions().subscribe(entity, Sensors.newSensor(Entity.class, 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149320932
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
+
+A simple use case is where we have two children, call them North and 
South, and we wish for North to be primary.  If North fails, however, we want 
to promote and fail over to South.  This can be done by:
+
+* adding this policy at the parent
+* setting ` ha.primary.weight` on North
+* optionally defining `promote` on North and South (if action is required 
there to promote it)
+* observing the `primary` sensor to see which is primary
+* optionally setting `propagate.primary.sensors: [ main.uri ]` to publish 
`main.uri` from whichever of North or South is active
+* optionally setting `primary.selection.mode: best` to switch back to 
North if it comes back online
+
+The policy works by listening for service-up changes in the target pool 
(children or members) and listening for `ha.primary.weight` sensor values from 
those elements.  
+On any change, it invokes an effector to perform the primary election.  
+By default, the effector invoked is `electPrimary`, but this can be 
changed with the `primary.election.effector` config key.  If this effector does 
not exist, the policy will add a default behaviour using 
`ElectPrimaryEffector`.  Details of the election are described in that 
effector, but to summarize, it will find an appropriate primary from the target 
pool and publish a sensor indicating who the new primary is.
+If the effector is not defined this policy will add one with the standard 
election behaviour
+({@link ElectPrimaryEffector}).
+That effector will also invoke `promote` and `demote` on the relevant 
entities.
+
+All the `primary.*` parameters accepted by that effector can be defined on 
this policy 
+and will be passed 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149326658
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
--- End diff --

This will mess up on rebind - if we've already propagated the effectors of 
the primary, then on rebind when it calls `setEntity`, it will find these 
effectors on the entity and will blacklist them.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149314968
  
--- Diff: 
api/src/main/java/org/apache/brooklyn/api/mgmt/ManagementContext.java ---
@@ -309,8 +310,14 @@
 /** As {@link #lookup(String, Class)} but not constraining the return 
type */
 public BrooklynObject lookup(String id);
 
-/** Finds an entity with the given ID known at this management context 
*/
-// TODO in future support policies etc
+/** As {@link #lookup(Predicate)} comparing the ID of the object with 
the given string */
 public  T lookup(String id, Class type); 
 
-}
\ No newline at end of file
+/** Finds a {@link BrooklynObject} known in this management context 
+ * satisfying the given predicate, or null */
+public  T lookup(Predicate 
filter);
--- End diff --

Perhaps mark these as `@Beta` for now?


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149336035
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryConfig.java ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.time.Duration;
+
+public interface ElectPrimaryConfig {
+
+public enum TargetMode { CHILDREN, MEMBERS, AUTO }
+public static final ConfigKey TARGET_MODE = 
ConfigKeys.builder(TargetMode.class, "primary.target.mode")
+.description("where should the policy look for primary candidates; 
one of 'children', 'members', or 'auto' (members if it's a group)")
+.defaultValue(TargetMode.AUTO)
+.build();
+
+public enum SelectionMode { FAILOVER, BEST, STRICT }
+public static final ConfigKey SELECTION_MODE = 
ConfigKeys.builder(SelectionMode.class, "primary.selection.mode")
+.description("under what circumstances should the primary change:  
`failover` to change only if an existing primary is unhealthy, `best` to change 
so one with the highest weight is always selected, or `strict` to act as `best` 
but fail if several advertise the highest weight (for use when the weight 
sensor is updated by the nodes and should tell us unambiguously who was 
elected)")
+.defaultValue(SelectionMode.FAILOVER)
+.build();
+
+public static final ConfigKey BEST_WAIT_TIMEOUT = 
ConfigKeys.newDurationConfigKey("primary.stopped.wait.timeout",
+"if the highest-ranking primary is not starting, the effector will 
wait this long for it to be starting before picking a less highly-weighted 
primary; "
++ "default 5s, typically long enough to avoid races where multiple 
children are started concurrently but they complete extremely quickly and one 
completes before a better one starts, "
++ "the short duration is sufficient for the theoretical best to 
become waiting where the `primary.starting.wait.timeout` applies",
+Duration.seconds(5));
+
+public static final ConfigKey BEST_STARTING_WAIT_TIMEOUT = 
ConfigKeys.newDurationConfigKey("primary.starting.wait.timeout",
+"if the highest-ranking primary is starting, the effector will 
wait this long for it to be running before picking a less highly-weighted 
primary "
++ "(or in the case of `strict` before failing if there are ties); "
++ "default 5m, typically long enough to avoid races where multiple 
children are started and a sub-optimal one comes online before the best one",
+Duration.minutes(5));
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ConfigKeys.newStringConfigKey("primary.sensor.name",
+"name of sensor to publish, defaulting to 'primary'",
+PrimaryDefaultSensorsAndEffectors.PRIMARY.getName());
+
+public static final ConfigKey PRIMARY_WEIGHT_NAME = 
ConfigKeys.newStringConfigKey("primary.weight.name",
+"config key or sensor to scan from candidate nodes to determine 
who should be primary",
+PrimaryDefaultSensorsAndEffectors.PRIMARY_WEIGHT_SENSOR.getName());
+
+public static final ConfigKey PROMOTE_EFFECTOR_NAME = 
ConfigKeys.newStringConfigKey("primary.promote.effector.name",
+"effector to invoke on promotion, default `promote` and with no 
error if not present (but if set explicitly it will cause an error if not 
present)",
--- End diff --

I don't see where this gives an error if not present and set explicitly.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149350380
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
--- End diff --

I lean towards putting these in a sub-package, such as 
`org.apache.brooklyn.policy.ha.failover`. There are several classes that all 
work together, and that are unrelated to the other ha policies (such as 
`ServiceReplacer` etc).


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149328337
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
+
blacklistedSensors.addAll(((EntityInternal)entity).getMutableEntityType().getSensors().keySet());
--- End diff --

Same for this: after rebind, when it calls `setEntity`, we'll add to 
`blacklistedSensors` all the entity's sensors which will include the sensors 
that have been propagated from the existing primary.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149341592
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149324342
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
+
blacklistedSensors.addAll(((EntityInternal)entity).getMutableEntityType().getSensors().keySet());
+for (Sensor s: Propagator.SENSORS_NOT_USUALLY_PROPAGATED) {
+blacklistedSensors.add(s.getName());
+}
+
blacklistedSensors.addAll(MutableSet.of(getConfig(PRIMARY_SENSOR_NAME), 
getConfig(ElectPrimaryConfig.PRIMARY_WEIGHT_NAME)));
+
+subscriptions().subscribe(entity, Sensors.newSensor(Entity.class, 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149351458
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PrimaryRunningEnricher.java 
---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceNotUpLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Records if the elected primary child/member is running, updating 
service state of this entity 
+ * if it isn't running when it should be via the service problems map 
which in the event of
+ * an issue will contain more information about primary status. */ 
+@SuppressWarnings("rawtypes")
+public class PrimaryRunningEnricher extends AbstractEnricher implements 
SensorEventListener {
--- End diff --

I'd unit-test this directly, rather than just as part of 
`ElectPrimaryTest.runSelectionModeTest`.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149314895
  
--- Diff: 
core/src/main/java/org/apache/brooklyn/core/mgmt/internal/AbstractManagementContext.java
 ---
@@ -536,8 +538,52 @@ public BrooklynObject lookup(String id) {
 result = getLocationManager().getLocation(id);
 if (result!=null && type.isInstance(result)) return (T)result;
 
-// TODO policies, enrichers, feeds; bundles?
-return null;
+return lookup((o) -> { return type.isInstance(o) && 
Objects.equal(id, o.getId()); });
--- End diff --

`lookupAll` is inefficient, because we don't have indexes for 
policies/enrichers/feeds.

You could guard this call, only doing it if 
`!(Entity.class.isAssignableFrom(type) || 
Location.class.isAssignableFrom(type))`.

Longer term, we should see how often this is called with 1000s of entities 
- we'll make immutable copies of the various collections each time, iterating 
over them. But we can defer that, assuming it's not too bad just now?!


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149340800
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryEffector.java ---
@@ -0,0 +1,440 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Callable;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.EntityInitializer;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.EffectorBody;
+import 
org.apache.brooklyn.core.effector.EffectorTasks.EffectorBodyTaskFactory;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic;
+import 
org.apache.brooklyn.core.entity.lifecycle.ServiceStateLogic.ServiceProblemsLogic;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.MutableMap;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.core.task.Tasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.time.Duration;
+import org.apache.brooklyn.util.time.Time;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Stopwatch;
+import com.google.common.collect.Iterables;
+
+/**
+This effector will scan candidates among children or members to determine 
which should be noted as "primary".  
+The primary is selected from service-up candidates based on a numeric 
weight as a sensor or config on the candidates 
+(`ha.primary.weight`, unless overridden), with higher weights being 
preferred.  
+In the case of ties, or a new candidate emerging with a weight higher than 
a current healthy primary, 
+behaviour can be configured with `primary.selection.mode`.
+
+Returns a map containing a message, newPrimary, oldPrimary, and a {@link 
ResultCode} code.
+*/
+public class ElectPrimaryEffector implements EntityInitializer, 
ElectPrimaryConfig {
+
+private static final Logger log = 
LoggerFactory.getLogger(ElectPrimaryEffector.class);
+
+public static enum ResultCode { PRIMARY_UNCHANGED, 
NEW_PRIMARY_ELECTED, NO_PRIMARY_AVAILABLE }
+
+public static final Effector EFFECTOR = 
Effectors.effector(Object.class, "electPrimary").
+description("Scan to detect whether there is or should be a new 
primary").buildAbstract();
+
+private final ConfigBag paramsCreationTime;
+
+public ElectPrimaryEffector(ConfigBag params) {
+this.paramsCreationTime = params;
+}
+
+public ElectPrimaryEffector(Map params) {
+this(ConfigBag.newInstance(params));
+}
+
+
+// wire up the entity to call the task factory to create the task on 
invocation
+
+@Override
+public void apply(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+
((EntityInternal)entity).getMutableEntityType().addEffector(makeEffector(paramsCreationTime));
+}
+
+public static Effector makeEffector(ConfigBag params) {
+return 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149315793
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
--- End diff --

javadoc messes up for me (at least in eclipse's javadoc view), rendering it 
as a single paragraph. Worth using `` etc, and ``.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149324681
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
--- End diff --

This field (`lastPrimary`) is not set anywhere. Also, it won't survive 
rebind.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149328640
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
+Set blacklistedEffectors = MutableSet.of("start", "stop", 
"restart", "promote", "demote",
+getConfig(ElectPrimaryConfig.PROMOTE_EFFECTOR_NAME), 
getConfig(ElectPrimaryConfig.DEMOTE_EFFECTOR_NAME));
+Set blacklistedSensors = MutableSet.of();
+
+@SuppressWarnings("unchecked")
+public void setEntity(@SuppressWarnings("deprecation") 
org.apache.brooklyn.api.entity.EntityLocal entity) {
+super.setEntity(entity);
+
+
blacklistedEffectors.addAll(((EntityInternal)entity).getMutableEntityType().getEffectors().keySet());
+
blacklistedSensors.addAll(((EntityInternal)entity).getMutableEntityType().getSensors().keySet());
+for (Sensor s: Propagator.SENSORS_NOT_USUALLY_PROPAGATED) {
+blacklistedSensors.add(s.getName());
+}
+
blacklistedSensors.addAll(MutableSet.of(getConfig(PRIMARY_SENSOR_NAME), 
getConfig(ElectPrimaryConfig.PRIMARY_WEIGHT_NAME)));
+
+subscriptions().subscribe(entity, Sensors.newSensor(Entity.class, 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149336746
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryConfig.java ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.util.time.Duration;
+
+public interface ElectPrimaryConfig {
+
+public enum TargetMode { CHILDREN, MEMBERS, AUTO }
+public static final ConfigKey TARGET_MODE = 
ConfigKeys.builder(TargetMode.class, "primary.target.mode")
+.description("where should the policy look for primary candidates; 
one of 'children', 'members', or 'auto' (members if it's a group)")
+.defaultValue(TargetMode.AUTO)
+.build();
+
+public enum SelectionMode { FAILOVER, BEST, STRICT }
+public static final ConfigKey SELECTION_MODE = 
ConfigKeys.builder(SelectionMode.class, "primary.selection.mode")
+.description("under what circumstances should the primary change:  
`failover` to change only if an existing primary is unhealthy, `best` to change 
so one with the highest weight is always selected, or `strict` to act as `best` 
but fail if several advertise the highest weight (for use when the weight 
sensor is updated by the nodes and should tell us unambiguously who was 
elected)")
+.defaultValue(SelectionMode.FAILOVER)
+.build();
+
+public static final ConfigKey BEST_WAIT_TIMEOUT = 
ConfigKeys.newDurationConfigKey("primary.stopped.wait.timeout",
+"if the highest-ranking primary is not starting, the effector will 
wait this long for it to be starting before picking a less highly-weighted 
primary; "
++ "default 5s, typically long enough to avoid races where multiple 
children are started concurrently but they complete extremely quickly and one 
completes before a better one starts, "
++ "the short duration is sufficient for the theoretical best to 
become waiting where the `primary.starting.wait.timeout` applies",
+Duration.seconds(5));
+
+public static final ConfigKey BEST_STARTING_WAIT_TIMEOUT = 
ConfigKeys.newDurationConfigKey("primary.starting.wait.timeout",
+"if the highest-ranking primary is starting, the effector will 
wait this long for it to be running before picking a less highly-weighted 
primary "
++ "(or in the case of `strict` before failing if there are ties); "
++ "default 5m, typically long enough to avoid races where multiple 
children are started and a sub-optimal one comes online before the best one",
+Duration.minutes(5));
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ConfigKeys.newStringConfigKey("primary.sensor.name",
+"name of sensor to publish, defaulting to 'primary'",
+PrimaryDefaultSensorsAndEffectors.PRIMARY.getName());
+
+public static final ConfigKey PRIMARY_WEIGHT_NAME = 
ConfigKeys.newStringConfigKey("primary.weight.name",
+"config key or sensor to scan from candidate nodes to determine 
who should be primary",
+PrimaryDefaultSensorsAndEffectors.PRIMARY_WEIGHT_SENSOR.getName());
+
+public static final ConfigKey PROMOTE_EFFECTOR_NAME = 
ConfigKeys.newStringConfigKey("primary.promote.effector.name",
+"effector to invoke on promotion, default `promote` and with no 
error if not present (but if set explicitly it will cause an error if not 
present)",
--- End diff --

It was not clear that we'll first try to invoke `promote()` on the parent 
entity, but if the 

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149312203
  
--- Diff: 
api/src/main/java/org/apache/brooklyn/api/mgmt/ManagementContext.java ---
@@ -309,8 +310,14 @@
 /** As {@link #lookup(String, Class)} but not constraining the return 
type */
 public BrooklynObject lookup(String id);
 
-/** Finds an entity with the given ID known at this management context 
*/
-// TODO in future support policies etc
+/** As {@link #lookup(Predicate)} comparing the ID of the object with 
the given string */
 public  T lookup(String id, Class type); 
 
-}
\ No newline at end of file
+/** Finds a {@link BrooklynObject} known in this management context 
+ * satisfying the given predicate, or null */
+public  T lookup(Predicate 
filter);
--- End diff --

I'm hesitant about adding this, but see why you're doing it (because it 
fits with the existing `lookup` naming, and is useful for you're use-case!).

I don't want to bloat the public `ManagementContext` api. In other places, 
we've grouped related methods (e.g. `entity.config().set(...)`, rather than 
`entity.setConfig(...)`). Perhaps we should group the lookup methods.

I also prefer the guava conventions, rather than returning null when not 
found. Should we go for `Maybe tryLookup(Predicate filter)`, or 
even `tryFind()`? That is different from the existing `lookup(String id)`, but 
is a better api and is consistent with lots of other parts of Brooklyn.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149326090
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
+
+private static final Logger log = 
LoggerFactory.getLogger(PropagatePrimaryEnricher.class);
+
+public static final ConfigKey PRIMARY_SENSOR_NAME = 
ElectPrimaryConfig.PRIMARY_SENSOR_NAME;
+
+public static final ConfigKey PROPAGATE_EFFECTORS = 
ConfigKeys.newBooleanConfigKey("propagate.effectors",
+"Whether to propagate effectors, default true (effectors already 
defined here will not be propagated)",
+true);
+
+public static final ConfigKey PROPAGATING_ALL = 
Propagator.PROPAGATING_ALL;
+public static final ConfigKey> 
PROPAGATING_ALL_BUT = Propagator.PROPAGATING_ALL_BUT;
+public static final ConfigKey> 
PROPAGATING = Propagator.PROPAGATING;
+public static final ConfigKey> SENSOR_MAPPING = Propagator.SENSOR_MAPPING;
+
+Entity lastPrimary;
+Propagator propagator;
+
+Set effectorsAddedForPrimary;
--- End diff --

This field will not survive rebind - if we failover after rebind, then 
presumably it would leave the old effectors pointing at the previous primary, 
and add duplicate effectors pointing at the new primary?


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149351706
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/PropagatePrimaryEnricher.java
 ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.brooklyn.api.effector.Effector;
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.mgmt.TaskAdaptable;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.effector.EffectorTasks.EffectorTaskFactory;
+import org.apache.brooklyn.core.enricher.AbstractEnricher;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import 
org.apache.brooklyn.core.entity.EntityInternal.SensorSupportInternal;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.enricher.stock.Propagator;
+import org.apache.brooklyn.util.collections.MutableSet;
+import org.apache.brooklyn.util.core.config.ConfigBag;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Objects;
+
+/** Makes all sensors/effectors available on primary mirrored at this node,
+ * apart from those already present here. */ 
+@SuppressWarnings("rawtypes")
+public class PropagatePrimaryEnricher extends AbstractEnricher implements 
SensorEventListener {
--- End diff --

I'd unit test this directly, rather than just as part of the 
`ElectPrimaryTest`.


---

[GitHub] brooklyn-server pull request #879: Elect primary / failover policies

[aledsage]

Github user aledsage commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/879#discussion_r149323199
  
--- Diff: 
policy/src/main/java/org/apache/brooklyn/policy/ha/ElectPrimaryPolicy.java ---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.policy.ha;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.brooklyn.api.entity.Entity;
+import org.apache.brooklyn.api.entity.Group;
+import org.apache.brooklyn.api.mgmt.Task;
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.api.sensor.Sensor;
+import org.apache.brooklyn.api.sensor.SensorEvent;
+import org.apache.brooklyn.api.sensor.SensorEventListener;
+import org.apache.brooklyn.config.ConfigKey;
+import org.apache.brooklyn.core.config.ConfigKeys;
+import org.apache.brooklyn.core.effector.Effectors;
+import org.apache.brooklyn.core.entity.AbstractEntity;
+import org.apache.brooklyn.core.entity.Attributes;
+import org.apache.brooklyn.core.entity.Entities;
+import org.apache.brooklyn.core.entity.EntityInternal;
+import org.apache.brooklyn.core.entity.StartableApplication;
+import org.apache.brooklyn.core.entity.lifecycle.Lifecycle;
+import org.apache.brooklyn.core.mgmt.BrooklynTaskTags;
+import org.apache.brooklyn.core.policy.AbstractPolicy;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.entity.group.DynamicGroup;
+import org.apache.brooklyn.util.collections.MutableList;
+import org.apache.brooklyn.util.collections.QuorumCheck.QuorumChecks;
+import org.apache.brooklyn.util.core.task.DynamicTasks;
+import org.apache.brooklyn.util.exceptions.Exceptions;
+import org.apache.brooklyn.util.exceptions.UserFacingException;
+import org.apache.brooklyn.util.text.Strings;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Throwables;
+import com.google.common.reflect.TypeToken;
+
+/**
+
+The ElectPrimaryPolicy acts to keep exactly one of its children or members 
as primary, promoting and demoting them when required.
+
+A simple use case is where we have two children, call them North and 
South, and we wish for North to be primary.  If North fails, however, we want 
to promote and fail over to South.  This can be done by:
+
+* adding this policy at the parent
+* setting ` ha.primary.weight` on North
+* optionally defining `promote` on North and South (if action is required 
there to promote it)
+* observing the `primary` sensor to see which is primary
+* optionally setting `propagate.primary.sensors: [ main.uri ]` to publish 
`main.uri` from whichever of North or South is active
+* optionally setting `primary.selection.mode: best` to switch back to 
North if it comes back online
+
+The policy works by listening for service-up changes in the target pool 
(children or members) and listening for `ha.primary.weight` sensor values from 
those elements.  
+On any change, it invokes an effector to perform the primary election.  
+By default, the effector invoked is `electPrimary`, but this can be 
changed with the `primary.election.effector` config key.  If this effector does 
not exist, the policy will add a default behaviour using 
`ElectPrimaryEffector`.  Details of the election are described in that 
effector, but to summarize, it will find an appropriate primary from the target 
pool and publish a sensor indicating who the new primary is.
+If the effector is not defined this policy will add one with the standard 
election behaviour
+({@link ElectPrimaryEffector}).
+That effector will also invoke `promote` and `demote` on the relevant 
entities.
+
+All the `primary.*` parameters accepted by that effector can be defined on 
this policy 
+and will be passed 

[GitHub] brooklyn-server issue #873: Upgrade types and bundles as per bundle manifest...

[ahgittin]

Github user ahgittin commented on the issue:

https://github.com/apache/brooklyn-server/pull/873
  
Thanks @aledsage @robertgmoss - comments addressed and merged


---

[GitHub] brooklyn-server pull request #873: Upgrade types and bundles as per bundle m...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-server/pull/873


---

[GitHub] brooklyn-server pull request #873: Upgrade types and bundles as per bundle m...

[ahgittin]

Github user ahgittin commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/873#discussion_r149347708
  
--- Diff: 
launcher/src/test/java/org/apache/brooklyn/launcher/BrooklynLauncherRebindCatalogOsgiTest.java
 ---
@@ -740,20 +744,82 @@ public void testRebindUpgradeEntity() throws 
Exception {
 Entity entity = Iterables.getOnlyElement( 
Iterables.getOnlyElement(launcherLast.getManagementContext().getApplications()).getChildren()
 );
 
 // assert it was updated
+Assert.assertEquals(entity.getCatalogItemId(), 
"simple-entity:2.0.0");
 
-Assert.assertEquals(entity.getCatalogItemId(), 
"simple-entity:1.0.0");
-// TODO when upgrades work at level of catalog item ID and bundle 
ID, do below instead of above
-//Assert.assertEquals(entity.getCatalogItemId(), 
"simple-entity:2.0.0");
+if 
(CatalogUpgrades.markerForCodeThatLoadsJavaTypesButShouldLoadRegisteredType()) {
+
Assert.assertEquals(Entities.deproxy(entity).getClass().getName(), 
BasicEntityImpl.class.getName());
+} else {
+
Assert.assertEquals(Entities.deproxy(entity).getClass().getName(), 
"com.example.brooklyn.test.osgi.entities.SimpleEntityImpl");
+}
+}
+
+@Test
+public void 
testRebindUpgradeReferencedEntityFromCatalogAndDeployment() throws Exception {
+File initialBomFileV2 = 
prepForRebindRemovedItemTestReturningBomV2(false, true);
--- End diff --

yes


---

[GitHub] brooklyn-server pull request #873: Upgrade types and bundles as per bundle m...

[ahgittin]

Github user ahgittin commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/873#discussion_r149347438
  
--- Diff: 
core/src/main/java/org/apache/brooklyn/core/typereg/BundleUpgradeParser.java ---
@@ -312,6 +335,90 @@ public static boolean 
contains(Iterable names, VersionedName
 public static boolean contains(VersionRangedName range, 
VersionedName name) {
 return range.getSymbolicName().equals(name.getSymbolicName()) 
&& range.getOsgiVersionRange().includes(name.getOsgiVersion());
 }
+
+@Beta
+public static void storeInManagementContext(CatalogUpgrades 
catalogUpgrades, ManagementContext managementContext) {
+
((BasicBrooklynTypeRegistry)managementContext.getTypeRegistry()).storeCatalogUpgradesInstructions(catalogUpgrades);
+}
+
+@Beta @Nonnull
+public static CatalogUpgrades 
getFromManagementContext(ManagementContext managementContext) {
+CatalogUpgrades result = 
((BasicBrooklynTypeRegistry)managementContext.getTypeRegistry()).getCatalogUpgradesInstructions();
+if (result!=null) {
+return result;
+}
+return builder().build();
+}
+
+@Beta
+public static String 
getBundleUpgradedIfNecessary(ManagementContext mgmt, String vName) {
+if (vName==null) return null;
+Maybe osgi = 
((ManagementContextInternal)mgmt).getOsgiManager();
+if (osgi.isAbsent()) {
+// ignore upgrades if not osgi
+return vName;
+}
+if 
(osgi.get().getManagedBundle(VersionedName.fromString(vName))!=null) {
+// bundle installed, prefer that to upgrade 
+return vName;
+}
+return getItemUpgradedIfNecessary(mgmt, vName, (cu,vn) -> 
cu.getUpgradesForBundle(vn));
+}
+@Beta
+public static String getTypeUpgradedIfNecessary(ManagementContext 
mgmt, String vName) {
+if (vName==null || mgmt.getTypeRegistry().get(vName)!=null) {
+// item found (or n/a), prefer that to upgrade
+return vName;
+}
+// callers should use BrooklynTags.newUpgradedFromTag if 
creating a spec,
+// then warn on instantiation, done only for entities 
currently.
+// (yoml will allow us to have one code path for all the 
different creation routines.)
+// persistence/rebind also warns.
+return getItemUpgradedIfNecessary(mgmt, vName, (cu,vn) -> 
cu.getUpgradesForType(vn));
+}
+
+private interface LookupFunction {
+public Set lookup(CatalogUpgrades cu, 
VersionedName vn);
+}
+private static String getItemUpgradedIfNecessary(ManagementContext 
mgmt, String vName, LookupFunction lookupF) {
+Set r = 
lookupF.lookup(getFromManagementContext(mgmt), VersionedName.fromString(vName));
+if (!r.isEmpty()) return r.iterator().next().toString();
+
+if (log.isTraceEnabled()) {
+log.trace("Could not find '"+vName+"' and no upgrades 
specified; subsequent failure or warning possible unless that is a direct java 
class reference");
+}
+return vName;
+}
+
+/** This method is used internally to mark places we need to 
update when we switch to persisting and loading
+ *  registered type IDs instead of java types, as noted on 
RebindIteration.load */
+@Beta
+public static boolean 
markerForCodeThatLoadsJavaTypesButShouldLoadRegisteredType() {
+// return true if we use registered types, and update callers 
not to need this method
+return false;
+}
+
+@Beta
+CatalogUpgrades withTypeCleared(VersionedName versionedName) {
+return 
builder().addAll(this).clearUpgradesProvidedByType(versionedName).build();
+}
+@Beta
+CatalogUpgrades withBundleCleared(VersionedName versionedName) {
+return 
builder().addAll(this).clearUpgradesProvidedByType(versionedName).build();
--- End diff --

yes!


---

[GitHub] brooklyn-server pull request #873: Upgrade types and bundles as per bundle m...

[ahgittin]

Github user ahgittin commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/873#discussion_r149346988
  
--- Diff: 
core/src/main/java/org/apache/brooklyn/core/mgmt/persist/XmlMementoSerializer.java
 ---
@@ -341,7 +371,7 @@ public void marshal(Object source, 
HierarchicalStreamWriter writer, MarshallingC
 public Object unmarshal(HierarchicalStreamReader reader, 
UnmarshallingContext context) {
 if (reader.hasMoreChildren()) {
 Class type = readClassType(reader, mapper);
-//Class type2 = context.getRequiredType();
+// could confirm it is subtype of context.getRequiredType()
--- End diff --

TODO to me implies a bit stronger intention, here it's a "not yet sure 
should we do"


---

[jira] [Updated] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BROOKLYN-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Bouron updated BROOKLYN-548:
---
Attachment: (was: screenshot-1.png)

> Brooklyn node ID changes on restart
> ---
>
> Key: BROOKLYN-548
> URL: https://issues.apache.org/jira/browse/BROOKLYN-548
> Project: Brooklyn
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Thomas Bouron
>
> I was testing Brooklyn HA and noticed something odd: when restarting an HA 
> Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.
> For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master 
> (not killed, stopped), slave took over as expected. Then I restarted the 
> previous master, it came back online as a slave (expected) but the node ID 
> was different. I was expecting it to keep its ID rather creating a new one.
> {code}
> {
> "planeId": "qDpn2Lau",
> "ownId": "nJagUrw8",
> "masterId": "nJagUrw8",
> "nodes": {
> "BgOjYcml": {
> "nodeId": "BgOjYcml",
> "nodeUri": null,
> "status": "TERMINATED",
> "localTimestamp": 1510052259169,
> "remoteTimestamp": 151005226
> },
> "iv3XfaEu": {
> "nodeId": "iv3XfaEu",
> "nodeUri": null,
> "status": "STANDBY",
> "localTimestamp": 1510054973636,
> "remoteTimestamp": 1510054974000
> },
> "nJagUrw8": {
> "nodeId": "nJagUrw8",
> "nodeUri": null,
> "status": "MASTER",
> "localTimestamp": 1510054975025,
> "remoteTimestamp": 1510054976000
> }
> },
> "links": {}
> }
> {code}
> On the attached screenshot, {{BgOjYcml}} is the same node as {{iv3XfaEu}}. 
> {{BgOjYcml}} was the ID of the node before restart



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BROOKLYN-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Bouron updated BROOKLYN-548:
---
Description: 
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one.

{code}
{
"planeId": "qDpn2Lau",
"ownId": "nJagUrw8",
"masterId": "nJagUrw8",
"nodes": {
"BgOjYcml": {
"nodeId": "BgOjYcml",
"nodeUri": null,
"status": "TERMINATED",
"localTimestamp": 1510052259169,
"remoteTimestamp": 151005226
},
"iv3XfaEu": {
"nodeId": "iv3XfaEu",
"nodeUri": null,
"status": "STANDBY",
"localTimestamp": 1510054973636,
"remoteTimestamp": 1510054974000
},
"nJagUrw8": {
"nodeId": "nJagUrw8",
"nodeUri": null,
"status": "MASTER",
"localTimestamp": 1510054975025,
"remoteTimestamp": 1510054976000
}
},
"links": {}
}
{code}

On the attached screenshot, {{BgOjYcml}} is the same node as {{iv3XfaEu}}. 
{{BgOjYcml}} was the ID of the node before restart

  was:
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one (see 
screenshot)


> Brooklyn node ID changes on restart
> ---
>
> Key: BROOKLYN-548
> URL: https://issues.apache.org/jira/browse/BROOKLYN-548
> Project: Brooklyn
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Thomas Bouron
>
> I was testing Brooklyn HA and noticed something odd: when restarting an HA 
> Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.
> For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master 
> (not killed, stopped), slave took over as expected. Then I restarted the 
> previous master, it came back online as a slave (expected) but the node ID 
> was different. I was expecting it to keep its ID rather creating a new one.
> {code}
> {
> "planeId": "qDpn2Lau",
> "ownId": "nJagUrw8",
> "masterId": "nJagUrw8",
> "nodes": {
> "BgOjYcml": {
> "nodeId": "BgOjYcml",
> "nodeUri": null,
> "status": "TERMINATED",
> "localTimestamp": 1510052259169,
> "remoteTimestamp": 151005226
> },
> "iv3XfaEu": {
> "nodeId": "iv3XfaEu",
> "nodeUri": null,
> "status": "STANDBY",
> "localTimestamp": 1510054973636,
> "remoteTimestamp": 1510054974000
> },
> "nJagUrw8": {
> "nodeId": "nJagUrw8",
> "nodeUri": null,
> "status": "MASTER",
> "localTimestamp": 1510054975025,
> "remoteTimestamp": 1510054976000
> }
> },
> "links": {}
> }
> {code}
> On the attached screenshot, {{BgOjYcml}} is the same node as {{iv3XfaEu}}. 
> {{BgOjYcml}} was the ID of the node before restart



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BROOKLYN-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Bouron updated BROOKLYN-548:
---
Description: 
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one.

{code}
{
  "planeId": "qDpn2Lau",
  "ownId": "nJagUrw8",
  "masterId": "nJagUrw8",
  "nodes": {
"BgOjYcml": {
  "nodeId": "BgOjYcml",
  "nodeUri": null,
  "status": "TERMINATED",
  "localTimestamp": 1510052259169,
  "remoteTimestamp": 151005226
},
"iv3XfaEu": {
  "nodeId": "iv3XfaEu",
  "nodeUri": null,
  "status": "STANDBY",
  "localTimestamp": 1510054973636,
  "remoteTimestamp": 1510054974000
},
"nJagUrw8": {
  "nodeId": "nJagUrw8",
  "nodeUri": null,
  "status": "MASTER",
  "localTimestamp": 1510054975025,
  "remoteTimestamp": 1510054976000
}
  },
  "links": {}
}
{code}

On the JSON above, {{BgOjYcml}} is the same node as {{iv3XfaEu}} but 
{{BgOjYcml}} was the ID of the node before restart.

  was:
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one.

{code}
{
"planeId": "qDpn2Lau",
"ownId": "nJagUrw8",
"masterId": "nJagUrw8",
"nodes": {
"BgOjYcml": {
"nodeId": "BgOjYcml",
"nodeUri": null,
"status": "TERMINATED",
"localTimestamp": 1510052259169,
"remoteTimestamp": 151005226
},
"iv3XfaEu": {
"nodeId": "iv3XfaEu",
"nodeUri": null,
"status": "STANDBY",
"localTimestamp": 1510054973636,
"remoteTimestamp": 1510054974000
},
"nJagUrw8": {
"nodeId": "nJagUrw8",
"nodeUri": null,
"status": "MASTER",
"localTimestamp": 1510054975025,
"remoteTimestamp": 1510054976000
}
},
"links": {}
}
{code}

On the JSON above, {{BgOjYcml}} is the same node as {{iv3XfaEu}} but 
{{BgOjYcml}} was the ID of the node before restart.


> Brooklyn node ID changes on restart
> ---
>
> Key: BROOKLYN-548
> URL: https://issues.apache.org/jira/browse/BROOKLYN-548
> Project: Brooklyn
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Thomas Bouron
>
> I was testing Brooklyn HA and noticed something odd: when restarting an HA 
> Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.
> For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master 
> (not killed, stopped), slave took over as expected. Then I restarted the 
> previous master, it came back online as a slave (expected) but the node ID 
> was different. I was expecting it to keep its ID rather creating a new one.
> {code}
> {
>   "planeId": "qDpn2Lau",
>   "ownId": "nJagUrw8",
>   "masterId": "nJagUrw8",
>   "nodes": {
> "BgOjYcml": {
>   "nodeId": "BgOjYcml",
>   "nodeUri": null,
>   "status": "TERMINATED",
>   "localTimestamp": 1510052259169,
>   "remoteTimestamp": 151005226
> },
> "iv3XfaEu": {
>   "nodeId": "iv3XfaEu",
>   "nodeUri": null,
>   "status": "STANDBY",
>   "localTimestamp": 1510054973636,
>   "remoteTimestamp": 1510054974000
> },
> "nJagUrw8": {
>   "nodeId": "nJagUrw8",
>   "nodeUri": null,
>   "status": "MASTER",
>   "localTimestamp": 1510054975025,
>   "remoteTimestamp": 1510054976000
> }
>   },
>   "links": {}
> }
> {code}
> On the JSON above, {{BgOjYcml}} is the same node as {{iv3XfaEu}} but 
> {{BgOjYcml}} was the ID of the node before restart.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BROOKLYN-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Bouron updated BROOKLYN-548:
---
Description: 
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one.

{code}
{
"planeId": "qDpn2Lau",
"ownId": "nJagUrw8",
"masterId": "nJagUrw8",
"nodes": {
"BgOjYcml": {
"nodeId": "BgOjYcml",
"nodeUri": null,
"status": "TERMINATED",
"localTimestamp": 1510052259169,
"remoteTimestamp": 151005226
},
"iv3XfaEu": {
"nodeId": "iv3XfaEu",
"nodeUri": null,
"status": "STANDBY",
"localTimestamp": 1510054973636,
"remoteTimestamp": 1510054974000
},
"nJagUrw8": {
"nodeId": "nJagUrw8",
"nodeUri": null,
"status": "MASTER",
"localTimestamp": 1510054975025,
"remoteTimestamp": 1510054976000
}
},
"links": {}
}
{code}

On the JSON above, {{BgOjYcml}} is the same node as {{iv3XfaEu}} but 
{{BgOjYcml}} was the ID of the node before restart.

  was:
I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one.

{code}
{
"planeId": "qDpn2Lau",
"ownId": "nJagUrw8",
"masterId": "nJagUrw8",
"nodes": {
"BgOjYcml": {
"nodeId": "BgOjYcml",
"nodeUri": null,
"status": "TERMINATED",
"localTimestamp": 1510052259169,
"remoteTimestamp": 151005226
},
"iv3XfaEu": {
"nodeId": "iv3XfaEu",
"nodeUri": null,
"status": "STANDBY",
"localTimestamp": 1510054973636,
"remoteTimestamp": 1510054974000
},
"nJagUrw8": {
"nodeId": "nJagUrw8",
"nodeUri": null,
"status": "MASTER",
"localTimestamp": 1510054975025,
"remoteTimestamp": 1510054976000
}
},
"links": {}
}
{code}

On the attached screenshot, {{BgOjYcml}} is the same node as {{iv3XfaEu}}. 
{{BgOjYcml}} was the ID of the node before restart


> Brooklyn node ID changes on restart
> ---
>
> Key: BROOKLYN-548
> URL: https://issues.apache.org/jira/browse/BROOKLYN-548
> Project: Brooklyn
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Thomas Bouron
>
> I was testing Brooklyn HA and noticed something odd: when restarting an HA 
> Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.
> For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master 
> (not killed, stopped), slave took over as expected. Then I restarted the 
> previous master, it came back online as a slave (expected) but the node ID 
> was different. I was expecting it to keep its ID rather creating a new one.
> {code}
> {
> "planeId": "qDpn2Lau",
> "ownId": "nJagUrw8",
> "masterId": "nJagUrw8",
> "nodes": {
> "BgOjYcml": {
> "nodeId": "BgOjYcml",
> "nodeUri": null,
> "status": "TERMINATED",
> "localTimestamp": 1510052259169,
> "remoteTimestamp": 151005226
> },
> "iv3XfaEu": {
> "nodeId": "iv3XfaEu",
> "nodeUri": null,
> "status": "STANDBY",
> "localTimestamp": 1510054973636,
> "remoteTimestamp": 1510054974000
> },
> "nJagUrw8": {
> "nodeId": "nJagUrw8",
> "nodeUri": null,
> "status": "MASTER",
> "localTimestamp": 1510054975025,
> "remoteTimestamp": 1510054976000
> }
> },
> "links": {}
> }
> {code}
> On the JSON above, {{BgOjYcml}} is the same node as {{iv3XfaEu}} but 
> {{BgOjYcml}} was the ID of the node before restart.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BROOKLYN-548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Bouron updated BROOKLYN-548:
---
Attachment: screenshot-1.png

> Brooklyn node ID changes on restart
> ---
>
> Key: BROOKLYN-548
> URL: https://issues.apache.org/jira/browse/BROOKLYN-548
> Project: Brooklyn
>  Issue Type: Bug
>Affects Versions: 0.12.0
>Reporter: Thomas Bouron
> Attachments: screenshot-1.png
>
>
> I was testing Brooklyn HA and noticed something odd: when restarting an HA 
> Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.
> For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master 
> (not killed, stopped), slave took over as expected. Then I restarted the 
> previous master, it came back online as a slave (expected) but the node ID 
> was different. I was expecting it to keep its ID rather creating a new one 
> (see screenshot)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (BROOKLYN-548) Brooklyn node ID changes on restart

[Thomas Bouron (JIRA)]

Thomas Bouron created BROOKLYN-548:
--

 Summary: Brooklyn node ID changes on restart
 Key: BROOKLYN-548
 URL: https://issues.apache.org/jira/browse/BROOKLYN-548
 Project: Brooklyn
  Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Thomas Bouron


I was testing Brooklyn HA and noticed something odd: when restarting an HA 
Brooklyn node, it changes it's node ID reported by `/v1/server/ha/states`.

For instance, I have 2 Brooklyn nodes, 1 master, 1 slave. I stopped master (not 
killed, stopped), slave took over as expected. Then I restarted the previous 
master, it came back online as a slave (expected) but the node ID was 
different. I was expecting it to keep its ID rather creating a new one (see 
screenshot)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] brooklyn-dist pull request #112: Reuse bundle's catalog.bom from server-cli ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-dist/pull/112


---

[GitHub] brooklyn-server issue #882: Reuse bundle's catalog.bom from server-cli modul...

[ahgittin]

Github user ahgittin commented on the issue:

https://github.com/apache/brooklyn-server/pull/882
  
your comment must have came in at the same time as mine noting you removed 
the repetition & weren't responsible.  i'll buy the idea that tutorial 
templates live in dist.  good to merge IMO.


---

[GitHub] brooklyn-library pull request #139: Update visitors-creation-script.sql

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-library/pull/139


---

[GitHub] brooklyn-server pull request #874: avoid UpdatingMapTest non-det failure

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/brooklyn-server/pull/874


---