date:20160620

[GitHub] brooklyn-server issue #208: Serialize complex objects as JSON in shell.env.

2016-06-20 Thread grkvlt

Github user grkvlt commented on the issue:

https://github.com/apache/brooklyn-server/pull/208
  
@neykov great, this should be very useful


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-server pull request #208: Serialize complex objects as JSON in shel...

2016-06-20 Thread grkvlt

Github user grkvlt commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/208#discussion_r67778236
  
--- Diff: 
software/base/src/test/java/org/apache/brooklyn/entity/software/base/ShellEnvironmentSerializerTest.java
 ---
@@ -0,0 +1,74 @@
+/*
+ * Copyright 2016 The Apache Software Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.brooklyn.entity.software.base;
+
+import static org.testng.Assert.assertEquals;
+
+import java.util.Date;
+
+import org.apache.brooklyn.core.test.BrooklynAppUnitTestSupport;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+public class ShellEnvironmentSerializerTest extends 
BrooklynAppUnitTestSupport {
+ShellEnvironmentSerializer ser;
+
+@Override
+@BeforeMethod(alwaysRun=true)
+public void setUp() throws Exception {
+super.setUp();
+ser = new ShellEnvironmentSerializer(mgmt);
+}
+
+@Test
+public void testSerialize() {
+String str = "some string \" with ' special; characters {}[]";
+Date dt = new Date();
+String appExpected = 
"{\"type\":\"org.apache.brooklyn.api.entity.Entity\",\"id\":\"" + app.getId() + 
"\"}";
+assertSerialize(str, str);
+assertSerialize(3.14, "3.14");
+assertSerialize(3.140, "3.14");
+assertSerialize(.140, "0.14");
+assertSerialize(0.140, "0.14");
+assertSerialize(Boolean.TRUE, "true");
+assertSerialize(Boolean.FALSE, "false");
+assertSerialize(dt, Long.toString(dt.getTime()));
+assertSerialize(null, null);
+assertSerialize(ImmutableList.of(str, 3.14, 0.14));
+assertSerialize(ImmutableMap.of("string", str, "num1", 3.14, 
"num2", 0.14));
--- End diff --

Just to be sure, can we have a test with serializing a map of maps?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs issue #79: Ops guides updates for monitoring and disk space

2016-06-20 Thread rdowner

Github user rdowner commented on the issue:

https://github.com/apache/brooklyn-docs/pull/79
  
How bizarre! Branch lives on the `rdowner` repository, but I inadvertently 
opened the PR under the `richardcloudsoft` user -- and GitHub let me do it??!! 
ð 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs pull request #79: Ops guides updates for monitoring and disk s...

2016-06-20 Thread drigodwin

Github user drigodwin commented on a diff in the pull request:

https://github.com/apache/brooklyn-docs/pull/79#discussion_r67708812
  
--- Diff: guide/ops/requirements.md ---
@@ -19,6 +19,43 @@ at least 8GB RAM and 100GB disk. The disk is just for 
logs, a small amount of pe
 any binaries for custom blueprints/integrations.
 
 
+### Disk Space
+
+There are three main consumers of disk space:
+
+* **Static files**: these are the Apache Brooklyn files themselves, plus
+  binaries for custom blueprints and integrations added to the `lib` 
directory.
+* **Persisted state**: when using [Persistence](persistence/index.html) -- 
which
+  is a prerequisite for [High Availability](high-availability.html) -- 
Brooklyn
+  will save data to a store location. Items in the persisted state include
+  metadata about the Brooklyn servers, catalog items, and metadata about 
all
+  running applications and entities.
+* **Log files**: Brooklyn writes detailed log files by default to its own
+  installation directory. This can be reconfigured to change the 
destination
--- End diff --

I dont think this is true for the RPM or deb which install services - 
perhaps add something about where they put logs?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs pull request #79: Ops guides updates for monitoring and disk s...

2016-06-20 Thread richardcloudsoft

GitHub user richardcloudsoft opened a pull request:

https://github.com/apache/brooklyn-docs/pull/79

Ops guides updates for monitoring and disk space

Some additions to the User Guide "Operations" section, covering starting, 
monitoring and stopping the server, and a discussion of its disk space usage.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rdowner/brooklyn-docs 
ops-update-monitoring-diskspace

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-docs/pull/79.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #79


commit 07101c40758f286a4a084ff6828c7e16a3dc999b
Author: Richard Downer 
Date:   2016-06-20T15:08:02Z

Ops guides updates for monitoring and disk space

Some additions to the User Guide "Operations" section, covering start,
monitoring and stopping the server, and a discussion of its disk space
usage.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Machine Metrics

2016-06-20 Thread Geoff Macartney

Grand thanks Andrew.

G

Gnu PGP key - http://is.gd/uI

> On 20 Jun 2016, at 15:25, Andrew Kennedy  
> wrote:
> 
> Geoff, I have added a `StopFeedsAtMachineTask` to the lifecycle stop tasks, 
> which will take care of this. Have a look at the updated PR. However, as Svet 
> pointed out, the Feed management code in Brooklyn is definitely in need of 
> some attention and refactoring...
> 
> Andrew.
> 
> On Mon, 20 Jun 2016 at 01:41 Geoff Macartney 
>  > wrote:
> I think that looks like a good approach, +1 from me.
> 
> Will have a think at the feed stop problem.
> 
> Cheers
> Geoff
> 
> 
> 
> Gnu PGP key - http://is.gd/uI 
> 
> 
>> On 19 Jun 2016, at 15:54, Andrew Kennedy > > wrote:
>> 
>> Hi.
>> 
>> Are people happy with the initializer approach? I think the only outstanding 
>> issue is something @neykov pointed out, which is that when using 
>> `Entity#addFeed()` the feeds are not stopped when the entity is stopped. I 
>> will have another look at fixing that, but suggestions welcome.
>> 
>> Andrew.
>> 
>> On Thu, 16 Jun 2016 at 04:49 Andrew Kennedy 
>> > 
>> wrote:
>> Thanks for the feedback.
>> 
>> I decided on an `EntityInitializer` instead, because it also adds enrichers 
>> of its own. The pull request has been updated with this change now. To use 
>> it, add YAML like the following:
>> 
>> ```
>> services:
>>   - type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess
>> brooklyn.initializers:
>>   - type: org.apache.brooklyn.entity.machine.AddMachineMetrics
>> ```
>> 
>> Cheers,
>> Andrew.
>> 
>> On Thu, 16 Jun 2016 at 11:53 Geoff Macartney 
>> > > wrote:
>> I agree with Sam and Svet, + 1 to making it an enricher, if possible, or at 
>> least somehow keeping it separate from SoftwareProcess.
>> 
>> 
>> 
>> Gnu PGP key - http://is.gd/uI 
>> 
>> 
>>> On 16 Jun 2016, at 10:41, Sam Corbett >> > wrote:
>>> 
>>> I share your concern that the placement of the feed is wrong. I feel like 
>>> we would be bending Brooklyn's abstractions to fit the case rather than 
>>> working out a better abstraction. Of course, deciding what data reflects a 
>>> software process is subjective and as you point out we already break the 
>>> encapsulation in a variety of places. Was the plan not always to make 
>>> locations into entities too? This would be trivial if that were the case.
>>> 
>>> I also think we're continually making SoftwareProcess too important. Svet's 
>>> suggestion of an enricher is a good one - I want to explicitly mix the 
>>> capability in to entities, not enable it with a flag.
>>> 
>>> Sam
>>> 
>>> 
>>> On 16/06/2016 09:13, Andrew Kennedy wrote:
 Hi.

 For the project I am working on, we would like to use the CPU utilization
 as one of the metrics for scaling a cluster. The existing `MachineEntity`
 has a sensor feed that produces this data, along with uptime and memory
 usage information. The feed works on Linux VMs only, currently, as is uses
 SSH commands on the host to generate the values i.e. the `uptime` command,
 or the contents of files in `/proc/`.

 I would like to propose moving the feed to `SoftwareProcess` so that it is
 available to all entities. It would be disabled normally, set by a
 `ConfigKey` flag. This would be named "metrics.machine.retrieve"
 to correspond to "metrics.usage.retrieve" which enables sensors in feeds
 that return application or process specific information. The
 `MachineEntity` would obviously have the default value set to "true", to
 maintain current behaviour.

 The only issue with this change is that the placement of the sensor feed
 feels slightly wrong. These are returning data about the _machine_ but the
 entity represents a _process_ on that machine, and there may in fact be
 multiple entities sharing a single machine, via `SameServerEntity`. The
 `MachineEntity` is used to represent a VM without any applications running
 on it, and would not normally be part of a blueprint, so these sensors are
 not normally accessible. There is some precedent for placing machine data
 on an entity, such as the `HOSTNAME` sensor, so I think the break in
 encapsulation is quite small.

 The PR containing the change is here:

 - https://github.com/apache/brooklyn-server/pull/204 

 I'd appreciate any comments on whether this is a useful

Re: Machine Metrics

2016-06-20 Thread Andrew Kennedy

Geoff, I have added a `StopFeedsAtMachineTask` to the lifecycle stop tasks,
which will take care of this. Have a look at the updated PR. However, as
Svet pointed out, the Feed management code in Brooklyn is definitely in
need of some attention and refactoring...

Andrew.

On Mon, 20 Jun 2016 at 01:41 Geoff Macartney <
geoff.macart...@cloudsoftcorp.com> wrote:

> I think that looks like a good approach, +1 from me.
>
> Will have a think at the feed stop problem.
>
> Cheers
> Geoff
>
>
> 
> Gnu PGP key - http://is.gd/uI
>
>
> On 19 Jun 2016, at 15:54, Andrew Kennedy 
> wrote:
>
> Hi.
>
> Are people happy with the initializer approach? I think the only
> outstanding issue is something @neykov pointed out, which is that when
> using `Entity#addFeed()` the feeds are not stopped when the entity is
> stopped. I will have another look at fixing that, but suggestions welcome.
>
> Andrew.
>
> On Thu, 16 Jun 2016 at 04:49 Andrew Kennedy <
> andrew.kenn...@cloudsoftcorp.com> wrote:
>
>> Thanks for the feedback.
>>
>> I decided on an `EntityInitializer` instead, because it also adds
>> enrichers of its own. The pull request has been updated with this change
>> now. To use it, add YAML like the following:
>>
>> ```
>> services:
>>   - type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess
>> brooklyn.initializers:
>>   - type: org.apache.brooklyn.entity.machine.AddMachineMetrics
>> ```
>>
>> Cheers,
>> Andrew.
>>
>> On Thu, 16 Jun 2016 at 11:53 Geoff Macartney <
>> geoff.macart...@cloudsoftcorp.com> wrote:
>>
>>> I agree with Sam and Svet, + 1 to making it an enricher, if possible, or
>>> at least somehow keeping it separate from SoftwareProcess.
>>>
>>>
>>> 
>>> Gnu PGP key - http://is.gd/uI
>>>
>>>
>>> On 16 Jun 2016, at 10:41, Sam Corbett 
>>> wrote:
>>>
>>> I share your concern that the placement of the feed is wrong. I feel
>>> like we would be bending Brooklyn's abstractions to fit the case rather
>>> than working out a better abstraction. Of course, deciding what data
>>> reflects a software process is subjective and as you point out we already
>>> break the encapsulation in a variety of places. Was the plan not always to
>>> make locations into entities too? This would be trivial if that were the
>>> case.
>>>
>>> I also think we're continually making SoftwareProcess too important.
>>> Svet's suggestion of an enricher is a good one - I want to explicitly mix
>>> the capability in to entities, not enable it with a flag.
>>>
>>> Sam
>>>
>>>
>>> On 16/06/2016 09:13, Andrew Kennedy wrote:
>>>
>>> Hi.
>>>
>>> For the project I am working on, we would like to use the CPU utilization
>>> as one of the metrics for scaling a cluster. The existing `MachineEntity`
>>> has a sensor feed that produces this data, along with uptime and memory
>>> usage information. The feed works on Linux VMs only, currently, as is
>>> uses
>>> SSH commands on the host to generate the values i.e. the `uptime`
>>> command,
>>> or the contents of files in `/proc/`.
>>>
>>> I would like to propose moving the feed to `SoftwareProcess` so that it
>>> is
>>> available to all entities. It would be disabled normally, set by a
>>> `ConfigKey` flag. This would be named "metrics.machine.retrieve"
>>> to correspond to "metrics.usage.retrieve" which enables sensors in feeds
>>> that return application or process specific information. The
>>> `MachineEntity` would obviously have the default value set to "true", to
>>> maintain current behaviour.
>>>
>>> The only issue with this change is that the placement of the sensor feed
>>> feels slightly wrong. These are returning data about the _machine_ but
>>> the
>>> entity represents a _process_ on that machine, and there may in fact be
>>> multiple entities sharing a single machine, via `SameServerEntity`. The
>>> `MachineEntity` is used to represent a VM without any applications
>>> running
>>> on it, and would not normally be part of a blueprint, so these sensors
>>> are
>>> not normally accessible. There is some precedent for placing machine data
>>> on an entity, such as the `HOSTNAME` sensor, so I think the break in
>>> encapsulation is quite small.
>>>
>>> The PR containing the change is here:
>>>
>>> - https://github.com/apache/brooklyn-server/pull/204
>>>
>>> I'd appreciate any comments on whether this is a useful change, as well
>>> as
>>> a review of the pull request...
>>>
>>> Thanks,
>>> Andrew.
>>>
>>>
>>>
>>> --
>>
>> Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft
>>
> --
>
> Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft
>
>
> --

Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft

[GitHub] brooklyn-server issue #204: Move machine metrics to SoftwareProcess entity

2016-06-20 Thread grkvlt

Github user grkvlt commented on the issue:

https://github.com/apache/brooklyn-server/pull/204
  
@neykov good feedback; will address your comments


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [PROPOSAL] catalog YAML format consistency

2016-06-20 Thread Svetoslav Neykov

+1 for the proposal.

I find the current item-items functionality logical. "item" is used in leaf 
items, "items" is used in non-leaf items. Forcing a non-leaf root just so we 
always have "items" is overhead.

Svet.


> On 20.06.2016 г., at 15:47, Aled Sage  wrote:
> 
> Hi all,
> 
> The YAML format for adding catalog items accepts several different ways of 
> defining them. This has led to our examples being inconsistent, our code more 
> complicated, and potential confusion for users when they see different things 
> that turn out to mean the same.
> 
> I think we should standardise on one approach, and deprecate the other ways.
> 
> ---
> 
> _*Current Code*_
> 
> An example .bom file is shown below:
> 
>brooklyn.catalog:
>  items:
>  - id: entity1
>version: "1.0.0"
>itemType: entity
>item:
>  type: org.apache.brooklyn.entity.machine.MachineEntity
> 
> Variants:
> 
> * If defining just a single item in the .bom file, you can optionally
>   miss out the "items".
> * You can miss out the "itemType" - it will guess at it by trying to
>   treat it as an entity, a template, a location or a policy. The
>   default is "entity".
> * You can include "services:" for entity or template types, or you can
>   miss it out if there is just one entity in the item.
> * Similar to "services:", you can include "brooklyn.policies:" or
>   "brooklyn.locations:".
>   If itemType is missing, this helps to infer the type. If it does not
>   agree with itemType, then we add it as the item type and it will
>   fail later.
> * You can define the item metadata at any level - it could be directly
>   under "brooklyn.catalog" (in which case it applies to all items), or
>   under a specific item (in which case it overrides any more general
>   metadata).
> 
> 
> An example of a .bom for a single item is shown below:
> 
> brooklyn.catalog:
>  id: entity2
>  version: "1.0.0"
>  itemType: entity
>  item:
>type: org.apache.brooklyn.entity.machine.MachineEntity
> 
> 
> ---
> 
> _*Proposal*_
> 
> I suggest we have the following stricter rules. Anything else is deprecated, 
> logging a warning.
> 
> * Always include "itemType".
> * For entity, policy and location: do not include "services:",
>   "brooklyn.policies:" or "brooklyn.locations:" - i.e. it will expect
>   exactly one type in the item.
> * For template, always expect "services:" (even if there is just one
>   thing). This is consistent with the YAML required when deploying an
>   application.
> * Always include "items", even if there is just one item in it.
>   (reasoning: we do not support "service" versus "services", so why
>   support "item").
> 
> 
> We should change the following (breaking backwards compatibility, because 
> it's really a bug):
> 
> * If the itemType differs from the actual type of the item, then fail.
> 
> Aled
> 
> p.s. I'm in two minds about "item" versus "items": it is simpler with the 
> single item, and having "item" underneath "items" means it's not quite like 
> the "services" analogy.
> 
>

[Proposal] Passing complex objects in shell.env

2016-06-20 Thread Svetoslav Neykov

Working on YAML-only blueprints I've hit a restriction where the only possible 
configuration type that users can pass to my blueprint is a string. Anything 
more complex gets its "toString()" method called and thus is impossible to 
reconstruct cleanly on the bash side.

I suggest that we serialize complex objects to JSON before passing them as 
environment variables to scripts. This will allow for blueprints like the 
following:

brooklyn.catalog:
  id: mapping-entity
  item:
type: VanillaSoftwareProcess
env:
  MAPPINGS: $brooklyn:config("mappings")

---

services:
- type:  mapping-entity
  brooklyn.config:
mappings:
  key1: val1
  key2: val2


---

On the bash side this results in an environment variable 
MAPPINGS={"key1":"val1","key2":"val2"} which is easily manipulated with jq 
binary.


I've implemented the change in 
https://github.com/apache/brooklyn-server/pull/208 
.

Svet.

[jira] [Created] (BROOKLYN-304) Config/Sensors JSON Serializer fails with "java.lang.OutOfMemoryError: GC overhead limit exceeded"

2016-06-20 Thread Svetoslav Neykov (JIRA)

Svetoslav Neykov created BROOKLYN-304:
-

 Summary: Config/Sensors JSON Serializer fails with 
"java.lang.OutOfMemoryError: GC overhead limit exceeded"
 Key: BROOKLYN-304
 URL: https://issues.apache.org/jira/browse/BROOKLYN-304
 Project: Brooklyn
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Svetoslav Neykov


Trying to serialize for example {{getClass().getClassLoader()}} fails with 
{{java.lang.OutOfMemoryError: GC overhead limit exceeded}}.

For more details see test {{ShellEnvironmentSerializerTest.testSerialize()}} 
(PR brooklyn-server#208}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] brooklyn-server pull request #208: Serialize complex objects as JSON in shel...

2016-06-20 Thread neykov

GitHub user neykov opened a pull request:

https://github.com/apache/brooklyn-server/pull/208

Serialize complex objects as JSON in shell.env.

Serialize complex objects to JSON instead of calling toString() when 
building the shell environment variables for `SoftwareProcess`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/neykov/brooklyn-server json-env

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-server/pull/208.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #208


commit 05283c8f1502b55060b011ad56b1cc36c9d13e71
Author: Svetoslav Neykov 
Date:   2016-06-17T15:04:11Z

Fix instantiating classes loaded from the bootstrap loader

getClassLoader() returns null for those, so use an explicit class loader. 
It's not used anyway in the code path, but needs to go through a checkNotNull.

commit 8aa2b75b6f1ece763d1a3bf668950fa0861f4b69
Author: Svetoslav Neykov 
Date:   2016-06-17T15:55:45Z

Move jackson serialization configuration to core

Needs to be used by both rest-resource & software-base.

commit b60a029aa41c0debd7cd4f5c52ac7873e2e9c10f
Author: Svetoslav Neykov 
Date:   2016-06-20T12:44:48Z

Convert complex objects to JSON before passing them as environment variables




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [PROPOSAL] catalog YAML format consistency

2016-06-20 Thread Alex Heneveld



I like this direction but I think we should stage as follows:

NOW - agree short-term preferred syntax for catalog items (but nothing 
deprecated, not major change)


SOON - define new improved and more powerful syntax, similar to 
preferred -- but allowing arbitrary types, interfaces, mixins


My thinking is we need a bigger overhaul (I've been mulling this over) 
so let's not do too much churn until then.


--a


On 20/06/2016 13:47, Aled Sage wrote:

Hi all,

The YAML format for adding catalog items accepts several different 
ways of defining them. This has led to our examples being 
inconsistent, our code more complicated, and potential confusion for 
users when they see different things that turn out to mean the same.


I think we should standardise on one approach, and deprecate the other 
ways.


---

_*Current Code*_

An example .bom file is shown below:

brooklyn.catalog:
  items:
  - id: entity1
version: "1.0.0"
itemType: entity
item:
  type: org.apache.brooklyn.entity.machine.MachineEntity

Variants:

 * If defining just a single item in the .bom file, you can optionally
   miss out the "items".
 * You can miss out the "itemType" - it will guess at it by trying to
   treat it as an entity, a template, a location or a policy. The
   default is "entity".
 * You can include "services:" for entity or template types, or you can
   miss it out if there is just one entity in the item.
 * Similar to "services:", you can include "brooklyn.policies:" or
   "brooklyn.locations:".
   If itemType is missing, this helps to infer the type. If it does not
   agree with itemType, then we add it as the item type and it will
   fail later.
 * You can define the item metadata at any level - it could be directly
   under "brooklyn.catalog" (in which case it applies to all items), or
   under a specific item (in which case it overrides any more general
   metadata).


An example of a .bom for a single item is shown below:

brooklyn.catalog:
  id: entity2
  version: "1.0.0"
  itemType: entity
  item:
type: org.apache.brooklyn.entity.machine.MachineEntity


---

_*Proposal*_

I suggest we have the following stricter rules. Anything else is 
deprecated, logging a warning.


 * Always include "itemType".
 * For entity, policy and location: do not include "services:",
   "brooklyn.policies:" or "brooklyn.locations:" - i.e. it will expect
   exactly one type in the item.
 * For template, always expect "services:" (even if there is just one
   thing). This is consistent with the YAML required when deploying an
   application.
 * Always include "items", even if there is just one item in it.
   (reasoning: we do not support "service" versus "services", so why
   support "item").


We should change the following (breaking backwards compatibility, 
because it's really a bug):


 * If the itemType differs from the actual type of the item, then fail.

Aled

p.s. I'm in two minds about "item" versus "items": it is simpler with 
the single item, and having "item" underneath "items" means it's not 
quite like the "services" analogy.

[PROPOSAL] catalog YAML format consistency

2016-06-20 Thread Aled Sage


Hi all,

The YAML format for adding catalog items accepts several different ways 
of defining them. This has led to our examples being inconsistent, our 
code more complicated, and potential confusion for users when they see 
different things that turn out to mean the same.


I think we should standardise on one approach, and deprecate the other ways.

---

_*Current Code*_

An example .bom file is shown below:

brooklyn.catalog:
  items:
  - id: entity1
version: "1.0.0"
itemType: entity
item:
  type: org.apache.brooklyn.entity.machine.MachineEntity

Variants:

 * If defining just a single item in the .bom file, you can optionally
   miss out the "items".
 * You can miss out the "itemType" - it will guess at it by trying to
   treat it as an entity, a template, a location or a policy. The
   default is "entity".
 * You can include "services:" for entity or template types, or you can
   miss it out if there is just one entity in the item.
 * Similar to "services:", you can include "brooklyn.policies:" or
   "brooklyn.locations:".
   If itemType is missing, this helps to infer the type. If it does not
   agree with itemType, then we add it as the item type and it will
   fail later.
 * You can define the item metadata at any level - it could be directly
   under "brooklyn.catalog" (in which case it applies to all items), or
   under a specific item (in which case it overrides any more general
   metadata).


An example of a .bom for a single item is shown below:

brooklyn.catalog:
  id: entity2
  version: "1.0.0"
  itemType: entity
  item:
type: org.apache.brooklyn.entity.machine.MachineEntity


---

_*Proposal*_

I suggest we have the following stricter rules. Anything else is 
deprecated, logging a warning.


 * Always include "itemType".
 * For entity, policy and location: do not include "services:",
   "brooklyn.policies:" or "brooklyn.locations:" - i.e. it will expect
   exactly one type in the item.
 * For template, always expect "services:" (even if there is just one
   thing). This is consistent with the YAML required when deploying an
   application.
 * Always include "items", even if there is just one item in it.
   (reasoning: we do not support "service" versus "services", so why
   support "item").


We should change the following (breaking backwards compatibility, 
because it's really a bug):


 * If the itemType differs from the actual type of the item, then fail.

Aled

p.s. I'm in two minds about "item" versus "items": it is simpler with 
the single item, and having "item" underneath "items" means it's not 
quite like the "services" analogy.

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

2016-06-20 Thread neykov

Github user neykov commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/204#discussion_r67669549
  
--- Diff: 
software/base/src/main/java/org/apache/brooklyn/entity/machine/MachineEntityImpl.java
 ---
@@ -61,82 +51,17 @@ public void init() {
 @Override
 protected void connectSensors() {
 super.connectSensors();
-
-// Sensors linux-specific
-if (!getMachine().getMachineDetails().getOsDetails().isLinux()) 
return;
-
-sensorFeed = SshFeed.builder()
-.entity(this)
-.period(Duration.THIRTY_SECONDS)
-.poll(new SshPollConfig(UPTIME)
-.command("cat /proc/uptime")
-
.onFailureOrException(Functions.constant(null))
-.onSuccess(new Function() {
-@Override
-public Duration apply(SshPollValue input) {
-return Duration.seconds( Double.valueOf( 
Strings.getFirstWord(input.getStdout()) ) );
-}
-}))
-.poll(new SshPollConfig(LOAD_AVERAGE)
-.command("uptime")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-String loadAverage = 
Strings.getFirstWordAfter(input.getStdout(), "load average:").replace(",", "");
-return Double.valueOf(loadAverage);
-}
-}))
-.poll(new SshPollConfig(CPU_USAGE)
-.command("cat /proc/stat")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-List cpuData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-Integer system = 
Integer.parseInt(cpuData.get(1));
-Integer user = 
Integer.parseInt(cpuData.get(3));
-Integer idle = 
Integer.parseInt(cpuData.get(4));
-return (double) (system + user) / (double) 
(system + user + idle);
-}
-}))
-.poll(new SshPollConfig(USED_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(2));
-}
-}))
-.poll(new SshPollConfig(FREE_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(3));
-}
-}))
-.poll(new SshPollConfig(TOTAL_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(1));
-}
-}))
-.build();
-
+Maybe location =

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

2016-06-20 Thread neykov

Github user neykov commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/204#discussion_r67669494
  
--- Diff: 
software/base/src/main/java/org/apache/brooklyn/entity/machine/MachineEntityImpl.java
 ---
@@ -61,82 +51,17 @@ public void init() {
 @Override
 protected void connectSensors() {
 super.connectSensors();
-
-// Sensors linux-specific
-if (!getMachine().getMachineDetails().getOsDetails().isLinux()) 
return;
-
-sensorFeed = SshFeed.builder()
-.entity(this)
-.period(Duration.THIRTY_SECONDS)
-.poll(new SshPollConfig(UPTIME)
-.command("cat /proc/uptime")
-
.onFailureOrException(Functions.constant(null))
-.onSuccess(new Function() {
-@Override
-public Duration apply(SshPollValue input) {
-return Duration.seconds( Double.valueOf( 
Strings.getFirstWord(input.getStdout()) ) );
-}
-}))
-.poll(new SshPollConfig(LOAD_AVERAGE)
-.command("uptime")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-String loadAverage = 
Strings.getFirstWordAfter(input.getStdout(), "load average:").replace(",", "");
-return Double.valueOf(loadAverage);
-}
-}))
-.poll(new SshPollConfig(CPU_USAGE)
-.command("cat /proc/stat")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-List cpuData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-Integer system = 
Integer.parseInt(cpuData.get(1));
-Integer user = 
Integer.parseInt(cpuData.get(3));
-Integer idle = 
Integer.parseInt(cpuData.get(4));
-return (double) (system + user) / (double) 
(system + user + idle);
-}
-}))
-.poll(new SshPollConfig(USED_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(2));
-}
-}))
-.poll(new SshPollConfig(FREE_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(3));
-}
-}))
-.poll(new SshPollConfig(TOTAL_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(1));
-}
-}))
-.build();
-
+Maybe location =

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

2016-06-20 Thread neykov

Github user neykov commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/204#discussion_r67668240
  
--- Diff: 
software/base/src/main/java/org/apache/brooklyn/entity/machine/MachineEntityImpl.java
 ---
@@ -61,82 +51,17 @@ public void init() {
 @Override
 protected void connectSensors() {
 super.connectSensors();
-
-// Sensors linux-specific
-if (!getMachine().getMachineDetails().getOsDetails().isLinux()) 
return;
-
-sensorFeed = SshFeed.builder()
-.entity(this)
-.period(Duration.THIRTY_SECONDS)
-.poll(new SshPollConfig(UPTIME)
-.command("cat /proc/uptime")
-
.onFailureOrException(Functions.constant(null))
-.onSuccess(new Function() {
-@Override
-public Duration apply(SshPollValue input) {
-return Duration.seconds( Double.valueOf( 
Strings.getFirstWord(input.getStdout()) ) );
-}
-}))
-.poll(new SshPollConfig(LOAD_AVERAGE)
-.command("uptime")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-String loadAverage = 
Strings.getFirstWordAfter(input.getStdout(), "load average:").replace(",", "");
-return Double.valueOf(loadAverage);
-}
-}))
-.poll(new SshPollConfig(CPU_USAGE)
-.command("cat /proc/stat")
-.onFailureOrException(Functions.constant(-1d))
-.onSuccess(new Function() {
-@Override
-public Double apply(SshPollValue input) {
-List cpuData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-Integer system = 
Integer.parseInt(cpuData.get(1));
-Integer user = 
Integer.parseInt(cpuData.get(3));
-Integer idle = 
Integer.parseInt(cpuData.get(4));
-return (double) (system + user) / (double) 
(system + user + idle);
-}
-}))
-.poll(new SshPollConfig(USED_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(2));
-}
-}))
-.poll(new SshPollConfig(FREE_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(3));
-}
-}))
-.poll(new SshPollConfig(TOTAL_MEMORY)
-.command("free | grep Mem:")
-.onFailureOrException(Functions.constant(-1L))
-.onSuccess(new Function() {
-@Override
-public Long apply(SshPollValue input) {
-List memoryData = Splitter.on(" 
").omitEmptyStrings().splitToList(Strings.getFirstLine(input.getStdout()));
-return Long.parseLong(memoryData.get(1));
-}
-}))
-.build();
-
+Maybe location =

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

2016-06-20 Thread neykov

Github user neykov commented on a diff in the pull request:

https://github.com/apache/brooklyn-server/pull/204#discussion_r67667163
  
--- Diff: 
core/src/main/java/org/apache/brooklyn/core/entity/AbstractEntity.java ---
@@ -1737,7 +1738,6 @@ public boolean removeAllPolicies() {
  * {@link BasicEnricherSupport}.
  */
 @Beta
-// TODO revert to private when config() is reverted to return 
SensorSupportInternal
--- End diff --

I believe this still needs to be addressed, though the comment has 
copy error referring to sensors.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

2016-06-20 Thread drigodwin

Github user drigodwin commented on a diff in the pull request:

https://github.com/apache/brooklyn-docs/pull/78#discussion_r67667031
  
--- Diff: guide/ops/high-availability-supplemental.md ---
@@ -0,0 +1,97 @@
+---
+title: High Availability (Supplemental)
+layout: website-normal
+---
+
+This document supplements the High Availability documentation available 
[here](http://brooklyn.apache.org/v/latest/ops/high-availability.html)
+and provides an example of how to configure a pair of Apache Brooklyn 
servers to run in master-standby mode with a shared NFS datastore
+
+### Prerequisites
+- Two VMs (or physical machines) have been provisioned
+- NFS or another suitable file system has been configured and is available 
to both VMs
+- An NFS folder has been mounted on both VMs at `/mnt/amp-persistence` and 
both machines can write to the folder
+
+### Launching
+To start, download the latest Apache Brooklyn release on both VMs, extract 
it and navigate to the root folder:
+
+{% highlight bash %}
+wget -O apache-brooklyn-0.9.0-bin.tar.gz 
"https://www.apache.org/dyn/closer.lua?action=download=brooklyn/apache-brooklyn-0.9.0/apache-brooklyn-0.9.0-bin.tar.gz;
--- End diff --

You should use apache-brooklyn_{{site.brooklyn-version}} instead of naming 
a specific version here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

2016-06-20 Thread drigodwin

Github user drigodwin commented on a diff in the pull request:

https://github.com/apache/brooklyn-docs/pull/78#discussion_r67666709
  
--- Diff: guide/ops/high-availability-supplemental.md ---
@@ -0,0 +1,97 @@
+---
+title: High Availability (Supplemental)
+layout: website-normal
+---
+
+This document supplements the High Availability documentation available 
[here](http://brooklyn.apache.org/v/latest/ops/high-availability.html)
--- End diff --

Would this not be better as a multiple section sub-section similar to 
locations?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

2016-06-20 Thread nakomis

GitHub user nakomis opened a pull request:

https://github.com/apache/brooklyn-docs/pull/78

Adds details of setting up a HA master-slave pair



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nakomis/brooklyn-docs ha-supplemental

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/brooklyn-docs/pull/78.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #78


commit a8f837a939a28a495a26403c78a0af624f107082
Author: Martin Harris 
Date:   2016-06-20T09:47:05Z

Adds details of setting up a HA master-slave pair




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] brooklyn-server issue #208: Serialize complex objects as JSON in shell.env.

[GitHub] brooklyn-server pull request #208: Serialize complex objects as JSON in shel...

[GitHub] brooklyn-docs issue #79: Ops guides updates for monitoring and disk space

[GitHub] brooklyn-docs pull request #79: Ops guides updates for monitoring and disk s...

[GitHub] brooklyn-docs pull request #79: Ops guides updates for monitoring and disk s...

Re: Machine Metrics

Re: Machine Metrics

[GitHub] brooklyn-server issue #204: Move machine metrics to SoftwareProcess entity

Re: [PROPOSAL] catalog YAML format consistency

[Proposal] Passing complex objects in shell.env

[jira] [Created] (BROOKLYN-304) Config/Sensors JSON Serializer fails with "java.lang.OutOfMemoryError: GC overhead limit exceeded"

[GitHub] brooklyn-server pull request #208: Serialize complex objects as JSON in shel...

Re: [PROPOSAL] catalog YAML format consistency

[PROPOSAL] catalog YAML format consistency

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

[GitHub] brooklyn-server pull request #204: Move machine metrics to SoftwareProcess e...

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

[GitHub] brooklyn-docs pull request #78: Adds details of setting up a HA master-slave...

21 matches

Site Navigation

Mail list logo

Footer information