[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628473#comment-15628473
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
One minor comment in AbstractBrooklynRestResource.java but overall LGTM 👍 .
I had a look at the code @neykov referred to and I suppose there is more to do
as noted in the comment in `getNonBlockingResolvingStructuredKey` but I agree
that that should not be part of this change.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628450#comment-15628450
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user asfgit closed the pull request at:
https://github.com/apache/brooklyn-server/pull/390
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628448#comment-15628448
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
LGTM, merging.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15628438#comment-15628438
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r86106651
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -340,9 +360,9 @@ public DslConfigSupplier(DslComponent component, String
keyName) {
public final Maybe getImmediately() {
Maybe targetEntityMaybe = component.getImmediately();
if (targetEntityMaybe.isAbsent()) return Maybe.absent("Target
entity not available");
-Entity targetEntity = targetEntityMaybe.get();
+EntityInternal targetEntity = (EntityInternal)
targetEntityMaybe.get();
-return
Maybe.of(targetEntity.getConfig(ConfigKeys.newConfigKey(Object.class,
keyName)));
+return
targetEntity.config().getNonBlocking(ConfigKeys.newConfigKey(Object.class,
keyName));
--- End diff --
Yes, I'd updated `AbstractConfigurationSupportInternal` line 142 to use
`.immediately(true)`, so I hope that covers the case of
`getNonBlockingResolvingSimple`. I didn't want to touch the "structure key"
code. We should probably change that at some point to go through ValueResolver,
even if that still ends up scheduling it in a task.
I therefore think this comment has been addressed. Am I missing something?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627940#comment-15627940
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r86080008
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/BrooklynDslCommon.java
---
@@ -525,15 +583,20 @@ public DslExternal(String providerName, String key) {
}
@Override
+public final Maybe getImmediately() {
+ManagementContextInternal managementContext =
DslExternal.managementContext();
+return
Maybe.of(managementContext.getExternalConfigProviderRegistry().getConfig(providerName,
key));
--- End diff --
Agree - it's fine calling this in `getImmediately`.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626647#comment-15626647
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
@neykov @geomacy latest comments addresses in the last two commits. Can you
take another look please?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626105#comment-15626105
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85981351
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/BrooklynDslCommon.java
---
@@ -525,15 +583,20 @@ public DslExternal(String providerName, String key) {
}
@Override
+public final Maybe getImmediately() {
+ManagementContextInternal managementContext =
DslExternal.managementContext();
+return
Maybe.of(managementContext.getExternalConfigProviderRegistry().getConfig(providerName,
key));
--- End diff --
@neykov This is an interesting one! The
`managementContext.getExternalConfigProviderRegistry().getConfig(...)` is
calling a different getConfig whose javadoc simply says:
```
/**
* Searches the named {@link ExternalConfigSupplier} for the config
value associated with the specified key.
* Quietly returns null if no config exists for the
specified key.
* Throws {@link IllegalArgumentException} if no {@link
ExternalConfigSupplier} exists for the passed name.
*/
```
The existing implementation delegates to `ExternalConfigSupplier.get(key)`.
There is no guidance on that whether the call could be blocking, or should
return reasonably promptly.
I'd assume it would call out to things like Vault, and would return the
value that currently exists or null. Such calls will not be super-fast, but
should not be blocked waiting for other entities either.
I therefore think this is good enough, to just call getConfig without
wrapping it in another task for timeout-purposes.
If it's not good enough, then we should really change `DslExternal` back so
that it doesn't even try to support `getImmediately()`. Or we could try to
change the interface of `ExternalConfigSupplierRegistry` etc.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626088#comment-15626088
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85979870
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -303,6 +330,12 @@ public DslConfigSupplier(DslComponent component,
String keyName) {
}
@Override
+public final Maybe getImmediately() {
+Entity targetEntity = component.get();
+return
Maybe.of(targetEntity.getConfig(ConfigKeys.newConfigKey(Object.class,
keyName)));
--- End diff --
Great spot! I'm writing a bunch more tests for this.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625650#comment-15625650
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
Couple more comments but otherwise looks good along with Svet's comments.
👍
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625487#comment-15625487
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85919560
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -224,6 +235,11 @@ public EntityId(DslComponent component) {
}
@Override
+public Maybe getImmediately() {
+return Maybe.of(component.get().getId());
--- End diff --
Should be `component.getImmediately().get().getId()`
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625488#comment-15625488
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85920203
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -303,6 +330,12 @@ public DslConfigSupplier(DslComponent component,
String keyName) {
}
@Override
+public final Maybe getImmediately() {
+Entity targetEntity = component.get();
+return
Maybe.of(targetEntity.getConfig(ConfigKeys.newConfigKey(Object.class,
keyName)));
--- End diff --
`getConfig` will block indefinitely, resolving task values.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625499#comment-15625499
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
There's pretty big overlap between the `getImmediate` & `newTask`
implementations - would be nice to reuse as much as possible, but I'm fine
merging those as is.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625491#comment-15625491
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85920090
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -303,6 +330,12 @@ public DslConfigSupplier(DslComponent component,
String keyName) {
}
@Override
+public final Maybe getImmediately() {
+Entity targetEntity = component.get();
--- End diff --
+1
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625489#comment-15625489
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85929615
--- Diff:
core/src/test/java/org/apache/brooklyn/util/core/task/ValueResolverTest.java ---
@@ -69,46 +59,65 @@ public void testTimeoutBig() {
Assert.assertEquals(result.get(), "foo");
}
-public void testNoExecutionContextOnCompleted() {
+public void testCompletedTaskReturnsResultImmediately() {
+// Call ValueResolver.getMaybe() from this thread, which has no
execution context.
+// However, the task has already been submitted and we have waited
for it to complete.
+// Therefore the ValueResolver can simply check for task.isDone()
and return its result immediately.
Task t = newSleepTask(Duration.ZERO, "foo");
app.getExecutionContext().submit(t).getUnchecked();
Maybe result =
Tasks.resolving(t).as(String.class).timeout(Duration.ZERO).getMaybe();
Assert.assertEquals(result.get(), "foo");
}
-public static Throwable assertThrowsOnMaybe(ValueResolver result) {
-try {
-result = result.clone();
-result.getMaybe();
-Assert.fail("should have thrown");
-return null;
-} catch (Exception e) { return e; }
+public void testUnsubmittedTaskWhenNoExecutionContextFails() {
+// ValueResolver.getMaybe() is called with no execution context.
Therefore it will not execute the task.
+Task t = newSleepTask(Duration.ZERO, "foo");
+Maybe result =
Tasks.resolving(t).as(String.class).timeout(Duration.ZERO).getMaybe();
+
+Assert.assertTrue(result.isAbsent(), "result="+result);
+Exception exception = ((Maybe.Absent)result).getException();
--- End diff --
Can use `Maybe.getException(result)`.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625490#comment-15625490
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85919317
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -88,6 +90,15 @@ public DslComponent(DslComponent scopeComponent, Scope
scope, String componentId
// ---
@Override
+public final Maybe getImmediately() {
+try {
+return Maybe.of(new EntityInScopeFinder(scopeComponent, scope,
componentId).call());
--- End diff --
[This](https://github.com/apache/brooklyn-server/pull/390/files#diff-e301b891976cba2942279245254a62d1R124)
will cause a thread to be created to resolve chained `scopeComponent`s. Should
call `getImmediately` in there as well.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625492#comment-15625492
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85924125
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/BrooklynDslCommon.java
---
@@ -525,15 +583,20 @@ public DslExternal(String providerName, String key) {
}
@Override
+public final Maybe getImmediately() {
+ManagementContextInternal managementContext =
DslExternal.managementContext();
+return
Maybe.of(managementContext.getExternalConfigProviderRegistry().getConfig(providerName,
key));
--- End diff --
This is not reusable since `getConfig` will block indefinitely, resolving
the value.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625381#comment-15625381
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
@aledsage I'm currently reviewing the PR, please hold on with the merge.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625376#comment-15625376
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on the issue:
https://github.com/apache/brooklyn-server/pull/390
Thanks @geomacy for the great comments - I've addressed those in an
additional commit, and rebased against master.
Once jenkins confirms the tests pass, I'll merge. If you do get a chance to
review the changes in the last commit before that, it would be great though.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625140#comment-15625140
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85909430
--- Diff:
core/src/test/java/org/apache/brooklyn/util/core/task/ValueResolverTest.java ---
@@ -41,6 +49,125 @@ public void setUp() throws Exception {
super.setUp();
}
+public void testTimeoutZero() {
+Maybe result =
Tasks.resolving(newSleepTask(Duration.TEN_SECONDS,
"foo")).as(String.class).context(app).timeout(Duration.ZERO).getMaybe();
+Assert.assertFalse(result.isPresent());
+}
+
+public void testTimeoutBig() {
+Maybe result = Tasks.resolving(newSleepTask(Duration.ZERO,
"foo")).as(String.class).context(app).timeout(Duration.TEN_SECONDS).getMaybe();
+Assert.assertEquals(result.get(), "foo");
+}
+
+public void testNoExecutionContextOnCompleted() {
--- End diff --
I'll add a comment for what I think it means (I just moved this test,
rather than adding it):
```
public void testCompletedTaskReturnsResultImmediately() {
// Call ValueResolver.getMaybe() from this thread, which has no
execution context.
// However, the task has already been submitted and we have waited
for it to complete.
// Therefore the ValueResolver can simply check for task.isDone()
and return its result immediately.
```
I'll also add some more tests for this area of the code.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625087#comment-15625087
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85906370
--- Diff:
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/brooklyn/dsl/DslTest.java
---
@@ -118,39 +160,64 @@ public void runConcurrentWorker(Supplier
taskSupplier) {
}
private static class DslTestWorker implements Runnable {
-protected TestApplication parent;
-protected BrooklynDslDeferredSupplier dsl;
+protected final TestApplication parent;
+protected final BrooklynDslDeferredSupplier dsl;
+protected final Class type;
protected EntitySpec childSpec =
EntitySpec.create(TestEntity.class);
-protected Class type;
-
+protected int resolverIterations = MANY_RESOLVER_ITERATIONS;
+protected boolean eventually = false;
+private boolean wrapInTaskForImmediately = true;
+
public DslTestWorker(TestApplication parent,
BrooklynDslDeferredSupplier dsl, Class type) {
-this.parent = parent;
-this.dsl = dsl;
-this.type = type;
+this.parent = checkNotNull(parent, "parent");
+this.dsl = checkNotNull(dsl, "dsl");
+this.type = checkNotNull(type, "type");
}
+public DslTestWorker resolverIterations(int val) {
+resolverIterations = val;
+return this;
+}
+
+public DslTestWorker eventually(boolean val) {
+eventually = val;
+return this;
+}
+
+public DslTestWorker wrapInTaskForImmediately(boolean val) {
+wrapInTaskForImmediately = val;
+return this;
+}
+
@Override
public void run() {
-TestEntity entity = parent.createAndManageChild(childSpec);
-for (int i = 0; i < RESOLVER_ITERATIONS; i++) {
+TestEntity entity = parent.addChild(childSpec);
+for (int i = 0; i < resolverIterations; i++) {
preResolve(entity);
--- End diff --
I'll rename this from "eventually" to "satisfiedAsynchronously".
What "eventually" meant was that the condition for getting the value
immediately is not yet satisfied; it will become satisfied asynchronously. For
example, when testing `attributeWhenReady` then another thread sets the
sensor's value after we have called `execDslEventually` - we expect that method
to block.
I'll also change the assertions so that we confirm we get a Maybe or the
real result when calling getImmediately in this situation.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625064#comment-15625064
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85905133
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -353,6 +386,33 @@ public DslSensorSupplier(DslComponent component,
Object sensorIndicator) {
}
@Override
+public Maybe> getImmediately() {
+return getImmediately(sensorName, false);
+}
+
+protected Maybe> getImmediately(Object si, boolean
resolved) {
+if (si instanceof Sensor) {
+return Maybe.>of((Sensor)si);
+} else if (si instanceof String) {
+Entity targetEntity = component.get();
+Sensor result = null;
+if (targetEntity!=null) {
+result =
targetEntity.getEntityType().getSensor((String)si);
+}
+if (result!=null) return Maybe.>of(result);
+return Maybe.>of(Sensors.newSensor(Object.class,
(String)si));
--- End diff --
I see - I'd overlooked the distinction here between getting the sensor
and/or its value - so this `newSensor(...si)` corresponds to the
`newSensor(...sensorName)` at line 283, but here we're returning the sensor.
Got it now.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625023#comment-15625023
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85903486
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -353,6 +386,33 @@ public DslSensorSupplier(DslComponent component,
Object sensorIndicator) {
}
@Override
+public Maybe> getImmediately() {
+return getImmediately(sensorName, false);
+}
+
+protected Maybe> getImmediately(Object si, boolean
resolved) {
+if (si instanceof Sensor) {
+return Maybe.>of((Sensor)si);
+} else if (si instanceof String) {
+Entity targetEntity = component.get();
+Sensor result = null;
+if (targetEntity!=null) {
+result =
targetEntity.getEntityType().getSensor((String)si);
+}
+if (result!=null) return Maybe.>of(result);
+return Maybe.>of(Sensors.newSensor(Object.class,
(String)si));
--- End diff --
Not sure I follow. In `attributeWhenReady`, if we're given a sensor name
then we will instantiate a new sensor object if it's not defined on the entity
type; and then we look up the sensor's value (returning Maybe.absent if it's
null). Here, we just want it to return the sensor object (rather than a sensor
value) so we do the first half: instantiate a new sensor object if it's not
defined on the entity type.
This is consistent with the blocking `get()` method. We return
`Maybe.absent()` if the `get()` method would have blocked, and otherwise we
return the same value as `get()` would have returned.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609469#comment-15609469
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85106043
--- Diff:
core/src/main/java/org/apache/brooklyn/util/core/task/ImmediateSupplier.java ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.brooklyn.util.core.task;
+
+import org.apache.brooklyn.util.guava.Maybe;
+
+/**
+ * A class that supplies objects of a single type, without blocking for
any significant length
+ * of time.
+ */
+public interface ImmediateSupplier {
+
+/**
+ * Indicates that we are unable to get the value immediately, because
that is not supported
+ * (e.g. because the supplier is composed of sub-tasks that do not
support {@link ImmediateSupplier}.
+ */
+public static class ImmediateUnsupportedException extends
RuntimeException {
--- End diff --
Might be nice to extend `UnsupportedOperationException` rather than
RuntimeException directly
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609467#comment-15609467
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85100137
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/BrooklynDslCommon.java
---
@@ -398,6 +411,58 @@ public DslObject(
this.config = MutableMap.copyOf(config);
}
+
+@Override
+public Maybe getImmediately() {
+Class type = this.type;
+if (type == null) {
--- End diff --
this `if` statement could be a method, it's shared with `newTask` below
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609473#comment-15609473
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85105289
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -353,6 +386,33 @@ public DslSensorSupplier(DslComponent component,
Object sensorIndicator) {
}
@Override
+public Maybe> getImmediately() {
+return getImmediately(sensorName, false);
+}
+
+protected Maybe> getImmediately(Object si, boolean
resolved) {
+if (si instanceof Sensor) {
+return Maybe.>of((Sensor)si);
+} else if (si instanceof String) {
+Entity targetEntity = component.get();
+Sensor result = null;
+if (targetEntity!=null) {
+result =
targetEntity.getEntityType().getSensor((String)si);
+}
+if (result!=null) return Maybe.>of(result);
+return Maybe.>of(Sensors.newSensor(Object.class,
(String)si));
--- End diff --
In `attributeWhenReady` above at line 286 you return a `Maybe.absent()` if
result is `null`; returning a `newSensor` here seems inconsistent, what is the
reason for the different behaviour here?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609472#comment-15609472
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85193147
--- Diff:
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/brooklyn/dsl/DslTest.java
---
@@ -118,39 +160,64 @@ public void runConcurrentWorker(Supplier
taskSupplier) {
}
private static class DslTestWorker implements Runnable {
-protected TestApplication parent;
-protected BrooklynDslDeferredSupplier dsl;
+protected final TestApplication parent;
+protected final BrooklynDslDeferredSupplier dsl;
+protected final Class type;
protected EntitySpec childSpec =
EntitySpec.create(TestEntity.class);
-protected Class type;
-
+protected int resolverIterations = MANY_RESOLVER_ITERATIONS;
+protected boolean eventually = false;
+private boolean wrapInTaskForImmediately = true;
+
public DslTestWorker(TestApplication parent,
BrooklynDslDeferredSupplier dsl, Class type) {
-this.parent = parent;
-this.dsl = dsl;
-this.type = type;
+this.parent = checkNotNull(parent, "parent");
+this.dsl = checkNotNull(dsl, "dsl");
+this.type = checkNotNull(type, "type");
}
+public DslTestWorker resolverIterations(int val) {
+resolverIterations = val;
+return this;
+}
+
+public DslTestWorker eventually(boolean val) {
+eventually = val;
+return this;
+}
+
+public DslTestWorker wrapInTaskForImmediately(boolean val) {
+wrapInTaskForImmediately = val;
+return this;
+}
+
@Override
public void run() {
-TestEntity entity = parent.createAndManageChild(childSpec);
-for (int i = 0; i < RESOLVER_ITERATIONS; i++) {
+TestEntity entity = parent.addChild(childSpec);
+for (int i = 0; i < resolverIterations; i++) {
preResolve(entity);
--- End diff --
Do you not want to do this under an "if(eventually)"?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609465#comment-15609465
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85111300
--- Diff:
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/brooklyn/dsl/DslTest.java
---
@@ -198,9 +275,44 @@ public ParentTestWorker(TestApplication parent,
BrooklynDslDeferredSupplier d
}
@Override
-protected void postResolve(Maybe actualValue) {
+protected void postResolve(TestEntity entity, Maybe
actualValue) {
assertEquals(actualValue.get(), parent);
}
}
+static Maybe execDslImmediately(final
BrooklynDslDeferredSupplier dsl, final Class type, final Entity context,
boolean execInTask) throws Exception {
+// Exec'ing immediately will call DSL in current thread. It needs
to find the context entity,
+// and does this using
BrooklynTaskTags.getTargetOrContextEntity(Tasks.current()).
+// If we are not in a task executed by the context entity, then
this lookup will fail.
+Callable> job = new Callable>() {
+public Maybe call() throws Exception {
+return Tasks.resolving(dsl).as(type)
+.context(context)
+.description("Computing "+dsl)
+.immediately(true)
+.getMaybe();
+}
+};
+if (execInTask) {
+Task> task =
((EntityInternal)context).getExecutionContext().submit(job);
+task.get(Asserts.DEFAULT_LONG_TIMEOUT);
+assertTrue(task.isDone());
+return task.get();
+
+} else {
+return job.call();
+}
+}
+
+static Maybe execDslRealRealyQuick(BrooklynDslDeferredSupplier
dsl, Class type, Entity context) {
--- End diff --
typo(?) `execDslRealRealQuick` or even better `execDslReallyReallyQuick`,
old-fashioned believer in adverbs that I am.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609468#comment-15609468
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85103989
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -353,6 +386,33 @@ public DslSensorSupplier(DslComponent component,
Object sensorIndicator) {
}
@Override
+public Maybe> getImmediately() {
+return getImmediately(sensorName, false);
+}
+
+protected Maybe> getImmediately(Object si, boolean
resolved) {
+if (si instanceof Sensor) {
+return Maybe.>of((Sensor)si);
+} else if (si instanceof String) {
+Entity targetEntity = component.get();
--- End diff --
again should this be `component.getImmediately().get()`, as in the
`attributeWhenReady` above?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609471#comment-15609471
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85203394
--- Diff:
core/src/test/java/org/apache/brooklyn/util/core/task/ValueResolverTest.java ---
@@ -41,6 +49,125 @@ public void setUp() throws Exception {
super.setUp();
}
+public void testTimeoutZero() {
+Maybe result =
Tasks.resolving(newSleepTask(Duration.TEN_SECONDS,
"foo")).as(String.class).context(app).timeout(Duration.ZERO).getMaybe();
+Assert.assertFalse(result.isPresent());
+}
+
+public void testTimeoutBig() {
+Maybe result = Tasks.resolving(newSleepTask(Duration.ZERO,
"foo")).as(String.class).context(app).timeout(Duration.TEN_SECONDS).getMaybe();
+Assert.assertEquals(result.get(), "foo");
+}
+
+public void testNoExecutionContextOnCompleted() {
--- End diff --
Don't follow what this is testing - how is it testing something about 'no
execution context'?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609466#comment-15609466
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85202543
--- Diff:
core/src/test/java/org/apache/brooklyn/core/entity/EntitySubscriptionTest.java
---
@@ -280,4 +279,55 @@ public void
testSubscriptionForInitialValueWhenNotValid() {
entity.subscriptions().subscribe(ImmutableMap.of("notifyOfInitialValue", true),
null, TestEntity.NAME, listener);
entity.subscriptions().subscribe(ImmutableMap.of("notifyOfInitialValue", true),
null, null, listener);
}
+
+@Test
+public void testContextEntityOnSubscriptionCallbackTask() {
+observedEntity.sensors().set(TestEntity.NAME, "myval");
+
entity.subscriptions().subscribe(ImmutableMap.of("notifyOfInitialValue", true),
observedEntity, TestEntity.NAME, listener);
+
+// notify-of-initial-value should give us our entity
+assertListenerCalledOnceWithContextEntityEventually(listener,
entity);
+listener.clearEvents();
+
+// as should subsequent events
+observedEntity.sensors().set(TestEntity.NAME, "myval2");
+assertListenerCalledOnceWithContextEntityEventually(listener,
entity);
+listener.clearEvents();
+
+// same for subscribing to children: context should be the
subscriber
+entity.subscriptions().subscribeToChildren(observedEntity,
TestEntity.SEQUENCE, listener);
+observedChildEntity.sensors().set(TestEntity.SEQUENCE, 123);
+assertListenerCalledOnceWithContextEntityEventually(listener,
entity);
+listener.clearEvents();
--- End diff --
should this be in a `tearDown`?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609463#comment-15609463
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85195730
--- Diff:
core/src/main/java/org/apache/brooklyn/util/core/task/ValueResolver.java ---
@@ -300,6 +313,18 @@ public T get() {
return Maybe.of((T) v);
try {
+if (immediately && v instanceof ImmediateSupplier) {
+final ImmediateSupplier supplier =
(ImmediateSupplier) v;
--- End diff --
Does the comment at line 383 need updated now that this is added?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609470#comment-15609470
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85102209
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/BrooklynDslCommon.java
---
@@ -525,6 +590,12 @@ public DslExternal(String providerName, String key) {
}
@Override
+public final Maybe getImmediately() {
--- End diff --
Could call this from within the `newTask` body.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609464#comment-15609464
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user geomacy commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/390#discussion_r85103885
--- Diff:
camp/camp-brooklyn/src/main/java/org/apache/brooklyn/camp/brooklyn/spi/dsl/methods/DslComponent.java
---
@@ -303,6 +330,12 @@ public DslConfigSupplier(DslComponent component,
String keyName) {
}
@Override
+public final Maybe getImmediately() {
+Entity targetEntity = component.get();
--- End diff --
should this be `component.getImmediately().get()`, as in the
`attributeWhenReady` above?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15591365#comment-15591365
]
ASF GitHub Bot commented on BROOKLYN-356:
-
GitHub user aledsage opened a pull request:
https://github.com/apache/brooklyn-server/pull/390
BROOKLYN-356: fix race in transformer by adding getImmediately
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aledsage/brooklyn-server
BROOKLYN-356-getImmediately
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/brooklyn-server/pull/390.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #390
commit 77b06dd664c03371746bf9c9c9cf808ead8e125d
Author: Aled Sage
Date: 2016-10-13T21:46:57Z
Subscription callbacks: task has contextEntity tag
commit 9b896cbc5abdb03d886a4c98d8323d63fa7bc26f
Author: Aled Sage
Date: 2016-10-18T18:08:18Z
BROOKLYN-356: threading for DSL task
Adds ImmediateSupplier, so can fetch Maybe of DSL value immediately,
in current thread (rather than invoking in task with a timeout).
commit 516128f56197763d5f15850336d2af85ce8197b4
Author: Aled Sage
Date: 2016-10-19T14:46:53Z
Reflections utils: improve use of generics
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569708#comment-15569708
]
Aled Sage commented on BROOKLYN-356:
I talked this over with [~svet]. We've done a very short-term improvement of
increasing the timeout. I'm looking at a longer term fix where we don't execute
so many tasks in different threads to get the value.
As Svet described, the {{Transformer}} executes a task to get the value (with a
timeout, because the value might not be available yet). It is trying to do a
non-blocking lookup of the value, but using the task api that requires the work
be executed in another thread. It therefore uses a timeout to terminate the
lookup if it hasn't returned promptly, but that obviously gives a race
condition.
A better solution for such a non-blocking lookup is to ask for the value
immediately, and to get back a {{Maybe}} (similar to guava's {{Optional}}, but
accepts null values). Each DSL (formatString, attributeWhenReady, etc) is of
type {{DeferredSupplier}}. I'm adding another interface ({{ImmediateSupplier}}
with a method {{Maybe getImmediately()}}) that we'll use to get the value
immediately, if we don't want to wait for it. However, it is a bit fiddly (e.g.
the DSL execution doesn't know the context entity, so I'm making some (small)
changes to the subscription context as well).
Even longer term, we could look at reversing the responsibilities: if the value
is an {{attributeWhenReady}} then that could do the subscription itself, and
call back to the transformer when the value becomes available (or changes).
Also, by using callbacks more, we can change some of the places that we do
blocking calls (so we improve our thread usage).
Below is the current sequence of task being executed (and the thread's
stacktrace for each), when executing {{TransformerEnricherWithDslTest}} and the
transformer retrieving the value:
{noformat}
task=Task[waiting on port]@J5UlSQ04;
thread=Thread[brooklyn-execmanager-S0LtDxD8-9,5,main];
submitTime=1476272773046; startTime=1476272773046; queuedTime=-1
task=Task[formatting '%d' with 1 task]@lNzNRuj1;
thread=Thread[brooklyn-execmanager-S0LtDxD8-11,5,main];
submitTime=1476272773045; startTime=1476272773045; queuedTime=-1
task=Task[formatting '%s' with 1 task]@ExQmzZBq;
thread=Thread[brooklyn-execmanager-S0LtDxD8-10,5,main];
submitTime=1476272773045; startTime=1476272773045; queuedTime=-1
task=Task[Resolving dependent value]@wQxaXZ77;
thread=Thread[brooklyn-execmanager-S0LtDxD8-7,5,main];
submitTime=1476272773024; startTime=1476272773025; queuedTime=-1
task=Task[LSM.publishInitialValue(Application[9ftxjzc5], Sensor: port
(java.lang.Integer));[subscription-delivery-entity-pf9ftxjzc5[Application[9ftxjzc5@C1UYl8il;
thread=Thread[brooklyn-execmanager-S0LtDxD8-1,5,main];
submitTime=1476272738192; startTime=1476272738192; queuedTime=-1
task=Task[waiting on port]@J5UlSQ04;
thread=Thread[brooklyn-execmanager-S0LtDxD8-9,5,main]
"brooklyn-execmanager-S0LtDxD8-9" daemon prio=5 tid=0x7facf22ab800
nid=0x660b at breakpoint[0x71be7000]
java.lang.Thread.State: RUNNABLE
at
org.apache.brooklyn.core.sensor.DependentConfiguration$WaitInTaskForAttributeReady.call(DependentConfiguration.java:252)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
task=Task[formatting '%d' with 1 task]@lNzNRuj1;
thread=Thread[brooklyn-execmanager-S0LtDxD8-11,5,main]
"brooklyn-execmanager-S0LtDxD8-11" daemon prio=5 tid=0x7facf3a8b800
nid=0x6c07 waiting on condition [0x71ded000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0007f64e81c0> (a
java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:425)
at java.util.concurrent.FutureTask.get(FutureTask.java:187)
at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
at org.apache.brooklyn.util.core.task.BasicTask.get(BasicTask.java:361)
at
org.apache.brooklyn.core.sensor.DependentConfiguration$1.call(DependentConfiguration.java:409)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
ja
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568872#comment-15568872
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user asfgit closed the pull request at:
https://github.com/apache/brooklyn-server/pull/381
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568764#comment-15568764
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on the issue:
https://github.com/apache/brooklyn-server/pull/381
Looks good. Nice fix for the long running test.
Will merge after tests complete.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568741#comment-15568741
]
ASF GitHub Bot commented on BROOKLYN-356:
-
GitHub user aledsage opened a pull request:
https://github.com/apache/brooklyn-server/pull/381
BROOKLYN-356: workaround, by increasing timeout
@neykov can you take a look please?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aledsage/brooklyn-server
BROOKLYN-356-increase-timeout
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/brooklyn-server/pull/381.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #381
commit c45777f33402b1326e637b8bd5314c295608d9a7
Author: Aled Sage
Date: 2016-10-12T13:16:52Z
JcloudsByonLocationResolverStubbedRebindTest: avoid taking 5 minutes
commit 0edce9879670560c245f2f6f4df3eae9fd2c27a8
Author: Aled Sage
Date: 2016-10-12T13:17:01Z
BROOKLYN-356: workaround, by increasing timeout
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568666#comment-15568666
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user asfgit closed the pull request at:
https://github.com/apache/brooklyn-server/pull/365
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568484#comment-15568484
]
Aled Sage commented on BROOKLYN-356:
An easy way to reproduce this (when debugging) is to change
{{DependentConfiguration.$WaitInTaskForAttributeReady.call()}} to add a sleep
at the start (e.g. {{Time.sleep(500);}}).
Then run
{{TransformerEnricherWithDslTest.testTransformerResolvesResolvableValues()}}.
It fails first time because the transformer's attempt to retrieve the value
times out after 200ms (whereas the task will take a minimum of 500ms).
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564634#comment-15564634
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on the issue:
https://github.com/apache/brooklyn-server/pull/365
Addressed comments and kicked another build.
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564581#comment-15564581
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user neykov commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/365#discussion_r82728162
--- Diff:
core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
---
@@ -507,23 +517,14 @@ public T call() {
try {
T result = null;
Throwable error = null;
-String oldThreadName = Thread.currentThread().getName();
try {
-if (RENAME_THREADS) {
--- End diff --
Correct. Details in the commit message:
```
Remove duplicate thread renaming
The code in beforeStartAtomicTask overrides the value set here. Keeping the
other one as it's called for scheduled tasks as well.
```
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548241#comment-15548241
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on the issue:
https://github.com/apache/brooklyn-server/pull/365
@neykov previous build failure here could well be fixed by
https://github.com/apache/brooklyn-server/pull/368 - our `BrooklynLeakListener`
wasn't finding any left-behind management contexts to stop them, so our memory
usage might have been climbing over time.
Once that PR is confirmed to work in jenkins (including checking its stdout
that the leak-detector really executed) then I'll merge
https://github.com/apache/brooklyn-server/pull/368).
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548215#comment-15548215
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on the issue:
https://github.com/apache/brooklyn-server/pull/365
@neykov looks very good to me. A couple of extremely minor things relating
to this PR (the comment, and confirming about the thread renaming). Can you
kick off the jenkins build again, and then merge this?
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548213#comment-15548213
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on the issue:
https://github.com/apache/brooklyn-server/pull/365
Jenkins build failure looks unrelated: when building "Brooklyn CAMP REST
API", got `Build timed out (after 60 minutes). Marking the build as aborted.`
It was in the middle of invoking
`DslAndRebindYamlTest.testDslAttributeWhenReadyPersistedWithoutLeakingResolvedValue()`.
Below is the logging. Note the long GC pause times (approx 0.2 seconds each
time). That suggests to me that it was doing a full GC. But it's hard to say on
a shared build machine that might be starved of resources.
We should (separately) investigate `DslAndRebindYamlTest` to see how
resource-hungry it is.
Unfortunately we don't have the "brooklyn gc" logging to tell us about the
memory usage over time (it's logged at debug). But then for all tests except
that one, the test would complete within one minute so we'd have deleted the
managementContext before the brooklynGC did any logging.
Maybe we want something like the `BrooklynLeakListener` to spawn a
scheduled executor in the background, and to log the memory usage of the VM
periodically so we can see what the test suite is doing over time?
```
2016-10-02 21:04:32,889 INFO TESTNG INVOKING: "Surefire test" -
org.apache.brooklyn.camp.brooklyn.DslAndRebindYamlTest.testDslAttributeWhenReadyPersistedWithoutLeakingResolvedValue()
2016-10-02 21:04:33,052 INFO Started application
BasicApplicationImpl{id=gpt66t7160}
2016-10-02 21:04:33,052 INFO Waiting on 3 task(s)
SUREFIRE-859: [GC 538982K->318659K(759808K), 0.1438800 secs]
SUREFIRE-859: [GC 542915K->355674K(767488K), 0.3288260 secs]
SUREFIRE-859: [GC 579930K->383114K(718336K), 0.2902200 secs]
SUREFIRE-859: [GC 558218K->401746K(736256K), 0.2247390 secs]
SUREFIRE-859: [GC 576850K->402834K(729088K), 0.2136890 secs]
SUREFIRE-859: [GC 559506K->395786K(711680K), 0.1718160 secs]
SUREFIRE-859: [GC 552458K->398330K(730624K), 0.1999770 secs]
SUREFIRE-859: [GC 550906K->397842K(709632K), 0.1918430 secs]
SUREFIRE-859: [GC 550418K->397994K(730112K), 0.1925270 secs]
SUREFIRE-859: [GC 548522K->395114K(730624K), 0.1828260 secs]
SUREFIRE-859: [GC 545642K->396522K(732160K), 0.1813480 secs]
SUREFIRE-859: [GC 549610K->398146K(731648K), 0.190 secs]
SUREFIRE-859: [GC 551234K->398202K(732672K), 0.1862850 secs]
SUREFIRE-859: [GC 552314K->396794K(732160K), 0.1816440 secs]
SUREFIRE-859: [GC 550906K->397530K(734208K), 0.1915190 secs]
SUREFIRE-859: [GC 554202K->398810K(733184K), 0.1886670 secs]
SUREFIRE-859: [GC 555482K->398810K(734720K), 0.1962980 secs]
SUREFIRE-859: [GC 557530K->398066K(734720K), 0.1893480 secs]
SUREFIRE-859: [GC 556786K->398322K(736768K), 0.1909740 secs]
SUREFIRE-859: [GC 560114K->400122K(735744K), 0.1974540 secs]
SUREFIRE-859: [GC 561914K->400274K(736768K), 0.2081040 secs]
SUREFIRE-859: [GC 563090K->399050K(736768K), 0.1930720 secs]
SUREFIRE-859: [GC 561866K->399466K(738304K), 0.1916980 secs]
SUREFIRE-859: [GC 564842K->400842K(737792K), 0.2071120 secs]
SUREFIRE-859: [GC 566218K->400866K(738816K), 0.2107460 secs]
SUREFIRE-859: [GC 567266K->399898K(738304K), 0.1974350 secs]
SUREFIRE-859: [GC 566298K->400474K(739840K), 0.2096590 secs]
SUREFIRE-859: [GC 568922K->401506K(739328K), 0.2051750 secs]
SUREFIRE-859: [GC 569954K->401882K(740352K), 0.2205450 secs]
SUREFIRE-859: [GC 571866K->400362K(740352K), 0.1994660 secs]
SUREFIRE-859: [GC 570346K->401922K(740864K), 0.2073610 secs]
SUREFIRE-859: [GC 572930K->402458K(740864K), 0.2085150 secs]
SUREFIRE-859: [GC 573466K->402618K(741376K), 0.2088340 secs]
SUREFIRE-859: [GC 574138K->400634K(740864K), 0.1965650 secs]
SUREFIRE-859: [GC 572154K->402330K(742400K), 0.2098550 secs]
SUREFIRE-859: [GC 575898K->403098K(741888K), 0.2152220 secs]
SUREFIRE-859: [GC 57K->401906K(742400K), 0.2178950 secs]
SUREFIRE-859: [GC 575986K->401714K(742400K), 0.2041080 secs]
Build timed out (after 60 minutes). Marking the build as aborted.
```
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548199#comment-15548199
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/365#discussion_r81934241
--- Diff:
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/brooklyn/enricher/TransformerEnricherWithDslTest.java
---
@@ -0,0 +1,78 @@
+/*
+ * Copyright 2016 The Apache Software Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.brooklyn.camp.brooklyn.enricher;
+
+import static org.testng.Assert.assertEquals;
+
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.camp.brooklyn.spi.dsl.methods.BrooklynDslCommon;
+import org.apache.brooklyn.core.entity.EntityAsserts;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.core.test.BrooklynAppUnitTestSupport;
+import org.apache.brooklyn.enricher.stock.Transformer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testng.annotations.Test;
+
+public class TransformerEnricherWithDslTest extends
BrooklynAppUnitTestSupport {
+private static final Logger LOG =
LoggerFactory.getLogger(TransformerEnricherWithDslTest.class);
+
+int START_PORT = 1;
+
+@Test(groups="Broken")
+public void testTransformerResolvesResolvableValues() {
+testTransformerResolvesResolvableValues(START_PORT, 200);
+}
+
+@Test(groups={"Integration", "Broken"}, invocationCount=10)
--- End diff --
Worth including a comment above the test that points people to
https://issues.apache.org/jira/browse/BROOKLYN-356
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548196#comment-15548196
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/365#discussion_r81934135
--- Diff:
camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/brooklyn/enricher/TransformerEnricherWithDslTest.java
---
@@ -0,0 +1,78 @@
+/*
+ * Copyright 2016 The Apache Software Foundation.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.brooklyn.camp.brooklyn.enricher;
+
+import static org.testng.Assert.assertEquals;
+
+import org.apache.brooklyn.api.sensor.AttributeSensor;
+import org.apache.brooklyn.api.sensor.EnricherSpec;
+import org.apache.brooklyn.camp.brooklyn.spi.dsl.methods.BrooklynDslCommon;
+import org.apache.brooklyn.core.entity.EntityAsserts;
+import org.apache.brooklyn.core.sensor.Sensors;
+import org.apache.brooklyn.core.test.BrooklynAppUnitTestSupport;
+import org.apache.brooklyn.enricher.stock.Transformer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testng.annotations.Test;
+
+public class TransformerEnricherWithDslTest extends
BrooklynAppUnitTestSupport {
+private static final Logger LOG =
LoggerFactory.getLogger(TransformerEnricherWithDslTest.class);
+
+int START_PORT = 1;
+
+@Test(groups="Broken")
+public void testTransformerResolvesResolvableValues() {
+testTransformerResolvesResolvableValues(START_PORT, 200);
+}
+
+@Test(groups={"Integration", "Broken"}, invocationCount=10)
+public void testTransformerResolvesResolvableValuesIntegration() {
--- End diff --
Nice test. Without the jitter, I ran it with `invocationCount=100` and it
failed 2 out of 100 times with errors like:
java.lang.AssertionError: entity=Application[s01l91es];
attribute=Sensor: port.transformed (java.lang.String) expected [10889] but
found [null]
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548174#comment-15548174
]
ASF GitHub Bot commented on BROOKLYN-356:
-
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/brooklyn-server/pull/365#discussion_r81932394
--- Diff:
core/src/main/java/org/apache/brooklyn/util/core/task/BasicExecutionManager.java
---
@@ -757,9 +758,23 @@ protected void internalBeforeStart(Map flags,
Task task) {
PerThreadCurrentTaskHolder.perThreadCurrentTask.set(task);
((TaskInternal)task).setStartTimeUtc(System.currentTimeMillis());
}
+
+jitterThreadStart(task);
+
invokeCallback(flags.get("newTaskStartCallback"), task);
}
+private void jitterThreadStart(Task task) {
--- End diff --
Great idea!
I wonder if we should make it non-static so it can be set by a test, and
unset in its `tearDown` method. But let's go with static. For now, we can use
it by setting the system property when doing `mvn clean install`.
Once we've fixed (most of) the failures that it exposes, then we can think
about enabling it for some tests in jenkins - for regression testing purposes.
But we want jenkins to be deterministic!
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BROOKLYN-356) The sensor Transformer enricher fails to resolve its targetValue indeterministically.
[
https://issues.apache.org/jira/browse/BROOKLYN-356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15540374#comment-15540374
]
ASF GitHub Bot commented on BROOKLYN-356:
-
GitHub user neykov opened a pull request:
https://github.com/apache/brooklyn-server/pull/365
[BROOKLYN-356] Transformer value resolver test
* Adds thread startup jittering so we can lure out any concurrency bugs
hiding in the code base. Testing `core` with this results in a handful of
failures. Similar problems are very apparent on the windows builds where the
machines have a single CPU and are frequently failing. I'd enable this by
default for testing only if it wasn't slowing the tests to a craw.
* Adds a test which shows the problem with the transformer resolver. It's
failing under normal conditions infrequently, so use with the above enabled.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/neykov/brooklyn-server
transformer-resolver-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/brooklyn-server/pull/365.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #365
commit 0a3f622717b08a1c04d5614f33fd44115cf8a40a
Author: Svetoslav Neykov
Date: 2016-10-02T09:20:58Z
Adds optional task startup jitter
Useful for troubleshooting/testing concurrency related code.
commit c71c896075ac6582d4a10a9bacdd3e294953c129
Author: Svetoslav Neykov
Date: 2016-10-02T13:26:49Z
Logging - link thread name to running task
commit b4fd40d3d1359b3ce3623801f6c72b465a8725cc
Author: Svetoslav Neykov
Date: 2016-10-02T09:52:13Z
Remove duplicate thread renaming
The code in beforeStartAtomicTask overrides the value set here. Keeping the
other one as it's called for scheduled tasks as well.
commit 361574c881c215503621d7b5a0a7d52423368b2b
Author: Svetoslav Neykov
Date: 2016-10-02T13:19:46Z
Test Transformer reliability when resolving DSL
> The sensor Transformer enricher fails to resolve its targetValue
> indeterministically.
> -
>
> Key: BROOKLYN-356
> URL: https://issues.apache.org/jira/browse/BROOKLYN-356
> Project: Brooklyn
> Issue Type: Bug
>Reporter: Svetoslav Neykov
>
> The sensor {{Transformer}} enricher fails to resolve its {{targetValue}}
> indeterministically, especially so on loaded systems. This is especially
> problematic if sourceSensor/triggerSensors change infrequently or do not
> change ever (when using default values).
> Bueprints using the {{Transformer}} behave perfectly fine during development
> and on test setups, but will fail on production machines (i.e. under load).
> The code [doing the value
> resolving|https://github.com/apache/brooklyn-server/blob/b59e7463a9b337c2d0e7931cd420d5bac68d8549/core/src/main/java/org/apache/brooklyn/enricher/stock/Transformer.java#L90-L94]
> tries to do so in a non-blocking fashion by spinning a thread to try to
> resolve and cancelling it after a short while without guarantees it ever got
> scheduled to run. It's more likely to fail when nesting DSLs, for example
> nesting several levels of {{$brooklyn:formatString}} and ending with a
> {{$brooklyn:attributeWhenRready}}. It's a common pattern in moderately
> complex blueprints. It needs to schedule a thread for each nesting level thus
> maximizing the chance that the value will not be resolved in the allotted
> time even if resolvabe. This is especially bad for sensors which don't get
> updated, for example {{PortAttributeSensorAndConfigKey} set only when
> initializing the entity or any config values which are always resolvable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
