[jira] [Commented] (APEXMALHAR-2542) enricher.ApplicationTest access to hard coded /tmp location
[
https://issues.apache.org/jira/browse/APEXMALHAR-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181517#comment-16181517
]
Thomas Weise commented on APEXMALHAR-2542:
--
The output is also written to the invalid location, but of course that won't be
noticed when the "test" actually doesn't verify anything. That could be another
JIRA to turn it into something more useful.
Example for property override in test:
https://github.com/apache/apex-malhar/blob/master/examples/exactly-once/src/test/java/org/apache/apex/examples/exactlyonce/ExactlyOnceJdbcOutputTest.java#L109
> enricher.ApplicationTest access to hard coded /tmp location
> ---
>
> Key: APEXMALHAR-2542
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2542
> Project: Apache Apex Malhar
> Issue Type: Bug
>Reporter: Thomas Weise
>Assignee: Ananth
>
> {code}
> 2017-09-25 18:34:37,900 [3/Enrich:POJOEnricher] WARN util.LoggerUtil
> shouldFetchLogFileInformation - Log information is unavailable. To enable log
> information log4j/logging should be configured with single FileAppender that
> has immediateFlush set to true and log level set to ERROR or greater.
> 2017-09-25 18:34:37,916 [3/Enrich:POJOEnricher] ERROR
> engine.StreamingContainer run - Abandoning deployment of operator
> OperatorDeployInfo[id=3,name=Enrich,type=GENERIC,checkpoint={,
> 0,
> 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=Enrich,sourceNodeId=2,sourcePortName=out,locality=,partitionMask=0,partitionKeys=]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=Console,bufferServer=localhost]]]
> due to setup failure.
> java.lang.RuntimeException: Unable to initialize cache manager
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:191)
> at
> com.datatorrent.contrib.enrich.POJOEnricher.activate(POJOEnricher.java:234)
> at com.datatorrent.stram.engine.Node.activate(Node.java:644)
> at
> com.datatorrent.stram.engine.GenericNode.activate(GenericNode.java:212)
> at
> com.datatorrent.stram.engine.StreamingContainer.setupNode(StreamingContainer.java:1364)
> at
> com.datatorrent.stram.engine.StreamingContainer.access$100(StreamingContainer.java:129)
> at
> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1413)
> Caused by: java.io.IOException: Provided path /tmp/circleMapping.txt is not a
> file
> at com.datatorrent.contrib.enrich.FSLoader.connect(FSLoader.java:190)
> at
> com.datatorrent.lib.db.cache.CacheManager.initialize(CacheManager.java:109)
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:189)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (APEXMALHAR-2542) enricher.ApplicationTest access to hard coded /tmp location
[
https://issues.apache.org/jira/browse/APEXMALHAR-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181500#comment-16181500
]
Thomas Weise commented on APEXMALHAR-2542:
--
Please never use /tmp, but rather the Java tmp directory. In this case it
appears the only thing the unit test needs to do is set the property to the
location under resources in the build workspace, if it is immutable.
> enricher.ApplicationTest access to hard coded /tmp location
> ---
>
> Key: APEXMALHAR-2542
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2542
> Project: Apache Apex Malhar
> Issue Type: Bug
>Reporter: Thomas Weise
>Assignee: Ananth
>
> {code}
> 2017-09-25 18:34:37,900 [3/Enrich:POJOEnricher] WARN util.LoggerUtil
> shouldFetchLogFileInformation - Log information is unavailable. To enable log
> information log4j/logging should be configured with single FileAppender that
> has immediateFlush set to true and log level set to ERROR or greater.
> 2017-09-25 18:34:37,916 [3/Enrich:POJOEnricher] ERROR
> engine.StreamingContainer run - Abandoning deployment of operator
> OperatorDeployInfo[id=3,name=Enrich,type=GENERIC,checkpoint={,
> 0,
> 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=Enrich,sourceNodeId=2,sourcePortName=out,locality=,partitionMask=0,partitionKeys=]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=Console,bufferServer=localhost]]]
> due to setup failure.
> java.lang.RuntimeException: Unable to initialize cache manager
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:191)
> at
> com.datatorrent.contrib.enrich.POJOEnricher.activate(POJOEnricher.java:234)
> at com.datatorrent.stram.engine.Node.activate(Node.java:644)
> at
> com.datatorrent.stram.engine.GenericNode.activate(GenericNode.java:212)
> at
> com.datatorrent.stram.engine.StreamingContainer.setupNode(StreamingContainer.java:1364)
> at
> com.datatorrent.stram.engine.StreamingContainer.access$100(StreamingContainer.java:129)
> at
> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1413)
> Caused by: java.io.IOException: Provided path /tmp/circleMapping.txt is not a
> file
> at com.datatorrent.contrib.enrich.FSLoader.connect(FSLoader.java:190)
> at
> com.datatorrent.lib.db.cache.CacheManager.initialize(CacheManager.java:109)
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:189)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (APEXMALHAR-2542) enricher.ApplicationTest access to hard coded /tmp location
[
https://issues.apache.org/jira/browse/APEXMALHAR-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181473#comment-16181473
]
Ananth commented on APEXMALHAR-2542:
The root cause is that the application is expecting a file in the path
/tmp/circleMapping.txt as the exception suggests. This path is being configured
in the application properties file.
There are a couple of options to fix this.
# Let the build system copy the file into the target location as a pre-build
step. - Not a great solution as the test will fail on local systems
# Copy the file in the resources folder as bundled in the jar to the actual
location that is configured in the properties file of the application. This
will work for almost all of the systems. The issue however is that this cannot
be a solution for a true cluster based test. ( as the copy is happening in the
driver/application code as opposed to the operator ). Also the path
"/tmp/circleMapping.txt" implies a Unix styled OS and hence this approach will
not work when being built on windows. (assuming no Cygwin styled build)
Option 2 is being implemented as a fix to the problem.
> enricher.ApplicationTest access to hard coded /tmp location
> ---
>
> Key: APEXMALHAR-2542
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2542
> Project: Apache Apex Malhar
> Issue Type: Bug
>Reporter: Thomas Weise
>Assignee: Ananth
>
> {code}
> 2017-09-25 18:34:37,900 [3/Enrich:POJOEnricher] WARN util.LoggerUtil
> shouldFetchLogFileInformation - Log information is unavailable. To enable log
> information log4j/logging should be configured with single FileAppender that
> has immediateFlush set to true and log level set to ERROR or greater.
> 2017-09-25 18:34:37,916 [3/Enrich:POJOEnricher] ERROR
> engine.StreamingContainer run - Abandoning deployment of operator
> OperatorDeployInfo[id=3,name=Enrich,type=GENERIC,checkpoint={,
> 0,
> 0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=input,streamId=Enrich,sourceNodeId=2,sourcePortName=out,locality=,partitionMask=0,partitionKeys=]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=output,streamId=Console,bufferServer=localhost]]]
> due to setup failure.
> java.lang.RuntimeException: Unable to initialize cache manager
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:191)
> at
> com.datatorrent.contrib.enrich.POJOEnricher.activate(POJOEnricher.java:234)
> at com.datatorrent.stram.engine.Node.activate(Node.java:644)
> at
> com.datatorrent.stram.engine.GenericNode.activate(GenericNode.java:212)
> at
> com.datatorrent.stram.engine.StreamingContainer.setupNode(StreamingContainer.java:1364)
> at
> com.datatorrent.stram.engine.StreamingContainer.access$100(StreamingContainer.java:129)
> at
> com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1413)
> Caused by: java.io.IOException: Provided path /tmp/circleMapping.txt is not a
> file
> at com.datatorrent.contrib.enrich.FSLoader.connect(FSLoader.java:190)
> at
> com.datatorrent.lib.db.cache.CacheManager.initialize(CacheManager.java:109)
> at
> com.datatorrent.contrib.enrich.AbstractEnricher.activate(AbstractEnricher.java:189)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (APEXCORE-784) Do not redirect Application Master console output in StramMiniClusterTest
[ https://issues.apache.org/jira/browse/APEXCORE-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181004#comment-16181004 ] ASF GitHub Bot commented on APEXCORE-784: - vrozov closed pull request #580: APEXCORE-784 Do not redirect Application Master console output in StramMiniClusterTest URL: https://github.com/apache/apex-core/pull/580 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Do not redirect Application Master console output in StramMiniClusterTest > - > > Key: APEXCORE-784 > URL: https://issues.apache.org/jira/browse/APEXCORE-784 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > > When the test is executed in CI environment logs redirected to > AppMaster.stdout and AppMaster.stderr are not accessible making it difficult > to troubleshoot CI build failure -- This message was sent by Atlassian JIRA (v6.4.14#64029)
