[
https://issues.apache.org/jira/browse/BEAM-6002?focusedWorklogId=163348&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-163348
]
ASF GitHub Bot logged work on BEAM-6002:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Nov/18 03:47
Start Date: 07/Nov/18 03:47
Worklog Time Spent: 10m
Work Description: kennknowles edited a comment on issue #6970:
[BEAM-6002] Rollbacks PR #6927
URL: https://github.com/apache/beam/pull/6970#issuecomment-436494768
The Dataflow runner stuff is a rate limit for writing to a single jar, the
DF worker. The limit on GCS is one write per second so this is pretty obvious.
The root cause is the Nexmark launcher sharing the staging location for all
jobs. TBH not sure why it is surfacing now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 163348)
Time Spent: 1h 40m (was: 1.5h)
> Nexmark tests timing out on all runners (crash loop due to metrics?)
> ---------------------------------------------------------------------
>
> Key: BEAM-6002
> URL: https://issues.apache.org/jira/browse/BEAM-6002
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Kenneth Knowles
> Assignee: Lukasz Gajowy
> Priority: Critical
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Spark/992/
> {code}
> 08:58:26 2018-11-06T16:58:26.035Z RUNNING Query0
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.elements, from namespace Query0.Events
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.bytes, from namespace Query0.Events
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.startTime for namespace Query0.Events
> 08:58:26 2018-11-06T16:58:26.035Z no activity
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.endTime for namespace Query0.Events
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.elements, from namespace Query0.Results
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.bytes, from namespace Query0.Results
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTime for namespace Query0.Results
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTime for namespace Query0.Results
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTimestamp for namespace Query0.Results
> 08:58:26 18/11/06 16:58:26 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTimestamp for namespace Query0.Results
> 08:58:41 2018-11-06T16:58:41.035Z RUNNING Query0
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.elements, from namespace Query0.Events
> 08:58:41 2018-11-06T16:58:41.036Z no activity
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.bytes, from namespace Query0.Events
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.startTime for namespace Query0.Events
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.endTime for namespace Query0.Events
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.elements, from namespace Query0.Results
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.bytes, from namespace Query0.Results
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTime for namespace Query0.Results
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTime for namespace Query0.Results
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTimestamp for namespace Query0.Results
> 08:58:41 18/11/06 16:58:41 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTimestamp for namespace Query0.Results
> 08:58:56 2018-11-06T16:58:56.036Z RUNNING Query0
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.elements, from namespace Query0.Events
> 08:58:56 2018-11-06T16:58:56.036Z no activity
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> event.bytes, from namespace Query0.Events
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.startTime for namespace Query0.Events
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric event.endTime for namespace Query0.Events
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.elements, from namespace Query0.Results
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get metric
> result.bytes, from namespace Query0.Results
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTime for namespace Query0.Results
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTime for namespace Query0.Results
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.startTimestamp for namespace Query0.Results
> 08:58:56 18/11/06 16:58:56 ERROR
> org.apache.beam.sdk.testutils.metrics.MetricsReader: Failed to get
> distribution metric result.endTimestamp for namespace Query0.Results
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
