[
https://issues.apache.org/jira/browse/BEAM-11264?focusedWorklogId=512630&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-512630
]
ASF GitHub Bot logged work on BEAM-11264:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Nov/20 22:21
Start Date: 16/Nov/20 22:21
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #13347:
URL: https://github.com/apache/beam/pull/13347#issuecomment-728365075
Well I can't find any documentation stating that Python tests will fail,
just that Dataflow tests will fail. But looking at the last [Python PreCommit
failure](https://ci-beam.apache.org/job/beam_PreCommit_Python_Phrase/2316) it
looks like the issue was hanging Dataflow tests:
```
12:24:47 > Task :sdks:python:test-suites:dataflow:py36:preCommitIT_batch_V2
12:24:47
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:24:29.045Z:
JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric
descriptors, so new user metrics of the form custom.googleapis.com/* will not
be created. However, all user metrics are also available in the metric
dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics,
you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
12:24:49
12:24:49 > Task :sdks:python:test-suites:dataflow:py37:preCommitIT_batch_V2
12:24:49
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:24:46.063Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
12:25:17
12:25:17 > Task :sdks:python:test-suites:dataflow:py36:preCommitIT_batch_V2
12:25:17
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:24:47.927Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
12:25:19
12:25:19 > Task :sdks:python:test-suites:dataflow:py37:preCommitIT_batch_V2
12:25:19
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:25:16.445Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
12:25:19
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:25:16.480Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
12:25:47
12:25:47 > Task :sdks:python:test-suites:dataflow:py36:preCommitIT_batch_V2
12:25:47
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:25:23.244Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
12:25:47
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-11-16T20:25:23.307Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
12:48:35 Build was aborted
```
It looks like the same suites didn't fail on your verification branch
though.. maybe something has changed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 512630)
Time Spent: 1h 50m (was: 1h 40m)
> pandas read methods dont reshuffle filenames
> --------------------------------------------
>
> Key: BEAM-11264
> URL: https://issues.apache.org/jira/browse/BEAM-11264
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: P2
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> So its not possible to process distinct files per worker.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
