[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346508&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346508
]
ASF GitHub Bot logged work on BEAM-8568:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Nov/19 07:14
Start Date: 20/Nov/19 07:14
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555871445
@kennknowles The tests always end up with the same state on this branch.
eg.
```
16:37:51 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInStreamingStaticSharding FAILED
16:37:51 java.lang.RuntimeException at WindowedWordCountIT.java:188
17:20:15
17:20:15 org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
17:20:15 java.lang.RuntimeException at WordCountIT.java:69
17:40:50
17:40:50 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInBatchStaticSharding FAILED
17:40:50 java.lang.RuntimeException at WindowedWordCountIT.java:188
18:15:28 Build timed out (after 120 minutes). Marking the build as aborted.
18:15:28 Build was aborted
18:15:28 Recording test results
```
When I checked the source code of these tests, they seem configured have
input and output configured `gcs://` paths. When I try to run them locally I
end up with...
```
com.google.api.client.http.HttpResponseException: 400 Bad Request
{
"error": "invalid_grant",
"error_description": "Bad Request"
}
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1102)
at
com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:227)
at
com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:181)
at
com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:167)
at
com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at
com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)
at
com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:381)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:357)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.queue(AbstractGoogleClientRequest.java:662)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.queue(AbstractGoogleJsonClientRequest.java:108)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.enqueueGetFileSize(GcsUtil.java:755)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.makeGetBatches(GcsUtil.java:608)
```
which also implies that they are using `gcs` instead of local fs.
Btw, this doesn't fail on Master branch:
[jenkins](https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Commit/5443/)
[PR](https://github.com/apache/beam/pull/10024)
What should I do next? I'm don't think this is ok to merge as the failure
seem to be deterministic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 346508)
Time Spent: 3.5h (was: 3h 20m)
> Local file system does not match relative path with wildcards
> -------------------------------------------------------------
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.16.0
> Reporter: Ondrej Cerny
> Assignee: David Moravek
> Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
