[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=349661=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-349661
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 26/Nov/19 09:35
Start Date: 26/Nov/19 09:35
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 349661)
Time Spent: 5h (was: 4h 50m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=349660=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-349660
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 26/Nov/19 09:34
Start Date: 26/Nov/19 09:34
Worklog Time Spent: 10m
Work Description: mxm commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-558544666
This is passing now. Thanks!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 349660)
Time Spent: 4h 50m (was: 4h 40m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=349427=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-349427
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 25/Nov/19 23:24
Start Date: 25/Nov/19 23:24
Worklog Time Spent: 10m
Work Description: Ardagan commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-558386949
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 349427)
Time Spent: 4h 40m (was: 4.5h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=347160=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347160
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 21/Nov/19 04:57
Start Date: 21/Nov/19 04:57
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-556917287
Tests pass on the release branch. https://gradle.com/s/5jl76y2tkiwmc
So something about this change is causing the error deterministically, as
you say. Since it is healthy on `master`, perhaps there are other coupled
commits that need to be cherrypicked.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 347160)
Time Spent: 4.5h (was: 4h 20m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=347156=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347156
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 21/Nov/19 04:42
Start Date: 21/Nov/19 04:42
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-556917287
Tests pass on the release branch...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 347156)
Time Spent: 4h 20m (was: 4h 10m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=347148=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347148
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 21/Nov/19 03:54
Start Date: 21/Nov/19 03:54
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-556908586
I could not identify a stuck job, even though the tests were stuck.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 347148)
Time Spent: 4h 10m (was: 4h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=347143=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347143
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 21/Nov/19 03:21
Start Date: 21/Nov/19 03:21
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-556902409
Hmm, confirmed that it times out. Running locally for me it is up to 50+
minutes. Presumably just stuck. I'll check.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 347143)
Time Spent: 4h (was: 3h 50m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=347133=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347133
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 21/Nov/19 02:25
Start Date: 21/Nov/19 02:25
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-556854842
Something to do with Jenkins. That gradle scan I posted was a run against
this PR's head (not the merge commit, so maybe that is the problem)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 347133)
Time Spent: 3h 50m (was: 3h 40m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346509
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 20/Nov/19 07:15
Start Date: 20/Nov/19 07:15
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555871767
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 346509)
Time Spent: 3h 40m (was: 3.5h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346508
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 20/Nov/19 07:14
Start Date: 20/Nov/19 07:14
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555871445
@kennknowles The tests always end up with the same state on this branch.
eg.
```
16:37:51 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInStreamingStaticSharding FAILED
16:37:51 java.lang.RuntimeException at WindowedWordCountIT.java:188
17:20:15
17:20:15 org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
17:20:15 java.lang.RuntimeException at WordCountIT.java:69
17:40:50
17:40:50 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInBatchStaticSharding FAILED
17:40:50 java.lang.RuntimeException at WindowedWordCountIT.java:188
18:15:28 Build timed out (after 120 minutes). Marking the build as aborted.
18:15:28 Build was aborted
18:15:28 Recording test results
```
When I checked the source code of these tests, they seem configured have
input and output configured `gcs://` paths. When I try to run them locally I
end up with...
```
com.google.api.client.http.HttpResponseException: 400 Bad Request
{
"error": "invalid_grant",
"error_description": "Bad Request"
}
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1102)
at
com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:227)
at
com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:181)
at
com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:167)
at
com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at
com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)
at
com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:381)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:357)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.queue(AbstractGoogleClientRequest.java:662)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.queue(AbstractGoogleJsonClientRequest.java:108)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.enqueueGetFileSize(GcsUtil.java:755)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.makeGetBatches(GcsUtil.java:608)
```
which also implies that they are using `gcs` instead of local fs.
Btw, this doesn't fail on Master branch:
[jenkins](https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Commit/5443/)
[PR](https://github.com/apache/beam/pull/10024)
What should I do next? I'm don't think this is ok to merge as the failure
seem to be deterministic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 346508)
Time Spent: 3.5h (was: 3h 20m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346506=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346506
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 20/Nov/19 07:13
Start Date: 20/Nov/19 07:13
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555871445
@kennknowles The tests always end up with the same state on this branch.
eg.
```
16:37:51 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInStreamingStaticSharding FAILED
16:37:51 java.lang.RuntimeException at WindowedWordCountIT.java:188
17:20:15
17:20:15 org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
17:20:15 java.lang.RuntimeException at WordCountIT.java:69
17:40:50
17:40:50 org.apache.beam.examples.WindowedWordCountIT >
testWindowedWordCountInBatchStaticSharding FAILED
17:40:50 java.lang.RuntimeException at WindowedWordCountIT.java:188
18:15:28 Build timed out (after 120 minutes). Marking the build as aborted.
18:15:28 Build was aborted
18:15:28 Recording test results
```
When I checked the source code of these tests, they seem configured have
input and output configured `gcs://` paths. When I try to run them locally I
end up with...
```
com.google.api.client.http.HttpResponseException: 400 Bad Request
{
"error": "invalid_grant",
"error_description": "Bad Request"
}
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1102)
at
com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:227)
at
com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:181)
at
com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:167)
at
com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at
com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)
at
com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:381)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:357)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.queue(AbstractGoogleClientRequest.java:662)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.queue(AbstractGoogleJsonClientRequest.java:108)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.enqueueGetFileSize(GcsUtil.java:755)
at
org.apache.beam.sdk.extensions.gcp.util.GcsUtil.makeGetBatches(GcsUtil.java:608)
```
which also implies that they are using `gcs` instead of local fs.
Btw, this doesn't fail on Master branch:
(jenkins)[https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Commit/5443/],
(PR)[https://github.com/apache/beam/pull/10024]
What should I do next? I'm don't think this is ok to merge as the failure
seem to be deterministic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 346506)
Time Spent: 3h 20m (was: 3h 10m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346493=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346493
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 20/Nov/19 06:34
Start Date: 20/Nov/19 06:34
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555860949
Looks good to me: https://gradle.com/s/c47e35q47qzpq
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 346493)
Time Spent: 3h 10m (was: 3h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=346477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346477
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 20/Nov/19 06:04
Start Date: 20/Nov/19 06:04
Worklog Time Spent: 10m
Work Description: kennknowles commented on issue #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555853543
Where are you seeing the logs indicating GCS? What I see in the logs is
entirely not useful. I can certainly try to reproduce this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 346477)
Time Spent: 3h (was: 2h 50m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=345558=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345558
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 18/Nov/19 20:24
Start Date: 18/Nov/19 20:24
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555193577
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 345558)
Time Spent: 2h 50m (was: 2h 40m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=34=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-34
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 18/Nov/19 20:18
Start Date: 18/Nov/19 20:18
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-555191503
@Ardagan @kennknowles I think I'm stuck here. The failing Dataflow tests
seem to be using GCS filesystem, not the Local one. Unfortunately I'm not able
to rerun these locally as I don't have google cloud access setup and I'm not
able to see `RuntimeException` cause in Jenkins test results. Are you able to
rerun these locally and give me a hint about what's going wrong?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 34)
Time Spent: 2h 40m (was: 2.5h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Assignee: David Moravek
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=345199=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345199
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 18/Nov/19 10:16
Start Date: 18/Nov/19 10:16
Worklog Time Spent: 10m
Work Description: mxm commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-554949519
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 345199)
Time Spent: 2.5h (was: 2h 20m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=344597=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-344597
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 15/Nov/19 21:28
Start Date: 15/Nov/19 21:28
Worklog Time Spent: 10m
Work Description: Ardagan commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-554535454
Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 344597)
Time Spent: 2h 20m (was: 2h 10m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=344488=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-344488
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 15/Nov/19 18:59
Start Date: 15/Nov/19 18:59
Worklog Time Spent: 10m
Work Description: Ardagan commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-554486131
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 344488)
Time Spent: 2h 10m (was: 2h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=343678=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-343678
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 14/Nov/19 19:13
Start Date: 14/Nov/19 19:13
Worklog Time Spent: 10m
Work Description: Ardagan commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-554035981
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 343678)
Time Spent: 2h (was: 1h 50m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=341843=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-341843
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 12/Nov/19 11:38
Start Date: 12/Nov/19 11:38
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-552857660
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 341843)
Time Spent: 1h 50m (was: 1h 40m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=341779=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-341779
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 12/Nov/19 09:27
Start Date: 12/Nov/19 09:27
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-552809126
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 341779)
Time Spent: 1h 40m (was: 1.5h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=341327=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-341327
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 11/Nov/19 14:54
Start Date: 11/Nov/19 14:54
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-552477142
Run Java_Examples_Dataflow PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 341327)
Time Spent: 1.5h (was: 1h 20m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=340010=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-340010
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 16:01
Start Date: 07/Nov/19 16:01
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10028: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028#issuecomment-551143318
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 340010)
Time Spent: 1h 20m (was: 1h 10m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339988
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 15:14
Start Date: 07/Nov/19 15:14
Worklog Time Spent: 10m
Work Description: dmvk commented on pull request #10028: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10028
backport of https://github.com/apache/beam/pull/10024
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339988)
Time Spent: 1h 10m (was: 1h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339986
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 15:07
Start Date: 07/Nov/19 15:07
Worklog Time Spent: 10m
Work Description: dmvk commented on pull request #10024: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339986)
Time Spent: 1h (was: 50m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339946
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 13:41
Start Date: 07/Nov/19 13:41
Worklog Time Spent: 10m
Work Description: blackml commented on pull request #10024: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024#discussion_r343654563
##
File path:
sdks/java/core/src/test/java/org/apache/beam/sdk/io/LocalFileSystemTest.java
##
@@ -209,6 +209,30 @@ public void testMatchWithGlob() throws Exception {
containsInAnyOrder(expected.toArray(new String[expected.size()])));
}
+ @Test
+ public void testMatchInDirectoryRelativeWithGlob() throws Exception {
Review comment:
I just simplified this test, to be more straightforward.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339946)
Time Spent: 50m (was: 40m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339829=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339829
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 09:56
Start Date: 07/Nov/19 09:56
Worklog Time Spent: 10m
Work Description: dmvk commented on issue #10024: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024#issuecomment-551007211
cc @lukecwik
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339829)
Time Spent: 40m (was: 0.5h)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339828=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339828
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 09:55
Start Date: 07/Nov/19 09:55
Worklog Time Spent: 10m
Work Description: dmvk commented on pull request #10024: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024#discussion_r343561744
##
File path:
sdks/java/core/src/test/java/org/apache/beam/sdk/io/LocalFileSystemTest.java
##
@@ -209,6 +209,30 @@ public void testMatchWithGlob() throws Exception {
containsInAnyOrder(expected.toArray(new String[expected.size()])));
}
+ @Test
+ public void testMatchInDirectoryRelativeWithGlob() throws Exception {
Review comment:
this test is copy pasted from `testMatchWithGlob`, can you move it to a
parametrized method, so the code can be reused?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339828)
Time Spent: 0.5h (was: 20m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339822=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339822
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 09:39
Start Date: 07/Nov/19 09:39
Worklog Time Spent: 10m
Work Description: blackml commented on issue #10024: [BEAM-8568] Fixed
problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024#issuecomment-551001004
R: @dmvk
Hi, would appreciate a review.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 339822)
Time Spent: 20m (was: 10m)
> Local file system does not match relative path with wildcards
> -
>
> Key: BEAM-8568
> URL: https://issues.apache.org/jira/browse/BEAM-8568
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Affects Versions: 2.16.0
>Reporter: Ondrej Cerny
>Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> CWD structure:
> {code}
> src/test/resources/input/sometestfile.txt
> {code}
>
> Code:
> {code:java}
> input
> .apply(Create.of("src/test/resources/input/*))
> .apply(FileIO.matchAll())
> .apply(FileIO.readMatches())
> {code}
> The code above doesn't match any file starting Beam 2.16.0. The regression
> has been introduced in BEAM-7854.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8568) Local file system does not match relative path with wildcards
[
https://issues.apache.org/jira/browse/BEAM-8568?focusedWorklogId=339819=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339819
]
ASF GitHub Bot logged work on BEAM-8568:
Author: ASF GitHub Bot
Created on: 07/Nov/19 09:29
Start Date: 07/Nov/19 09:29
Worklog Time Spent: 10m
Work Description: blackml commented on pull request #10024: [BEAM-8568]
Fixed problem that LocalFileSystem no longer supports wil…
URL: https://github.com/apache/beam/pull/10024
…dcard relative paths.
LocalFileSystem no longer supports wildcard relative paths.
CWD structure:
src/test/resources/input/sometestfile.txt
Code:
input
.apply(Create.of("src/test/resources/input/*))
.apply(FileIO.matchAll())
.apply(FileIO.readMatches())
The code above doesn't match any file starting Beam 2.16.0. The regression
has been introduced in BEAM-7854.
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
Python | [