[
https://issues.apache.org/jira/browse/BEAM-8889?focusedWorklogId=617754&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-617754
]
ASF GitHub Bot logged work on BEAM-8889:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Jul/21 19:00
Start Date: 01/Jul/21 19:00
Worklog Time Spent: 10m
Work Description: veblush commented on a change in pull request #14817:
URL: https://github.com/apache/beam/pull/14817#discussion_r662526265
##########
File path:
sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java
##########
@@ -763,7 +751,7 @@ public void onSuccess(StorageObject response, HttpHeaders
httpHeaders)
@Override
public void onFailure(GoogleJsonError e, HttpHeaders httpHeaders)
throws IOException {
IOException ioException;
- if (errorExtractor.itemNotFound(e)) {
+ if (e.getCode() == HttpStatusCodes.STATUS_CODE_NOT_FOUND) {
Review comment:
This is not possible anymore because ApiErrorExtrator doesn't support
GoogleJsonError anymore by
https://github.com/GoogleCloudDataproc/hadoop-connectors/pull/327. I copied the
same routine from the old ApiErrorExtrator to do the same thing.
##########
File path:
sdks/java/extensions/google-cloud-platform-core/src/test/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilTest.java
##########
@@ -772,7 +773,11 @@ public void testGCSChannelCloseIdempotent() throws
IOException {
GoogleCloudStorageReadOptions.builder().setFastFailOnNotFound(false).build();
SeekableByteChannel channel =
new GoogleCloudStorageReadChannel(
- null, "dummybucket", "dummyobject", null, new
ClientRequestHelper<>(), readOptions);
+ null,
+ new StorageResourceId("dummybucket", "dummyobject"),
Review comment:
done!
##########
File path:
sdks/java/extensions/google-cloud-platform-core/src/test/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilTest.java
##########
@@ -772,7 +773,11 @@ public void testGCSChannelCloseIdempotent() throws
IOException {
GoogleCloudStorageReadOptions.builder().setFastFailOnNotFound(false).build();
SeekableByteChannel channel =
new GoogleCloudStorageReadChannel(
Review comment:
good point. done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 617754)
Remaining Estimate: 126h 40m (was: 126h 50m)
Time Spent: 41h 20m (was: 41h 10m)
> Make GcsUtil use GoogleCloudStorage
> -----------------------------------
>
> Key: BEAM-8889
> URL: https://issues.apache.org/jira/browse/BEAM-8889
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Affects Versions: 2.16.0
> Reporter: Esun Kim
> Assignee: VASU NORI
> Priority: P2
> Labels: gcs
> Fix For: 2.22.0
>
> Original Estimate: 168h
> Time Spent: 41h 20m
> Remaining Estimate: 126h 40m
>
> [GcsUtil|https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java]
> is a primary class to access Google Cloud Storage on Apache Beam. Current
> implementation directly creates GoogleCloudStorageReadChannel and
> GoogleCloudStorageWriteChannel by itself to read and write GCS data rather
> than using
> [GoogleCloudStorage|https://github.com/GoogleCloudPlatform/bigdata-interop/blob/master/gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorage.java]
> which is an abstract class providing basic IO capability which eventually
> creates channel objects. This request is about updating GcsUtil to use
> GoogleCloudStorage to create read and write channel, which is expected
> flexible because it can easily pick up the new change; e.g. new channel
> implementation using new protocol without code change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
