[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336308=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336308 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 17:14 Start Date: 30/Oct/19 17:14 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336308) Time Spent: 2h 40m (was: 2.5h) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Fix For: 2.18.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336303=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336303 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 17:07 Start Date: 30/Oct/19 17:07 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340746507 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: This was a problem with the HTTP client. (`PubsubMessage::decodeData` and `Base64` Javadocs clearly state that they will return null if the input is null, but this behavior relied on Guava's `BaseEncoding` implementation, which AFAICT never made such a promise.) Apparently this was fixed in version 1.30.0 of the HTTP client: https://github.com/googleapis/google-http-java-client/pull/644 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336303) Time Spent: 2.5h (was: 2h 20m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336285=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336285 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:52 Start Date: 30/Oct/19 16:52 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340736852 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: It looks like Guavas 20 and Guava 26 Base64 didn't allow nulls so checking getData should be fine replacement for prior logic. It may also be that the test is wrong and should have really called encodeData with an empty byte array to get the no data case for testing but it is unclear from the pubsub documentation what the contents of the field maybe so this is the safer route. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336285) Time Spent: 2h 20m (was: 2h 10m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336283=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336283 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:50 Start Date: 30/Oct/19 16:50 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340736852 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: It looks like Guavas 20 and Guava 26 Base64 didn't allow nulls. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336283) Time Spent: 2h 10m (was: 2h) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336273=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336273 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:40 Start Date: 30/Oct/19 16:40 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340731086 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: It looks like the Base64 library version went through several changes from [1.27](https://github.com/googleapis/google-http-java-client/blob/v1.27.0/google-http-client/src/main/java/com/google/api/client/util/Base64.java) -> [1.28](https://github.com/googleapis/google-http-java-client/blob/v1.28.0/google-http-client/src/main/java/com/google/api/client/util/Base64.java), mainly swapping from commons codec to Guava's version of base64 encoding. Maybe Guava made a backwards incompatible change or the override from Guava 26.0-android back to 20.0 that we are now forcing because of the upgrade is causing the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336273) Time Spent: 2h (was: 1h 50m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336263=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336263 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:32 Start Date: 30/Oct/19 16:32 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340726421 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: I downloaded the source for both versions ([1.27](https://repo1.maven.org/maven2/com/google/apis/google-api-services-pubsub/v1-rev20181105-1.27.0/google-api-services-pubsub-v1-rev20181105-1.27.0-sources.jar), [1.28](https://repo1.maven.org/maven2/com/google/apis/google-api-services-pubsub/v1-rev20181213-1.28.0/google-api-services-pubsub-v1-rev20181213-1.28.0-sources.jar)) and they both invoke the Base64 library: - decodeData() uses com.google.api.client.util.Base64.decodeBase64 - encodeData() uses com.google.api.client.util.Base64.encodeBase64URLSafeString It looks like there was a change in the Base64 library implementation since it now doesn't take null and return null. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336263) Time Spent: 1h 50m (was: 1h 40m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336262=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336262 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:31 Start Date: 30/Oct/19 16:31 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340726421 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: I downloaded the source for both versions ([1.27](https://repo1.maven.org/maven2/com/google/apis/google-api-services-pubsub/v1-rev20181105-1.27.0/google-api-services-pubsub-v1-rev20181105-1.27.0-sources.jar), [1.28](https://repo1.maven.org/maven2/com/google/apis/google-api-services-pubsub/v1-rev20181213-1.28.0/google-api-services-pubsub-v1-rev20181213-1.28.0-sources.jar)) and they both invoke the Base64 library: decodeData() uses com.google.api.client.util.Base64.decodeBase64 encodeData() uses com.google.api.client.util.Base64.encodeBase64URLSafeString It looks like there was a change in the Base64 library implementation since it now doesn't take null and return null. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336262) Time Spent: 1h 40m (was: 1.5h) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336254=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336254 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:17 Start Date: 30/Oct/19 16:17 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340718765 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: I'm assuming [PubsubJsonClientTest.pullOneMessageWithNoData](https://github.com/apache/beam/blob/1deab4ce2ef5d721389e29def6acbe399fccdc9a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClientTest.java#L124) started to fail. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336254) Time Spent: 1.5h (was: 1h 20m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336251 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:15 Start Date: 30/Oct/19 16:15 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#discussion_r340717350 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java ## @@ -169,7 +169,7 @@ public int publish(TopicPath topic, List outgoingMessages) thro @Nullable Map attributes = pubsubMessage.getAttributes(); // Payload. - byte[] elementBytes = pubsubMessage.decodeData(); + byte[] elementBytes = pubsubMessage.getData() == null ? null : pubsubMessage.decodeData(); Review comment: I don't think we should be modifying the code to make a test pass. Is the way the test is setting this up is bad or was there an actual API change which now allows getData to be null? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336251) Time Spent: 1h 20m (was: 1h 10m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=336248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-336248 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 30/Oct/19 16:11 Start Date: 30/Oct/19 16:11 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-547985634 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 336248) Time Spent: 1h 10m (was: 1h) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=335305=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-335305 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 29/Oct/19 00:35 Start Date: 29/Oct/19 00:35 Worklog Time Spent: 10m Work Description: ibzib commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-547205391 @lukecwik The version choices indicated above required one minor change to pass tests. PTAL (testExampleEchoPipeline is a [known flake](https://issues.apache.org/jira/browse/BEAM-4637).) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 335305) Time Spent: 1h (was: 50m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=335189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-335189 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 28/Oct/19 21:02 Start Date: 28/Oct/19 21:02 Worklog Time Spent: 10m Work Description: ibzib commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-547144364 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 335189) Time Spent: 50m (was: 40m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: runner-dataflow >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=328253=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328253 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 15/Oct/19 01:22 Start Date: 15/Oct/19 01:22 Worklog Time Spent: 10m Work Description: ibzib commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-541994450 I made some more conservative version choices. The diff, besides the obvious/expected 1.27->1.28 changes, is pretty minimal, as follows. `google-http-client` picked up some new dependencies. Before: ``` ||+--- com.google.http-client:google-http-client:1.27.0 |||+--- com.google.code.findbugs:jsr305:3.0.2 |||+--- com.google.guava:guava:20.0 |||+--- org.apache.httpcomponents:httpclient:4.5.5 ||||+--- org.apache.httpcomponents:httpcore:4.4.9 ||||+--- commons-logging:commons-logging:1.2 ||||\--- commons-codec:commons-codec:1.10 |||\--- com.google.j2objc:j2objc-annotations:1.1 ``` After: ``` ||+--- com.google.http-client:google-http-client:1.28.0 |||+--- com.google.code.findbugs:jsr305:3.0.2 |||+--- com.google.guava:guava:26.0-android -> 20.0 |||+--- com.google.j2objc:j2objc-annotations:1.1 |||+--- io.opencensus:opencensus-api:0.18.0 ||||\--- io.grpc:grpc-context:1.14.0 |||\--- io.opencensus:opencensus-contrib-http-util:0.18.0 ||| +--- io.opencensus:opencensus-api:0.18.0 (*) ||| \--- com.google.guava:guava:20.0 ``` The existing `org.apache.httpcomponents:httpclient` was added under `google-api-client`: ``` ||+--- com.google.http-client:google-http-client-apache:2.0.0 |||+--- com.google.http-client:google-http-client:1.28.0 (*) |||\--- org.apache.httpcomponents:httpclient:4.5.5 ||| +--- org.apache.httpcomponents:httpcore:4.4.9 ||| +--- commons-logging:commons-logging:1.2 ||| \--- commons-codec:commons-codec:1.10 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 328253) Time Spent: 40m (was: 0.5h) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: sdk-java-core >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=328183=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328183 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 14/Oct/19 22:14 Start Date: 14/Oct/19 22:14 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-541953652 Pointing google-cloud-bigquery at 1.97.0 picks up the google-cloud-clients [0.115.0-alpha bom](https://repo.maven.apache.org/maven2/com/google/cloud/google-cloud-clients/0.115.0-alpha/google-cloud-clients-0.115.0-alpha.pom). The bom defines a bunch of dependency versions of compatible libraries. Also, the versions of libraries that you selected may not be compatible with the related google support libraries like gax, auth, Please provide the transitive dependency diff including version overrides where we squash a resolved version for an older/newer version. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 328183) Time Spent: 0.5h (was: 20m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: sdk-java-core >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=328171=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328171 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 14/Oct/19 21:54 Start Date: 14/Oct/19 21:54 Worklog Time Spent: 10m Work Description: ibzib commented on issue #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792#issuecomment-541944256 Judging by [here](https://repo.maven.apache.org/maven2/com/google/cloud/google-cloud-bigquery/), it looks like these bigquery version are way off from $google_clients_version, so I'm guessing they're semantically different? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 328171) Time Spent: 20m (was: 10m) > Upgrade Dataflow Java Client API > > > Key: BEAM-8398 > URL: https://issues.apache.org/jira/browse/BEAM-8398 > Project: Beam > Issue Type: Task > Components: sdk-java-core >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8398) Upgrade Dataflow Java Client API
[ https://issues.apache.org/jira/browse/BEAM-8398?focusedWorklogId=328150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-328150 ] ASF GitHub Bot logged work on BEAM-8398: Author: ASF GitHub Bot Created on: 14/Oct/19 21:12 Start Date: 14/Oct/19 21:12 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #9792: [BEAM-8398] Upgrade Dataflow Java Client API URL: https://github.com/apache/beam/pull/9792 We need to pick up a newer version of the Dataflow API to add support for the new `worker_region` and `worker_zone` flags ([BEAM-8251](https://issues.apache.org/jira/browse/BEAM-8251)). Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[