[
https://issues.apache.org/jira/browse/BEAM-6076?focusedWorklogId=166924&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-166924
]
ASF GitHub Bot logged work on BEAM-6076:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Nov/18 16:06
Start Date: 16/Nov/18 16:06
Worklog Time Spent: 10m
Work Description: lgajowy commented on issue #7066: [BEAM-6076] Fetch
only required fields for BigQuery Table
URL: https://github.com/apache/beam/pull/7066#issuecomment-439441290
I think this will settle the problem. As far as I can tell, the problematic
field was [estimated
rows](https://github.com/googleapis/google-cloud-java/blob/3e851558080b5173d0cc3363b96a84ff66a892e5/google-cloud-clients/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/StandardTableDefinition.java#L116)
field that sometimes seems to be null (hence NPE when calling `longValue()` on
it). The proposed change skips invoking that code so that we do not spot the
bug anymore (this "problematic" field was not needed either way).
Other than that such query will be a little bit more resource-efficient
since the response body is smaller now (not fetching all fields of Table JSON).
@apilloud could you take a look?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 166924)
Time Spent: 20m (was: 10m)
> NEXMark flakiness: NPE thrown from BigQuery client library once in a while
> --------------------------------------------------------------------------
>
> Key: BEAM-6076
> URL: https://issues.apache.org/jira/browse/BEAM-6076
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Lukasz Gajowy
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> It seems that once in a while the library that is used to connect to BigQuery
> throws exceptions like this:
> {code:java}
> Exception in thread "main"
> java.lang.NullPointerException
> at
> com.google.cloud.bigquery.StandardTableDefinition$StreamingBuffer.fromPb(StandardTableDefinition.java:116)
> at
> com.google.cloud.bigquery.StandardTableDefinition.fromPb(StandardTableDefinition.java:225)
> at com.google.cloud.bigquery.TableDefinition.fromPb(TableDefinition.java:155)
> at com.google.cloud.bigquery.TableInfo$BuilderImpl.<init>(TableInfo.java:183)
> at com.google.cloud.bigquery.Table.fromPb(Table.java:593)
> at com.google.cloud.bigquery.BigQueryImpl.getTable(BigQueryImpl.java:410)
> at
> org.apache.beam.sdk.testutils.publishing.BigQueryClient.createTableIfNotExists(BigQueryClient.java:74)
> at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:184)
> at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:148)
> at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:98)
> at org.apache.beam.sdk.nexmark.Main.main(Main.java:423){code}
> Similar error found in the network:
> [https://github.com/googleapis/google-cloud-java/issues/1689]
> +Example logs:+
>
> [https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Spark/1085/console]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
