Peter Brumblay created BEAM-4369:
------------------------------------
Summary: BigQueryIO.readTableRows() throws NullPointerException
with TableReference and decorator
Key: BEAM-4369
URL: https://issues.apache.org/jira/browse/BEAM-4369
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.4.0
Reporter: Peter Brumblay
Assignee: Chamikara Jayalath
When attaching a partition decorator to the table id, readTableRows() throws a
NullPointerException.
{code:java}
TableReference tableRef = BigQueryHelpers.parseTableSpec(options.getBqTable());
tableRef.setTableId(tableRef.getTableId() + "$" + options.getPartitionValue());
//Regardless of whether this should work with the table decorator,
//it should not throw a NullPointerException
PCollection<TableRow> rowsByPartition = p.apply("Read live data",
BigQueryIO.readTableRows().from(tableRef));
{code}
Exception:
Exception in thread "main"
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
java.lang.NullPointerException
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at
com.fearlesstg.CannotReadFromPartitionedTable.main(CannotReadFromPartitionedTable.java:26)
Caused by: java.lang.NullPointerException
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableSource.getEstimatedSizeBytes(BigQueryTableSource.java:108)
at
org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:207)
at
org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:87)
at
org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:62)
Detailed example:
[https://github.com/pbrumblay/bigqueryioproblems/blob/master/src/main/java/com/fearlesstg/CannotReadFromPartitionedTable.java]
Personally, I would also like this pattern to be supported as intended and not
have to construct queries with IN / WHERE clauses to pull partitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
