Kenneth Jung created BEAM-11226:
-----------------------------------
Summary: Support larger tables in BigQuery storage API bounded
source
Key: BEAM-11226
URL: https://issues.apache.org/jira/browse/BEAM-11226
Project: Beam
Issue Type: Improvement
Components: io-java-gcp
Affects Versions: 2.25.0
Reporter: Kenneth Jung
The BigQuery storage API client specifies a default deadline of 10 minutes for
calls to
[CreateReadSession](https://cloud.google.com/bigquery/docs/reference/storage#create_a_session).
However, for large tables, this operation can take longer than ten minutes,
and as a result, read session creation simply fails for these tables. Some Beam
runners, such as Google Cloud Dataflow, allow much longer for initial split
calls on bounded sources -- 2 hours, in the case of Dataflow. We can support
larger BigQuery tables with the storage API source by raising the client-side
deadline for CreateReadSession to match this 2-hour limit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
