[ https://issues.apache.org/jira/browse/BEAM-12865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anonymous updated BEAM-12865: ----------------------------- Status: Triage Needed (was: Resolved) > Allow customising batch duration when streaming with WriteToBigQuery > -------------------------------------------------------------------- > > Key: BEAM-12865 > URL: https://issues.apache.org/jira/browse/BEAM-12865 > Project: Beam > Issue Type: New Feature > Components: io-py-gcp > Affects Versions: Not applicable > Reporter: Quentin Sommer > Priority: P2 > Fix For: 2.36.0 > > Time Spent: 18h > Remaining Estimate: 0h > > Hi, > We allow customising the {{batch_size}} when streaming to BigQuery but the > batch duration (used by {{GroupIntoBatches}}) is set to > {{DEFAULT_BATCH_BUFFERING_DURATION_LIMIT_SEC}} (0.2) > I'd like to add the option to specify the batch duration to allow better > batching for scenarios with little data throughput. > It will use the {{triggering_frequency}} param already used when doing batch > file loads -- This message was sent by Atlassian Jira (v8.20.10#820010)