This is an automated email from the ASF dual-hosted git repository.
ahmedabualsaud pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new d6068ad66b0 Updating Storage API Autosharding documentation to include
that it doesn't work on Runner V2 (#28233)
d6068ad66b0 is described below
commit d6068ad66b0a28a6dd628bb8ef48f1e2182acb4b
Author: Ahmed Abualsaud <[email protected]>
AuthorDate: Mon Sep 18 18:27:08 2023 +0000
Updating Storage API Autosharding documentation to include that it doesn't
work on Runner V2 (#28233)
* add documentation
* doc for python too
---
.../www/site/content/en/documentation/io/built-in/google-bigquery.md | 4 ++++
1 file changed, 4 insertions(+)
diff --git
a/website/www/site/content/en/documentation/io/built-in/google-bigquery.md
b/website/www/site/content/en/documentation/io/built-in/google-bigquery.md
index 24314dc1180..eae98b84d2c 100644
--- a/website/www/site/content/en/documentation/io/built-in/google-bigquery.md
+++ b/website/www/site/content/en/documentation/io/built-in/google-bigquery.md
@@ -788,6 +788,8 @@ BigQuery Storage Write API for Python SDK currently has
some limitations on supp
{{< paragraph class="language-py" >}}
**Note:** If you want to run WriteToBigQuery with Storage Write API from the
source code, you need to run `./gradlew
:sdks:java:io:google-cloud-platform:expansion-service:build` to build the
expansion-service jar. If you are running from a released Beam SDK, the jar
will already be included.
+**Note:** Auto sharding is not currently supported for Python's Storage Write
API.
+
{{< /paragraph >}}
#### Exactly-once semantics
@@ -877,6 +879,8 @@ explicitly enable this using
[`withAutoSharding`](https://beam.apache.org/releas
***Note:*** `STORAGE_WRITE_API` will default to dynamic sharding when
`numStorageWriteApiStreams` is set to 0 or is unspecified.
+
+***Note:*** Auto sharding with `STORAGE_WRITE_API` is supported on Dataflow's
legacy runner, but **not** on Runner V2
{{< /paragraph >}}
When using `STORAGE_WRITE_API`, the PCollection returned by
