Valentyn Tymofieiev created BEAM-7326:
-----------------------------------------
Summary: Document that Beam BigQuery IO expects users to pass
base64-encoded bytes, and BQ IO serves base64-encoded bytes to the user.
Key: BEAM-7326
URL: https://issues.apache.org/jira/browse/BEAM-7326
Project: Beam
Issue Type: Bug
Components: io-java-gcp, io-python-gcp
Reporter: Valentyn Tymofieiev
BYTES is one of the Datatypes supported by Google Cloud BigQuery, and Apache
Beam BigQuery IO connector.
Current implementation of BigQuery connector in Java and Python SDKs expects
that users base64-encode bytes before passing them to BigQuery IO, see
discussion on dev: [1]
This needs to be reflected in public documentation, see [2-4]
cc: [~juta] [~chamikara] [~pabloem]
cc: [~rebo] [~kedin] FYI and to advise whether similar action needs to be done
for Go SDK and/or Beam SQL.
[1]
https://lists.apache.org/thread.html/f35c836887014e059527ed1a806e730321e2f9726164a3030575f455@%3Cdev.beam.apache.org%3E
[2] https://beam.apache.org/documentation/io/built-in/google-bigquery/
[3]
https://beam.apache.org/releases/pydoc/2.12.0/apache_beam.io.gcp.bigquery.html
[4]
https://beam.apache.org/releases/javadoc/2.12.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
