ahmedabu98 commented on code in PR #25685:
URL: https://github.com/apache/beam/pull/25685#discussion_r1146282748
##########
sdks/python/apache_beam/io/gcp/bigquery_tools.py:
##########
@@ -103,6 +107,19 @@
_DATASET_PATTERN = r'\w{1,1024}'
_TABLE_PATTERN = r'[\p{L}\p{M}\p{N}\p{Pc}\p{Pd}\p{Zs}$]{1,1024}'
+BIGQUERY_TYPE_TO_PYTHON_TYPE = {
+ "STRING": str,
+ "BOOL": bool,
+ "BOOLEAN": bool,
+ "BYTES": bytes,
+ "INT64": np.int64,
+ "INTEGER": np.int64,
+ "FLOAT64": np.float64,
+ "FLOAT": np.float64,
+ "NUMERIC": decimal.Decimal,
+ "TIMESTAMP": apache_beam.utils.timestamp.Timestamp,
Review Comment:
Yup, I only included the BQ types that were supported by Beam types. I'll
create a tracking issue for supporting other types. As it stands today, these
are the only types users would be able to write with the Storage Write xlang
connector.
The other BQ types can be represented as strings, so a possible workaround
would be to add a new configuration field for the BQ schema and pass it to
Java. In the Java SchemaTransform we can use the input schema instead of
`.useBeamSchema()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
