[
https://issues.apache.org/jira/browse/BEAM-14273?focusedWorklogId=763914&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763914
]
ASF GitHub Bot logged work on BEAM-14273:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Apr/22 22:06
Start Date: 28/Apr/22 22:06
Worklog Time Spent: 10m
Work Description: ahmedabu98 commented on code in PR #17431:
URL: https://github.com/apache/beam/pull/17431#discussion_r861344279
##########
sdks/python/apache_beam/io/gcp/bigquery.py:
##########
@@ -2190,6 +2190,20 @@ def expand(self, pcoll):
'A schema must be provided when writing to BigQuery using '
'Avro based file loads')
+ if self.schema and type(self.schema) is dict:
+
+ def find_in_nested_dict(schema):
+ for field in schema['fields']:
+ if field['type'] == 'JSON':
+ raise ValueError(
+ 'Found JSON type in table schema. JSON data '
+ 'insertion is currently not supported with '
+ 'FILE_LOADS write method.')
+ elif field['type'] == 'STRUCT':
+ find_in_nested_dict(field)
Review Comment:
i don't like the sound of it either, but thought of suggesting because the
link doesn't add that nuance (in case the user wants to know why it's not
supported yet).
we could just mention that it is supported for the other write methods and
put the link for more details?
Issue Time Tracking
-------------------
Worklog Id: (was: 763914)
Time Spent: 4.5h (was: 4h 20m)
> Update BQ connector to support new JSON type (Python)
> -----------------------------------------------------
>
> Key: BEAM-14273
> URL: https://issues.apache.org/jira/browse/BEAM-14273
> Project: Beam
> Issue Type: New Feature
> Components: io-py-gcp
> Reporter: Ahmed Abualsaud
> Assignee: Ahmed Abualsaud
> Priority: P2
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> BQ has a new JSON type that is defined here:
> [https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#json_type]
> We should update Beam BQ Java and Python connectors to support that for
> various read methods (export jobs, storage API) and write methods (load jobs,
> streaming inserts, storage API).
> We should also add integration tests that exercise reading from /writing to
> BQ tables with columns that has JSON type.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
