msievers13 commented on PR #28564: URL: https://github.com/apache/airflow/pull/28564#issuecomment-1376169609
@VladaZakharova we're using composer version 2.1.2 and airflow version 2.3.4 this is the latest version GCP allows us to upgrade to. As @apallerlamudi mentioned above, since we upgraded to this version, the gcs_to_bigquery operator has not worked for us for JSON files. We have several dags getting similar errors using this operator. The dags we have transferring CSV files are working the same as always. The API error our JSON dags are getting all say this: ` "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project."` and the error we see in the dag logs says something like this: Invalid field name "(some field)". Fields must contain only letters, numbers, and underscores, start with a letter or underscore, and be at most 300 characters long. We're setting the `gcp_conn_id` in the dag args and it's set the same was on all dags. Really not sure what has changed to be causing these errors with JSON files only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
