[
https://issues.apache.org/jira/browse/AIRFLOW-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221188#comment-16221188
]
Maximilian Roos commented on AIRFLOW-1503:
------------------------------------------
Is there a reason we need to put this connection in the DB, rather than using
`google_cloud_default`? I don't think there's any additional information that
the connection params provide.
This is an example of what we have to do in order to run BQ tasks without
manually configuring through the UI (or let me know if we're making a mistake):
https://stackoverflow.com/questions/45626406/assertionerror-internal-no-default-project-is-specified
> AssertionError: INTERNAL: No default project is specified
> ---------------------------------------------------------
>
> Key: AIRFLOW-1503
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1503
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
> Affects Versions: Airflow 1.8
> Environment: Unix platform
> Reporter: chaitanya
> Priority: Minor
> Labels: beginner
>
> Hi ,
> New to airflow. Tried to run BigQuery query and store the result in another
> table. Getting the following error.
> Please let me know where to default project.
> Code:
> sql_bigquery = BigQueryOperator(
> task_id='sql_bigquery',
> use_legacy_sql=False,
> write_disposition='WRITE_TRUNCATE',
> allow_large_results=True,
> bql='''
> #standardSQL
> SELECT ID, Name, Group, Mark, RATIO_TO_REPORT(Mark)
> OVER(PARTITION BY Group) AS percent FROM `tensile-site-168620.temp.marks`
> ''',
> destination_dataset_table='temp.percentage',
> dag=dag
> )
> Error Message:
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 28, in <module>
> args.func(args)
> File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 585,
> in test
> ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
> File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 53,
> in wrapper
> result = func(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1374,
> in run
> result = task_copy.execute(context=context)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/bigquery_operator.py",
> line 82, in execute
> self.allow_large_results, self.udf_config, self.use_legacy_sql)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/bigquery_hook.py",
> line 228, in run_query
> default_project_id=self.project_id)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/bigquery_hook.py",
> line 917, in _split_tablename
> assert default_project_id is not None, "INTERNAL: No default project is
> specified"
> AssertionError: INTERNAL: No default project is specified
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
