kaxil commented on issue #26234:
URL: https://github.com/apache/airflow/issues/26234#issuecomment-1240671881
@abhishekbhakat pinged me that it worked fine with inline, logs below:
```
worker-1
*** Reading local file:
/usr/local/airflow/logs/dag_id=postgres_proc/run_id=manual__2022-09-08T12:45:52.814823+00:00/task_id=task_1/attempt=1.log
[2022-09-08 12:45:54,591] {plugins_manager.py:292} DEBUG - Plugins are
already loaded. Skipping.
[2022-09-08 12:45:54,594] {__init__.py:50} DEBUG - Loading core task runner:
StandardTaskRunner
[2022-09-08 12:45:54,896] {taskinstance.py:1115} DEBUG -
previous_execution_date was called
[2022-09-08 12:45:54,928] {base_task_runner.py:70} DEBUG - Planning to run
as the user
[2022-09-08 12:45:54,950] {taskinstance.py:879} DEBUG - Refreshing
TaskInstance <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [queued]> from DB
[2022-09-08 12:45:54,991] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Trigger Rule' PASSED: True, The task instance did not have any
upstream tasks.
[2022-09-08 12:45:55,027] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Task Instance Not Running' PASSED: True, Task is not in running
state.
[2022-09-08 12:45:55,028] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Previous Dagrun State' PASSED: True, The task did not have
depends_on_past set.
[2022-09-08 12:45:55,029] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Not In Retry Period' PASSED: True, The task instance was not marked
for retrying.
[2022-09-08 12:45:55,029] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Task Instance State' PASSED: True, Task state queued was valid.
[2022-09-08 12:45:55,029] {taskinstance.py:1179} INFO - Dependencies all met
for <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [queued]>
[2022-09-08 12:45:55,047] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Trigger Rule' PASSED: True, The task instance did not have any
upstream tasks.
[2022-09-08 12:45:55,081] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Pool Slots Available' PASSED: True, There are enough open slots in
default_pool to execute the task
[2022-09-08 12:45:55,082] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Previous Dagrun State' PASSED: True, The task did not have
depends_on_past set.
[2022-09-08 12:45:55,082] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Not In Retry Period' PASSED: True, The task instance was not marked
for retrying.
[2022-09-08 12:45:55,083] {taskinstance.py:1189} DEBUG - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [queued]>
dependency 'Task Concurrency' PASSED: True, Task concurrency is not set.
[2022-09-08 12:45:55,083] {taskinstance.py:1179} INFO - Dependencies all met
for <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [queued]>
[2022-09-08 12:45:55,084] {taskinstance.py:1376} INFO -
--------------------------------------------------------------------------------
[2022-09-08 12:45:55,084] {taskinstance.py:1377} INFO - Starting attempt 1
of 1
[2022-09-08 12:45:55,084] {taskinstance.py:1378} INFO -
--------------------------------------------------------------------------------
[2022-09-08 12:45:55,155] {events.py:43} DEBUG - session flush listener:
added ['running'] unchanged () deleted ['queued'] - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [running]>
[2022-09-08 12:45:55,156] {listener.py:85} DEBUG - OpenLineage listener got
notification about task instance start
[2022-09-08 12:45:55,160] {manager.py:79} DEBUG - extractor for <class
'airflow.providers.***.operators.***.PostgresOperator'> is <class
'openlineage.airflow.extractors.***_extractor.PostgresExtractor'>
[2022-09-08 12:45:55,160] {manager.py:35} DEBUG - Using extractor
PostgresExtractor task_type=PostgresOperator airflow_dag_id=***_proc
task_id=task_1 airflow_run_id=manual__2022-09-08T12:45:52.814823+00:00
[2022-09-08 12:45:55,161] {postgres_extractor.py:41} DEBUG - Sending SQL to
parser: CALL display_message('ola!');
[2022-09-08 12:45:55,178] {__init__.py:27} ERROR - SQL parser failed: sql
parser error: Expected an SQL statement, found: CALL
[2022-09-08 12:45:55,179] {postgres_extractor.py:43} DEBUG - Got meta None
[2022-09-08 12:45:55,180] {manager.py:42} DEBUG - Found task metadata for
operation task_1: TaskMetadata(name='***_proc.task_1', inputs=[], outputs=[],
run_facets={}, job_facets={'sql':
SqlJobFacet(_producer='https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow',
_schemaURL='https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/SqlJobFacet',
query="CALL display_message('ola!');")})
[2022-09-08 12:45:55,202] {factory.py:122} ERROR - Did not find
openlineage.yml and OPENLINEAGE_URL is not set
[2022-09-08 12:45:55,203] {factory.py:43} WARNING - Couldn't initialize
transport; will print events to console.
[2022-09-08 12:45:55,206] {console.py:21} INFO - {"eventTime":
"2022-09-08T12:45:55.030220Z", "eventType": "START", "inputs": [], "job":
{"facets": {"sql": {"_producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL":
"https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/SqlJobFacet",
"query": "CALL display_message('ola!');"}}, "name": "***_proc.task_1",
"namespace": "default"}, "outputs": [], "producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"run": {"facets": {"airflow_runArgs": {"_producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL":
"https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/BaseFacet",
"externalTrigger": true}, "airflow_version": {"_producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL": "https://raw.
githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/BaseFacet",
"airflowVersion": "2.3.3+astro.1", "openlineageAirflowVersion": "0.10.0",
"operator": "airflow.providers.***.operators.***.PostgresOperator", "taskInfo":
"{'_BaseOperator__init_kwargs': {'task_id': 'task_1', 'sql': \"CALL
display_message('ola!');\", '***_conn_id': 'metadata'},
'_BaseOperator__from_mapped': False, 'task_id': 'task_1', 'task_group':
<weakproxy at 0x402c0724f0 to TaskGroup at 0x402c05b220>, 'owner': 'airflow',
'email': None, 'email_on_retry': True, 'email_on_failure': True,
'execution_timeout': None, 'on_execute_callback': None, 'on_failure_callback':
None, 'on_success_callback': None, 'on_retry_callback': None,
'_pre_execute_hook': None, '_post_execute_hook': None, 'executor_config': {},
'run_as_user': None, 'retries': 0, 'queue': 'default', 'pool': 'default_pool',
'pool_slots': 1, 'sla': None, 'trigger_rule': <TriggerRule.ALL_SUCCESS:
'all_success'>, 'depends_on_past': F
alse, 'ignore_first_depends_on_past': True, 'wait_for_downstream': False,
'retry_delay': datetime.timedelta(seconds=300), 'retry_exponential_backoff':
False, 'max_retry_delay': None, 'params': <airflow.models.param.ParamsDict
object at 0x402c06f850>, 'priority_weight': 1, 'weight_rule':
<WeightRule.DOWNSTREAM: 'downstream'>, 'resources': None,
'max_active_tis_per_dag': None, 'do_xcom_push': True, 'doc_md': None,
'doc_json': None, 'doc_yaml': None, 'doc_rst': None, 'doc': None,
'upstream_task_ids': set(), 'downstream_task_ids': set(), 'start_date':
DateTime(2022, 9, 1, 0, 0, 0, tzinfo=Timezone('UTC')), 'end_date': None,
'_dag': <DAG: ***_proc>, '_log': <Logger airflow.task.operators (DEBUG)>,
'inlets': [], 'outlets': [], '_inlets': [], '_outlets': [],
'_BaseOperator__instantiated': True, 'sql': \"CALL display_message('ola!');\",
'***_conn_id': 'metadata', 'autocommit': False, 'parameters': None, 'database':
None, 'runtime_parameters': None, 'hook': None}"}, "nominalTime": {"_producer
":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL":
"https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/NominalTimeRunFacet",
"nominalStartTime": "2022-09-08T12:45:52.814823Z"}, "parentRun": {"_producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL":
"https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/ParentRunFacet",
"job": {"name": "***_proc", "namespace": "default"}, "run": {"runId":
"762744a2-486e-330c-b364-8fc0d73f3413"}}}, "runId":
"f6b6cdfa-4112-4425-95c7-511078ef5cc0"}}
[2022-09-08 12:45:55,213] {taskinstance.py:1397} INFO - Executing
<Task(PostgresOperator): task_1> on 2022-09-08 12:45:52.814823+00:00
[2022-09-08 12:45:55,220] {standard_task_runner.py:52} INFO - Started
process 100 to run task
[2022-09-08 12:45:55,226] {standard_task_runner.py:79} INFO - Running:
['airflow', 'tasks', 'run', '***_proc', 'task_1',
'manual__2022-09-08T12:45:52.814823+00:00', '--job-id', '20', '--raw',
'--subdir', 'DAGS_FOLDER/stored_proc.py', '--cfg-path', '/tmp/tmp_g5acyu3',
'--error-file', '/tmp/tmphoqzacne']
[2022-09-08 12:45:55,227] {standard_task_runner.py:80} INFO - Job 20:
Subtask task_1
[2022-09-08 12:45:55,229] {cli_action_loggers.py:66} DEBUG - Calling
callbacks: [<function default_action_log at 0x40263a5b80>]
[2022-09-08 12:45:55,375] {warnings.py:109} WARNING -
/usr/local/lib/python3.9/site-packages/airflow/configuration.py:528:
DeprecationWarning: The sql_alchemy_conn option in [core] has been moved to the
sql_alchemy_conn option in [database] - the old setting has been used, but
please update your config.
option = self._get_environment_variables(deprecated_key,
deprecated_section, key, section)
[2022-09-08 12:45:55,376] {settings.py:398} DEBUG - Disposing DB connection
pool (PID 100)
[2022-09-08 12:45:55,377] {settings.py:260} DEBUG - Setting up DB connection
pool (PID 100)
[2022-09-08 12:45:55,378] {settings.py:330} DEBUG -
settings.prepare_engine_args(): Using NullPool
[2022-09-08 12:45:55,905] {taskinstance.py:1115} DEBUG -
previous_execution_date was called
[2022-09-08 12:45:55,933] {task_command.py:371} INFO - Running
<TaskInstance: postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00
[running]> on host worker-1
[2022-09-08 12:45:55,936] {taskinstance.py:879} DEBUG - Refreshing
TaskInstance <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [running]> from DB
[2022-09-08 12:45:56,024] {events.py:43} DEBUG - session flush listener:
added () unchanged ['running'] deleted () - <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [running]>
[2022-09-08 12:45:56,198] {taskinstance.py:1115} DEBUG -
previous_execution_date was called
[2022-09-08 12:45:56,230] {taskinstance.py:951} DEBUG - Clearing XCom data
[2022-09-08 12:45:56,716] {taskinstance.py:1589} INFO - Exporting the
following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=***_proc
AIRFLOW_CTX_TASK_ID=task_1
AIRFLOW_CTX_EXECUTION_DATE=2022-09-08T12:45:52.814823+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-09-08T12:45:52.814823+00:00
[2022-09-08 12:45:56,717] {__init__.py:146} DEBUG - Preparing lineage inlets
and outlets
[2022-09-08 12:45:56,719] {__init__.py:190} DEBUG - inlets: [], outlets: []
[2022-09-08 12:45:56,719] {dbapi.py:200} DEBUG - Executing 1 statements
[2022-09-08 12:45:56,880] {base.py:68} INFO - Using connection ID 'metadata'
for task execution.
[2022-09-08 12:45:56,981] {dbapi.py:231} INFO - Running statement: CALL
display_message('ola!');, parameters: None
[2022-09-08 12:45:57,008] {dbapi.py:239} INFO - Rows affected: 1
[2022-09-08 12:45:57,009] {postgres.py:94} INFO - NOTICE: Procedure
Parameter: ola!
[2022-09-08 12:45:57,011] {__init__.py:107} DEBUG - Lineage called with
inlets: [], outlets: []
[2022-09-08 12:45:57,012] {taskinstance.py:879} DEBUG - Refreshing
TaskInstance <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [running]> from DB
[2022-09-08 12:45:57,183] {taskinstance.py:1426} DEBUG - Clearing
next_method and next_kwargs.
[2022-09-08 12:45:57,184] {taskinstance.py:1415} INFO - Marking task as
SUCCESS. dag_id=***_proc, task_id=task_1, execution_date=20220908T124552,
start_date=20220908T124555, end_date=20220908T124557
[2022-09-08 12:45:57,185] {taskinstance.py:2356} DEBUG - Task Duration set
to 2.15429
[2022-09-08 12:45:57,232] {events.py:43} DEBUG - session flush listener:
added ['success'] unchanged () deleted ['running'] - <TaskInstance:
postgres_proc.task_1 manual__2022-09-08T12:45:52.814823+00:00 [success]>
[2022-09-08 12:45:57,232] {listener.py:119} DEBUG - OpenLineage listener got
notification about task instance success
[2022-09-08 12:45:57,234] {manager.py:35} DEBUG - Using extractor
PostgresExtractor task_type=PostgresOperator airflow_dag_id=***_proc
task_id=task_1 airflow_run_id=manual__2022-09-08T12:45:52.814823+00:00
[2022-09-08 12:45:57,235] {postgres_extractor.py:41} DEBUG - Sending SQL to
parser: CALL display_message('ola!');
[2022-09-08 12:45:57,237] {__init__.py:27} ERROR - SQL parser failed: sql
parser error: Expected an SQL statement, found: CALL
[2022-09-08 12:45:57,237] {postgres_extractor.py:43} DEBUG - Got meta None
[2022-09-08 12:45:57,238] {manager.py:42} DEBUG - Found task metadata for
operation task_1: TaskMetadata(name='***_proc.task_1', inputs=[], outputs=[],
run_facets={}, job_facets={'sql':
SqlJobFacet(_producer='https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow',
_schemaURL='https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/SqlJobFacet',
query="CALL display_message('ola!');")})
[2022-09-08 12:45:57,240] {console.py:21} INFO - {"eventTime":
"2022-09-08T12:45:57.184510Z", "eventType": "COMPLETE", "inputs": [], "job":
{"facets": {"sql": {"_producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"_schemaURL":
"https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/OpenLineage.json#/definitions/SqlJobFacet",
"query": "CALL display_message('ola!');"}}, "name": "***_proc.task_1",
"namespace": "default"}, "outputs": [], "producer":
"https://github.com/OpenLineage/OpenLineage/tree/0.10.0/integration/airflow",
"run": {"facets": {}, "runId": "f6b6cdfa-4112-4425-95c7-511078ef5cc0"}}
[2022-09-08 12:45:57,245] {cli_action_loggers.py:84} DEBUG - Calling
callbacks: []
[2022-09-08 12:45:57,292] {local_task_job.py:156} INFO - Task exited with
return code 0
[2022-09-08 12:45:57,293] {taskinstance.py:879} DEBUG - Refreshing
TaskInstance <TaskInstance: postgres_proc.task_1
manual__2022-09-08T12:45:52.814823+00:00 [running]> from DB
[2022-09-08 12:45:57,702] {dagrun.py:643} DEBUG - number of tis tasks for
<DagRun postgres_proc @ 2022-09-08 12:45:52.814823+00:00:
manual__2022-09-08T12:45:52.814823+00:00, externally triggered: True>: 0 task(s)
[2022-09-08 12:45:57,705] {local_task_job.py:273} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]