subhramit commented on code in PR #71976:
URL: https://github.com/apache/airflow/pull/71976#discussion_r3875345255
##########
providers/influxdb/src/airflow/providers/influxdb/operators/influxdb3.py:
##########
@@ -40,6 +40,11 @@ class InfluxDB3Operator(BaseOperator):
:param sql: The SQL query to be executed
:param influxdb3_conn_id: Reference to :ref:`InfluxDB 3 connection id
<howto/connection:influxdb3>`.
+ :param deferrable: Run the query from the triggerer instead of holding a
worker slot for its
+ duration. Requires ``influxdb3-python>=0.12.0``. Note that InfluxDB 3
streams results over
+ Arrow Flight rather than exposing a job that can be polled, so the
whole result set still flows back
+ through XCom -- deferring helps with long-running queries returning
modest result sets
+ (aggregations, freshness probes), not with very large extracts.
Review Comment:
Was just meaning to say that since the InfluxDB 3 client doesn’t expose a
poll-based async job model here (see PR desc for details), `deferrable=True`
releases the worker slot while the query runs in the triggerer, but the full
result still comes back through XCom when the task resumes. So the main benefit
is for long-running queries with relatively small result sets, not for very
large extracts
Refined with better wording now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
