pankajkoti commented on code in PR #31201:
URL: https://github.com/apache/airflow/pull/31201#discussion_r1204789753
##########
airflow/providers/apache/livy/operators/livy.py:
##########
@@ -207,4 +207,5 @@ def execute_complete(self, context: Context, event:
dict[str, Any]) -> Any:
self.task_id,
event["response"],
)
+ context["ti"].xcom_push(key="app_id",
value=self.get_hook().get_batch(event["batch_id"])["appId"])
Review Comment:
yes, I am curious to understand the general consensus. If no downstream
tasks are going to pull the XCOM information, would it clean up the record in
the database or it stay in there forever? My thought process is that if it's
not going to get used (maybe users need it? not sure) then why populate the db
with 1 more record?
And I tried to follow the previous commit to publish only in case of success
and apparently, we have not got feedback yet that it might be needed for failed
cases too, no? If the general consensus across the project is to push only for
success then perhaps we could wait until users come up with this need for this
operator that they also need this for failed tasks. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
