uranusjr commented on code in PR #74794:
URL: https://github.com/apache/airflow/pull/74794#discussion_r4548646282


##########
airflow-core/src/airflow/dag_processing/processor.py:
##########
@@ -257,8 +261,11 @@ def _parse_file(msg: DagFileParseRequest, log: 
FilteringBoundLogger) -> DagFileP
     )
 
     if msg.callback_requests:
+        targets_loaded = any(_is_callback_target_loaded(bag, request) for 
request in msg.callback_requests)

Review Comment:
   But doesn’t the schema change anyway with DagCallbacksStarted being added? 



##########
airflow-core/src/airflow/dag_processing/processor.py:
##########
@@ -183,6 +200,32 @@ class DagFileParsingResult(BaseModel):
 ]
 
 
[email protected]()
+class _DagParseCommsDecoder(CommsDecoder[ToDagProcessor, ToManager]):
+    _received_api_error: bool = attrs.field(default=False, init=False)
+    _pending_request_ids: set[int] = attrs.field(factory=set, init=False)
+
+    @property
+    def api_request_failed(self) -> bool:
+        """Include unanswered requests interrupted by the Dag import 
timeout."""
+        return self._received_api_error or bool(self._pending_request_ids)
+
+    def _make_frame(self, msg: ToManagerMessage):
+        frame = super()._make_frame(msg)
+        self._pending_request_ids.add(frame.id)
+        return frame
+
+    def _from_frame(self, frame) -> ToDagProcessor | None:
+        # Match IDs so another response cannot hide an interrupted sync or 
async request.
+        self._pending_request_ids.discard(frame.id)
+        try:
+            return super()._from_frame(frame)
+        except AirflowRuntimeError as error:
+            if error.error.error == ErrorType.API_SERVER_ERROR:
+                self._received_api_error = True
+            raise

Review Comment:
   Would it be better for the Task SDK to expose this instead, e.g. a public 
`CommsDecoder.has_failed_requests`? Overriding here would mean the dag 
processor will need to be in the same package as the rest of the SDK. It’s a 
choice, but I’m not sure we want to commit yet.



##########
airflow-core/src/airflow/dag_processing/processor.py:
##########
@@ -143,8 +153,15 @@ class DagFileParsingResult(BaseModel):
     type: Literal["DagFileParsingResult"] = "DagFileParsingResult"
 
 
+class DagCallbacksStarted(BaseModel):
+    """Request permission to begin a callback batch after importing its Dag 
file."""

Review Comment:
   Is this docstring stale? The server does not seem to do anything about 
permission.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to