uranusjr commented on code in PR #69080:
URL: https://github.com/apache/airflow/pull/69080#discussion_r3496550842
##########
java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/execution/Comm.kt:
##########
@@ -109,19 +112,22 @@ class CoordinatorComm(
}
@Throws(ApiError::class)
- suspend fun communicateImpl(body: Any): Any {
- var frame: IncomingFrame? = null
+ suspend fun communicateImpl(body: Any): Any =
+ commMutex.withLock {
+ val requestId = nextId.fetchAndAdd(1)
+ var frame: IncomingFrame? = null
- suspend fun handle(f: IncomingFrame) {
- frame = f
- }
- sendMessage(nextId.fetchAndAdd(1), body)
- processOnce(::handle)
- if (frame == null) {
- throw ApiError("No response received")
+ suspend fun handle(f: IncomingFrame) {
+ frame = f
+ }
+ sendMessage(requestId, body)
+ processOnce(::handle)
+ val received = frame ?: throw ApiError("No response received")
+ if (received.id != requestId) {
+ throw ApiError("response id ${received.id} does not match request id
$requestId")
+ }
+ received.body ?: Unit
Review Comment:
I’m not saying we shouldn’t use locks, but if we do multiplexing we should
be able to reduce the locking surface quite a bit, essentially only needs to
lock during send and during receive, not the entire communication
(serialize-send-receive-deserialize) process.
This can be explored in a different PR if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
