AynSiddiqui commented on code in PR #1609:
URL: https://github.com/apache/arrow-go/pull/1609#discussion_r4626159194
##########
arrow/cdata/cdata_exports.go:
##########
@@ -535,7 +543,11 @@ func asyncTaskQueue(ctx context.Context, schema
*arrow.Schema, recordStream chan
}
// request another batch now that we've processed this
one
- C.goCallRequest(producer, C.int64_t(1))
+ h.mu.Lock()
Review Comment:
You're right, I couldn't back that sentence up. I removed it. The
description now says what the spec text does: only synchronous `on_next_task`
and `on_error` calls are forbidden from `request`, and `abi.h` only says later
`release` calls "should" be scheduled. It also lists a synchronous `release`
from inside `request` (deadlock on `h.mu`) as a known case next to the
lock-ordering one.
##########
arrow/cdata/cdata_exports.go:
##########
@@ -511,17 +512,24 @@ func (rr *cRecordReader) release() {
}
type cAsyncStreamHandler struct {
+ // mu guards producer: once release returns the producer may be freed,
+ // so it must not be called again.
+ mu sync.Mutex
producer *CArrowAsyncProducer
taskQueue chan taskState
ctx context.Context
}
-func asyncTaskQueue(ctx context.Context, schema *arrow.Schema, recordStream
chan<- RecordMessage, taskQueue <-chan taskState, producer
*CArrowAsyncProducer) {
+func asyncTaskQueue(ctx context.Context, schema *arrow.Schema, recordStream
chan<- RecordMessage, taskQueue <-chan taskState, h *cAsyncStreamHandler) {
defer close(recordStream)
for {
select {
case <-ctx.Done():
- C.goCallCancel(producer)
+ h.mu.Lock()
+ if h.producer != nil {
+ C.goCallCancel(h.producer)
+ }
+ h.mu.Unlock()
return
Review Comment:
Agreed, that's a follow-up and not part of this PR. After `cancel` the
consumer could keep draining `taskQueue` and call `extract_data` with NULL for
each task until `release` closes the channel, which `asyncTaskExtract` now
handles. There is a related producer-side gap I found by reading:
`exportAsyncProducer` allocates a handle slot and `cgo.Handle` per task
(`exports.go:431`) and frees them only when `on_next_task` fails
(`exports.go:443-444`), while `asyncTaskExtract` (`exports.go:332-345`)
releases the record but never frees them. I haven't reproduced that at runtime.
Happy to open an issue for both once this PR merges.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]