toddmeng-db commented on code in PR #3217:
URL: https://github.com/apache/arrow-adbc/pull/3217#discussion_r2249010407
##########
csharp/src/Drivers/Databricks/DatabricksOperationStatusPoller.cs:
##########
@@ -69,13 +72,18 @@ private async Task PollOperationStatus(CancellationToken
cancellationToken)
var operationHandle = _statement.OperationHandle;
if (operationHandle == null) break;
Review Comment:
Yea, during my testing, if stop() canceled the cancellationToken, it would
result in some strange behavior, GetOperationStatus issues causing
CloseOperation to fail.
I think it is because in THTTPTransport (used by SparkHttpConnection ->
DatabricksHttpconnection), a new Stream is created when the request is flushed.
If cancellation happens before this, that stream doesn't get cleared:
https://github.com/apache/thrift/blob/master/lib/netstd/Thrift/Transport/Client/THttpTransport.cs#L281
In the proxy logs, I remember seeing requests sent out with
GetOperationStatus and CloseOperationStatus while testing another PR
I think we are safe in HiveServer2Statement, but we might need to adjust
CancellationToken in DatabricksReader, CloudFetchResultFetcher, and
DatabricksCompositeReader
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
