toddmeng-db commented on code in PR #3217:
URL: https://github.com/apache/arrow-adbc/pull/3217#discussion_r2249010407
##########
csharp/src/Drivers/Databricks/DatabricksOperationStatusPoller.cs:
##########
@@ -69,13 +72,18 @@ private async Task PollOperationStatus(CancellationToken
cancellationToken)
var operationHandle = _statement.OperationHandle;
if (operationHandle == null) break;
Review Comment:
I think it is because in THTTPTransport (used by SparkHttpConnection ->
DatabricksHttpconnection), a new Stream is created when the request is flushed.
If cancellation happens before this, that stream doesn't get cleared:
https://github.com/apache/thrift/blob/master/lib/netstd/Thrift/Transport/Client/THttpTransport.cs#L281
Yes, during testing, got some errors. In the proxy logs, I remember seeing
requests sent out with GetOperationStatus and CloseOperationStatus while
testing another PR
I think we are safe in HiveServer2Statement, but we might need to adjust
CancellationToken in DatabricksReader, CloudFetchResultFetcher, and
DatabricksCompositeReader
##########
csharp/src/Drivers/Databricks/DatabricksOperationStatusPoller.cs:
##########
@@ -69,13 +72,18 @@ private async Task PollOperationStatus(CancellationToken
cancellationToken)
var operationHandle = _statement.OperationHandle;
if (operationHandle == null) break;
Review Comment:
I think it is because in THTTPTransport (used by SparkHttpConnection ->
DatabricksHttpconnection), a new Stream is created when the request is flushed.
If cancellation happens before this, that stream doesn't get cleared:
https://github.com/apache/thrift/blob/master/lib/netstd/Thrift/Transport/Client/THttpTransport.cs#L281
Yes, during testing, got some errors. In the proxy logs, I remember seeing
requests sent out with both GetOperationStatus and CloseOperationStatus (in the
same request) while testing another PR
I think we are safe in HiveServer2Statement, but we might need to adjust
CancellationToken in DatabricksReader, CloudFetchResultFetcher, and
DatabricksCompositeReader
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
