lidavidm commented on code in PR #974:
URL: https://github.com/apache/arrow-adbc/pull/974#discussion_r1288961140
##########
c/validation/adbc_validation_util.h:
##########
@@ -206,11 +206,11 @@ struct GetObjectsReader {
}
~GetObjectsReader() { AdbcGetObjectsDataDelete(get_objects_data_); }
- struct AdbcGetObjectsData* operator*() {
- return get_objects_data_;
+ struct AdbcGetObjectsData* operator*() {
Review Comment:
Whitespace errors?
##########
go/adbc/driver/snowflake/connection.go:
##########
@@ -33,6 +33,11 @@ import (
"github.com/snowflakedb/gosnowflake"
)
+const (
+ defaultStatementQueueSize = 200
+ defaultPrefectConcurrency = 10
Review Comment:
Prefect -> Prefetch?
##########
go/adbc/driver/snowflake/statement.go:
##########
@@ -33,13 +33,15 @@ import (
)
const (
- OptionStatementQueueSize = "adbc.rpc.result_queue_size"
+ OptionStatementQueueSize = "adbc.rpc.result_queue_size"
+ OptionStatementPrefetchConcurrency =
"adbc.snowflake.rpc.prefetch-concurrency"
Review Comment:
can we also add these to `adbc_driver_snowflake/__init__.py`?
##########
docs/source/driver/snowflake.rst:
##########
@@ -207,10 +207,17 @@ In addition, results are potentially fetched in parallel
from multiple endpoints
A limited number of batches are queued per endpoint, though data is always
returned to the client in the order of the endpoints.
-The queue size can be changed by setting an option on the
:cpp:class:`AdbcStatement`:
+To manage the performance of result fetching there are two options to control
+buffering and concurrency behavior. These options are only available to be set
+on the :cpp:class:`AdbcStatement` object:
``adbc.rpc.result_queue_size``
- The number of batches to queue per endpoint. Defaults to 5.
+ The number of batches to queue in the record reader. Defaults to 200.
+ Must be an integer > 0.
+
+``adbc.snowflake.rpc.prefetch-concurrency``
Review Comment:
Can we be consistent with underscores, not dashes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
