dongjoon-hyun commented on code in PR #38958:
URL: https://github.com/apache/spark/pull/38958#discussion_r1041991912
##########
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala:
##########
@@ -34,4 +35,12 @@ private[spark] object Connect {
.version("3.4.0")
.stringConf
.createOptional
+
+ val CONNECT_GRPC_ARROW_MAX_BATCH_SIZE =
+ ConfigBuilder("spark.connect.grpc.arrow.maxBatchSize")
+ .doc("When using Apache Arrow, limit the maximum size of one arrow batch
that " +
+ "can be sent from server side to client side.")
Review Comment:
BTW, do we need to mention that we use only 70% of this variable for now?
Otherwise, it seems that `spark.connect.grpc.arrow.maxBatchSize` could be
misleading, @zhengruifeng .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
