ueshin commented on a change in pull request #31682:
URL: https://github.com/apache/spark/pull/31682#discussion_r586925305
##########
File path: core/src/main/scala/org/apache/spark/api/python/SerDeUtil.scala
##########
@@ -78,7 +78,7 @@ private[spark] object SerDeUtil extends Logging {
* Choose batch size based on size of objects
*/
private[spark] class AutoBatchedPickler(iter: Iterator[Any]) extends
Iterator[Array[Byte]] {
- private val pickle = new Pickler()
+ private val pickle = new Pickler(true, false)
Review comment:
~nit: I'd prefer to use named arguments for readability: `new
Pickler(useMemo=true, valueCompare=false)`.~
nvm, as the library is in Java, we can't specify the names.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
