dongjoon-hyun commented on a change in pull request #31682:
URL: https://github.com/apache/spark/pull/31682#discussion_r588085701
##########
File path: core/src/main/scala/org/apache/spark/api/python/SerDeUtil.scala
##########
@@ -131,7 +132,8 @@ private[spark] object SerDeUtil extends Logging {
}
private def checkPickle(t: (Any, Any)): (Boolean, Boolean) = {
- val pickle = new Pickler
+ val pickle = new Pickler(/* useMemo = */ true,
+ /* valueCompare = */ false)
Review comment:
ditto.
```scala
- val pickle = new Pickler(/* useMemo = */ true,
- /* valueCompare = */ false)
+ val pickle = new Pickler(/* useMemo = */ true, /* valueCompare = */
false)
```
##########
File path: core/src/main/scala/org/apache/spark/api/python/SerDeUtil.scala
##########
@@ -78,7 +78,8 @@ private[spark] object SerDeUtil extends Logging {
* Choose batch size based on size of objects
*/
private[spark] class AutoBatchedPickler(iter: Iterator[Any]) extends
Iterator[Array[Byte]] {
- private val pickle = new Pickler()
+ private val pickle = new Pickler(/* useMemo = */ true,
+ /* valueCompare = */ false)
Review comment:
nit. One-line may look better.
```scala
- private val pickle = new Pickler(/* useMemo = */ true,
- /* valueCompare = */ false)
+ private val pickle = new Pickler(/* useMemo = */ true, /* valueCompare
= */ false)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
