peter-toth commented on pull request #31682: URL: https://github.com/apache/spark/pull/31682#issuecomment-788145063
> > During serialization GenericRowWithSchema(1.0, 1.0) will be memoized first > > Is this a global cache or per-query? This looks scary and I would like to disable it. Can be specified when we create a new `Pickler` (https://github.com/irmen/Pyrolite/blob/pyrolite-4.21/java/src/main/java/net/razorvine/pickle/Pickler.java#L119) and we create it per python evaluation: https://github.com/apache/spark/blob/branch-3.0/sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala#L49 but there are some other places where we use it... Shall I move this PR into this direction and disable `valueCompare`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
