Hey Sean, Sounds good to me. At least, it's not worse than any versions prior to 2.3.0 which has a global ScalaReflectionLock. In addition, if someone hits a performance regression caused by this, they probably are creating too many Encoders. Reusing Encoders is a better solution for this case.
Best regards, Shixiong On Thu, Mar 14, 2019 at 2:25 PM Sean Owen <sro...@gmail.com> wrote: > This is worth a look: https://github.com/apache/spark/pull/24085 > > Scala has a surprising thread-safety bug in the "<:<" operator that's > used to check subtypes, which can lead to incorrect results in > non-trivial situations. > > The fix on the table is to introduce a global lock to protect a lot of > the Scala-related reflection code to resolve it. This may be the best > we can do, but 'global lock' sounds ominous. > > Any thoughts on other ways to resolve it? I'm not sure there are. > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
