Re: SQLListener concurrency bug?

Oh, sorry, I was wrong. Concurrent collections in Scala are available since 2.8. Any objections against replacing mutable list and synchronized with a concurrent collection like or based on TrieMap for instance? On Wed, Jun 28, 2017, 16:05 Oleksandr Vayda wrote: >

Re: SQLListener concurrency bug?

Cool. I will be happy to create a PR. The simplest and most obvious solution that came to my mind was using Java concurrent collections instead of Scala mutable. Don't you mind to have this bit of Java inside Spark? :) Or perhaps we could use Scala concurrent collections, but they are only

Re: SQLListener concurrency bug?

Right now they are safe because the caller also calls synchronized when using them. This is to avoid copying objects. It's probably a bad design. If you want to refactor them, PR is welcome. On Mon, Jun 26, 2017 at 2:27 AM, Oleksandr Vayda wrote: > Hi all, > > Reading

SQLListener concurrency bug?

Hi all, Reading the source code of the org.apache.spark.sql.execution.ui.SQLListener, specifically this place - https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala#L328 def getFailedExecutions: Seq[SQLExecutionUIData] =