brkyvz commented on issue #26225: [SPARK-29568][SS] Stop existing running streams when a new stream is launched URL: https://github.com/apache/spark/pull/26225#issuecomment-553207189 If stopActiveRunOnRestart is false, this piece of code is not even executed. Another error is thrown earlier. On Tue, Nov 12, 2019, 6:04 PM Tathagata Das <[email protected]> wrote: > *@tdas* commented on this pull request. > ------------------------------ > > In > sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala > <https://github.com/apache/spark/pull/26225#discussion_r345538033>: > > > > + activeRunOpt.foreach(_.stop()) > + > + activeQueriesSharedLock.synchronized { > + // We still can have a race condition when two concurrent instances try to start the same > + // stream, while a third one was already active. In this case, we throw a > + // ConcurrentModificationException. > + val oldActiveQuery = sparkSession.sharedState.activeStreamingQueries.put( > + query.id, query.streamingQuery) // we need to put the StreamExecution, not the wrapper > + if (oldActiveQuery != null) { > + throw new ConcurrentModificationException( > + "Another instance of this query was just started by a concurrent session.") > > This is not the correct error message when stopActiveRunOnRestart is > false. > If the active run was stopped, then this error message is correct. > If the active run was not stopped, then this error will be thrown and > therefore should simply say that there is an active run (run id ...). > > In other words, this can stay as the same message as it was in Spark > 2.4,.... may be improved by adding the run id. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/26225?email_source=notifications&email_token=ABIAE67GGGFBE62ES54DPNDQTNODTA5CNFSM4JD6YV42YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCLKS3FA#pullrequestreview-315960724>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABIAE65ERNLXRWXYE3U7HK3QTNODTANCNFSM4JD6YV4Q> > . >
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
