[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20646 I opened a new PR to test out an alternate approach. PTAL - https://github.com/apache/spark/pull/20650/files?w=1 (note the `w=1`, that is to ignore whitespaces diffs in the diff view). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87572/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87572/testReport)** for PR 20646 at commit [`1df90e7`](https://github.com/apache/spark/commit/1df90e796e9388d7992bc55f9f87bfd71af2f7f9). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20646 Actually, I am having second thoughts about this. This is fundamentally changing how the tests work, especially for stress tests. The stress tests actually test these corner cases (by randomly adding successive AddData) about what if data was being added while the previously added data is being picked up. With this change, we will accidentally not test those race-condition-prone cases. Second, we are taking multiple locks here in multiple sources, and the StreamExecution is likely to take the same locks. I am really afraid that we are introducing deadlocks by doing this. I am still thinking what the right approach here. I think it should be - Explicit synchronized adding of data to multiple sources. - Not holding locks in multiple sources. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 (https://issues.apache.org/jira/browse/SPARK-23369 was already filed for previous flake) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87572/testReport)** for PR 20646 at commit [`1df90e7`](https://github.com/apache/spark/commit/1df90e796e9388d7992bc55f9f87bfd71af2f7f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 > java.lang.RuntimeException: [unresolved dependency: com.sun.jersey#jersey-core;1.14: configuration not found in com.sun.jersey#jersey-core;1.14: 'master(compile)'. Missing configuration: 'compile'. It was required from org.apache.hadoop#hadoop-yarn-common;2.6.5 compile] Surely unrelated to this change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87571/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87571/testReport)** for PR 20646 at commit [`1df90e7`](https://github.com/apache/spark/commit/1df90e796e9388d7992bc55f9f87bfd71af2f7f9). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87571/testReport)** for PR 20646 at commit [`1df90e7`](https://github.com/apache/spark/commit/1df90e796e9388d7992bc55f9f87bfd71af2f7f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 @tdas --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
