[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20552 This is obsolete - we're changing the lifecycle of DataWriter. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87417/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87417/testReport)** for PR 20552 at commit [`66270c5`](https://github.com/apache/spark/commit/66270c530ff7d7a82f9661b42d158d0b4544b8b7). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87417/testReport)** for PR 20552 at commit [`66270c5`](https://github.com/apache/spark/commit/66270c530ff7d7a82f9661b42d158d0b4544b8b7). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20552 Filed SPARK-23416 for the unrelated failure in build 87241. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87241/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87241/testReport)** for PR 20552 at commit [`a33a35c`](https://github.com/apache/spark/commit/a33a35ccbae7350519a3faf8d5d3d6f35692feb3). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20552 It's my intent to say that other data sources built by general developers aren't supposed to use batch ids in the executors for any purpose. In addition to the issue you mentioned, I don't think there's a compelling reason to do so in the DataSourceV2 model, and I worry it's easy to write implementations that seem correct but aren't that way. Since this interface is still evolving, I think it makes sense to revisit the question if we notice a scenario where it's infeasible to rewrite a piece of transactional logic to not use the batch ID in the executor. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87241/testReport)** for PR 20552 at commit [`a33a35c`](https://github.com/apache/spark/commit/a33a35ccbae7350519a3faf8d5d3d6f35692feb3). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87231/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87231/testReport)** for PR 20552 at commit [`87d0bc8`](https://github.com/apache/spark/commit/87d0bc8ce23ab5a95ba0b5432d6b58042b32bdac). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87231/testReport)** for PR 20552 at commit [`87d0bc8`](https://github.com/apache/spark/commit/87d0bc8ce23ab5a95ba0b5432d6b58042b32bdac). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87230/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87230/testReport)** for PR 20552 at commit [`44de1ea`](https://github.com/apache/spark/commit/44de1ea878fb65e4e04ac6cd594f2e7c72ea2d5e). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class ForeachWriterProvider[T: Encoder](writer: ForeachWriter[T]) extends StreamWriteSupport ` * `case class ForeachInternalWriter[T: Encoder](` * `case class ForeachWriterFactory[T: Encoder](writer: ForeachWriter[T], encoder: ExpressionEncoder[T])` * `class ForeachDataWriter[T : Encoder](` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20552 **[Test build #87230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87230/testReport)** for PR 20552 at commit [`44de1ea`](https://github.com/apache/spark/commit/44de1ea878fb65e4e04ac6cd594f2e7c72ea2d5e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20552 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20552: [SPARK-23099][SS] Migrate foreach sink to DataSourceV2
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20552 /cc @tdas --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
