[GitHub] spark issue #23090: [SPARK-26118][Web UI] Introducing spark.ui.requestHeader...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23090 **[Test build #4434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4434/testReport)** for PR 23090 at commit [`a343f93`](https://github.com/apache/spark/commit/a343f933d15aacc1e0407a0b23eceb0b4da7a5bc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22149: [SPARK-25158][SQL]Executor accidentally exit because Scr...
Github user LuciferYang commented on the issue: https://github.com/apache/spark/pull/22149 @gatorsmile should we fix this problem? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99044/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99032/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99044/testReport)** for PR 23062 at commit [`04a34c4`](https://github.com/apache/spark/commit/04a34c4ee7c43120721659713fb8024e90282a0c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23087 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99036/testReport)** for PR 22583 at commit [`c876776`](https://github.com/apache/spark/commit/c876776738704089bb13d488f3cce9483eeed9f0). * This patch **fails Spark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23087 **[Test build #99032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99032/testReport)** for PR 23087 at commit [`6a39012`](https://github.com/apache/spark/commit/6a39012a6f2f226003bd2d6ffac63374781aedb4). * This patch passes all tests. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99036/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99043/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99043/testReport)** for PR 23062 at commit [`8c25fd0`](https://github.com/apache/spark/commit/8c25fd0c843576fb86b1881b57a6eb445e1c5461). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23004 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99040/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23004 **[Test build #99040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99040/testReport)** for PR 23004 at commit [`0748deb`](https://github.com/apache/spark/commit/0748deb01a2f2ee5d17a814983b174eeebf3f6c1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99041/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99041/testReport)** for PR 22583 at commit [`672bccb`](https://github.com/apache/spark/commit/672bccb64e75b009179e00fe6ede9bf34b5b4dbb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99034/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23087 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23087: [SPARK-26124][BUILD] Update plugins to latest versions
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23087 **[Test build #99034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99034/testReport)** for PR 23087 at commit [`93cd718`](https://github.com/apache/spark/commit/93cd71846f8220c0979af079451588680f1e8d46). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99037/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99037/testReport)** for PR 22583 at commit [`c9465e6`](https://github.com/apache/spark/commit/c9465e623099690ec29b30a94aaa0b346d5dacfe). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/23084 @cloud-fan, I think not kill executor. the memory allocate fails, and the caller is notified to perform other operations. for example, HashAggregateExec will carry out spill the map and fallback to sort-based. please correct if i understand something wrong. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99035/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #99035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99035/testReport)** for PR 23054 at commit [`0ffdb4b`](https://github.com/apache/spark/commit/0ffdb4b9f443849a85f43be689712f7ec4805a2b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 cc @vanzin @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22635: [SPARK-25591][PySpark][SQL] Avoid overwriting deserializ...
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/22635 @viirya I appologize, as I mentioned in my [comment in SPARK-26019](https://issues.apache.org/jira/browse/SPARK-26019?focusedCommentId=16692233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16692233), it's due to another change https://github.com/apache/spark/commit/15fc2372269159ea2556b028d4eb8860c4108650#diff-c3339bbf2b850b79445b41e9eecf57c4R249 - error happens in `authenticate_and_accum_updates()` and that's a new code that was brought by that code. Thanks for looking at that anyway! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5173/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23084 **[Test build #99045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99045/testReport)** for PR 23084 at commit [`400bcd5`](https://github.com/apache/spark/commit/400bcd5e6672281f352b36ef64219ec50bb2c2f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23084: [SPARK-26117][CORE][SQL]use SparkOutOfMemoryError instea...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23084 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23039: [SPARK-26066][SQL] Move truncatedString to sql/catalyst ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23039 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23039: [SPARK-26066][SQL] Move truncatedString to sql/catalyst ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23039 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99030/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23039: [SPARK-26066][SQL] Move truncatedString to sql/catalyst ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23039 **[Test build #99030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99030/testReport)** for PR 23039 at commit [`082254d`](https://github.com/apache/spark/commit/082254dd5ef36d84acd1103e7e2aedf4c922ea10). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23004 Looks fine to me --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99031/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99029/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99031/testReport)** for PR 23062 at commit [`4cba3cc`](https://github.com/apache/spark/commit/4cba3ccaaa39f7e2103ce7d4f953aba49e942a28). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99029/testReport)** for PR 23062 at commit [`39979e4`](https://github.com/apache/spark/commit/39979e43add48c9f1597ff8314434ff679ac5483). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5172/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99044/testReport)** for PR 23062 at commit [`04a34c4`](https://github.com/apache/spark/commit/04a34c4ee7c43120721659713fb8024e90282a0c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234861969
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -109,6 +109,35 @@ object TestingUDT {
}
}
+/** An example derived from Twitter/Scrooge codegen for thrift */
+object ScroogeLikeExample {
+ def apply(x: Int): ScroogeLikeExample = new Immutable(x)
+
+ def unapply(_item: ScroogeLikeExample): Option[Int] = Some(_item.x)
+
+ class Immutable(val x: Int) extends ScroogeLikeExample
+}
+
+trait ScroogeLikeExample extends Product1[Int] with Serializable {
+ import ScroogeLikeExample._
+
+ def x: Int
+
+ override def _1: Int = x
+
+ def copy(x: Int = this.x): ScroogeLikeExample = new Immutable(x)
+
+ override def canEqual(other: Any): Boolean =
other.isInstanceOf[ScroogeLikeExample]
+
+ private def _equals(x: ScroogeLikeExample, y: ScroogeLikeExample):
Boolean =
+ x.productArity == y.productArity &&
--- End diff --
That's OK, leave in Product, if it's actually testing the case you have in
mind. Yes I know equals() is needed. The new implementation looks good.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user drewrobb commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234861629
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -109,6 +109,35 @@ object TestingUDT {
}
}
+/** An example derived from Twitter/Scrooge codegen for thrift */
+object ScroogeLikeExample {
+ def apply(x: Int): ScroogeLikeExample = new Immutable(x)
+
+ def unapply(_item: ScroogeLikeExample): Option[Int] = Some(_item.x)
+
+ class Immutable(val x: Int) extends ScroogeLikeExample
+}
+
+trait ScroogeLikeExample extends Product1[Int] with Serializable {
+ import ScroogeLikeExample._
+
+ def x: Int
+
+ override def _1: Int = x
+
+ def copy(x: Int = this.x): ScroogeLikeExample = new Immutable(x)
+
+ override def canEqual(other: Any): Boolean =
other.isInstanceOf[ScroogeLikeExample]
+
+ private def _equals(x: ScroogeLikeExample, y: ScroogeLikeExample):
Boolean =
+ x.productArity == y.productArity &&
--- End diff --
I'm worried about changing the tests to use a concrete subtype, because the
reflection calls might behave differently in that case either now or later on.
I simplified a little more. `canEqual` is necessary to implement product.
`equals` is necessary or tests will not pass (it will check object pointer
equality), and `hashCode` is needed for scalastyle to pass since `equals` is
necessary.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234859506
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -109,6 +109,35 @@ object TestingUDT {
}
}
+/** An example derived from Twitter/Scrooge codegen for thrift */
+object ScroogeLikeExample {
+ def apply(x: Int): ScroogeLikeExample = new Immutable(x)
+
+ def unapply(_item: ScroogeLikeExample): Option[Int] = Some(_item.x)
+
+ class Immutable(val x: Int) extends ScroogeLikeExample
+}
+
+trait ScroogeLikeExample extends Product1[Int] with Serializable {
+ import ScroogeLikeExample._
+
+ def x: Int
+
+ override def _1: Int = x
+
+ def copy(x: Int = this.x): ScroogeLikeExample = new Immutable(x)
+
+ override def canEqual(other: Any): Boolean =
other.isInstanceOf[ScroogeLikeExample]
+
+ private def _equals(x: ScroogeLikeExample, y: ScroogeLikeExample):
Boolean =
+ x.productArity == y.productArity &&
--- End diff --
Hm, actually that probably won't work any more or less. OK, it's because
there is an Encoder for Product. You can still simplify the equals() and so on
I think, but looks like that's easier than a new Encoder. Or is it sufficient
to test a Seq of a concrete subtype of ScroogeLikeExample?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23086 **[Test build #99042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99042/testReport)** for PR 23086 at commit [`4407d51`](https://github.com/apache/spark/commit/4407d513fa9b127901c74f600ec00ae2aeb43b34). * This patch **fails Java style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class DataSourceV2StreamingScanExec(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99042/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234858573
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -109,6 +109,35 @@ object TestingUDT {
}
}
+/** An example derived from Twitter/Scrooge codegen for thrift */
+object ScroogeLikeExample {
+ def apply(x: Int): ScroogeLikeExample = new Immutable(x)
+
+ def unapply(_item: ScroogeLikeExample): Option[Int] = Some(_item.x)
+
+ class Immutable(val x: Int) extends ScroogeLikeExample
+}
+
+trait ScroogeLikeExample extends Product1[Int] with Serializable {
+ import ScroogeLikeExample._
+
+ def x: Int
+
+ override def _1: Int = x
+
+ def copy(x: Int = this.x): ScroogeLikeExample = new Immutable(x)
+
+ override def canEqual(other: Any): Boolean =
other.isInstanceOf[ScroogeLikeExample]
+
+ private def _equals(x: ScroogeLikeExample, y: ScroogeLikeExample):
Boolean =
+ x.productArity == y.productArity &&
--- End diff --
Just use SparkSession.createDataset?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5170/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23086 **[Test build #99042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99042/testReport)** for PR 23086 at commit [`4407d51`](https://github.com/apache/spark/commit/4407d513fa9b127901c74f600ec00ae2aeb43b34). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5171/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99043/testReport)** for PR 23062 at commit [`8c25fd0`](https://github.com/apache/spark/commit/8c25fd0c843576fb86b1881b57a6eb445e1c5461). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23004: [SPARK-26004][SQL] InMemoryTable support StartsWi...
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/23004#discussion_r234857747 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -237,6 +237,13 @@ case class InMemoryTableScanExec( if list.forall(ExtractableLiteral.unapply(_).isDefined) && list.nonEmpty => list.map(l => statsFor(a).lowerBound <= l.asInstanceOf[Literal] && l.asInstanceOf[Literal] <= statsFor(a).upperBound).reduce(_ || _) + +case StartsWith(a: AttributeReference, ExtractableLiteral(l)) => --- End diff -- Added to pr description. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23090: [SPARK-26118][Web UI] Introducing spark.ui.requestHeader...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23090 **[Test build #4434 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4434/testReport)** for PR 23090 at commit [`a343f93`](https://github.com/apache/spark/commit/a343f933d15aacc1e0407a0b23eceb0b4da7a5bc). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23004 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5168/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5169/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99041/testReport)** for PR 22583 at commit [`672bccb`](https://github.com/apache/spark/commit/672bccb64e75b009179e00fe6ede9bf34b5b4dbb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23004: [SPARK-26004][SQL] InMemoryTable support StartsWith pred...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23004 **[Test build #99040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99040/testReport)** for PR 23004 at commit [`0748deb`](https://github.com/apache/spark/commit/0748deb01a2f2ee5d17a814983b174eeebf3f6c1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23086: [SPARK-25528][SQL] data source v2 API refactor (b...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23086#discussion_r234855810
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/Batch.java ---
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.sources.v2.reader;
+
+import org.apache.spark.annotation.InterfaceStability;
+
+/**
+ * A physical representation of a data source scan for batch queries. This
interface is used to
+ * provide physical information, like how many partitions the scanned data
has, and how to read
+ * records from the partitions.
+ */
+@InterfaceStability.Evolving
+public interface Batch {
--- End diff --
I don't have a strong preference. I feel it's a little more clear to
distinguish between scan and batch
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23092: [SPARK-26094][CORE][STREAMING] createNonEcFile creates p...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23092: [SPARK-26094][CORE][STREAMING] createNonEcFile creates p...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99027/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23092: [SPARK-26094][CORE][STREAMING] createNonEcFile creates p...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23092 **[Test build #99027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99027/testReport)** for PR 23092 at commit [`c52010a`](https://github.com/apache/spark/commit/c52010a3fa291e85f75f862d2d75363e8505fcc7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23086 **[Test build #99038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99038/testReport)** for PR 23086 at commit [`83818fa`](https://github.com/apache/spark/commit/83818fafc8790cbe62ba1e0f2c0e18410328a256). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class DataSourceV2StreamingScanExec(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99038/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99039/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99039/testReport)** for PR 23062 at commit [`83a1987`](https://github.com/apache/spark/commit/83a19876e76d8ec4afb36d023e37575586e1d19e). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5167/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23062: [SPARK-8288][SQL] ScalaReflection can use companion obje...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23062 **[Test build #99039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99039/testReport)** for PR 23062 at commit [`83a1987`](https://github.com/apache/spark/commit/83a19876e76d8ec4afb36d023e37575586e1d19e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user drewrobb commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234854202
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -788,12 +788,37 @@ object ScalaReflection extends ScalaReflection {
}
/**
- * Finds an accessible constructor with compatible parameters. This is a
more flexible search
- * than the exact matching algorithm in `Class.getConstructor`. The
first assignment-compatible
- * matching constructor is returned. Otherwise, it returns `None`.
+ * Finds an accessible constructor with compatible parameters. This is a
more flexible search than
+ * the exact matching algorithm in `Class.getConstructor`. The first
assignment-compatible
+ * matching constructor is returned if it exists. Otherwise, we check
for additional compatible
+ * constructors defined in the companion object as `apply` methods.
Otherwise, it returns `None`.
*/
- def findConstructor(cls: Class[_], paramTypes: Seq[Class[_]]):
Option[Constructor[_]] = {
-Option(ConstructorUtils.getMatchingAccessibleConstructor(cls,
paramTypes: _*))
+ def findConstructor(cls: Class[_], paramTypes: Seq[Class[_]]):
Option[Seq[AnyRef] => Any] = {
--- End diff --
Ok, that makes total sense. I've made this change also
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23086 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5165/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99025/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23086: [SPARK-25528][SQL] data source v2 API refactor (batch re...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23086 **[Test build #99038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99038/testReport)** for PR 23086 at commit [`83818fa`](https://github.com/apache/spark/commit/83818fafc8790cbe62ba1e0f2c0e18410328a256). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5166/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99037/testReport)** for PR 22583 at commit [`c9465e6`](https://github.com/apache/spark/commit/c9465e623099690ec29b30a94aaa0b346d5dacfe). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23038 **[Test build #99025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99025/testReport)** for PR 23038 at commit [`ed85016`](https://github.com/apache/spark/commit/ed8501667745f8bf606c37ae88ff8b47223124a8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23062: [SPARK-8288][SQL] ScalaReflection can use compani...
Github user drewrobb commented on a diff in the pull request:
https://github.com/apache/spark/pull/23062#discussion_r234853788
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -109,6 +109,35 @@ object TestingUDT {
}
}
+/** An example derived from Twitter/Scrooge codegen for thrift */
+object ScroogeLikeExample {
+ def apply(x: Int): ScroogeLikeExample = new Immutable(x)
+
+ def unapply(_item: ScroogeLikeExample): Option[Int] = Some(_item.x)
+
+ class Immutable(val x: Int) extends ScroogeLikeExample
+}
+
+trait ScroogeLikeExample extends Product1[Int] with Serializable {
+ import ScroogeLikeExample._
+
+ def x: Int
+
+ override def _1: Int = x
+
+ def copy(x: Int = this.x): ScroogeLikeExample = new Immutable(x)
+
+ override def canEqual(other: Any): Boolean =
other.isInstanceOf[ScroogeLikeExample]
+
+ private def _equals(x: ScroogeLikeExample, y: ScroogeLikeExample):
Boolean =
+ x.productArity == y.productArity &&
--- End diff --
My previous answer was not complete. `Product1` is also necessary so that
the implicit `Encoders.product[T <: Product : TypeTag]` will work with this
class, if omitted the DatasetSuite test will not compile:
```
[error]
/home/drew/spark/sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala:1577:
value toDS is not a member of
Seq[org.apache.spark.sql.catalyst.ScroogeLikeExample]
[error] val ds = data.toDS
```
I could add some new encoder, but I think that might be worse as the goal
of this PR is for Scrooge classes to work with the provided implicit encoders.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #99036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99036/testReport)** for PR 22583 at commit [`c876776`](https://github.com/apache/spark/commit/c876776738704089bb13d488f3cce9483eeed9f0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99024/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23081 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23081 **[Test build #99024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99024/testReport)** for PR 23081 at commit [`51e0108`](https://github.com/apache/spark/commit/51e0108c127442c4cad537a849962c4b28f8149d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5164/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of non-struct type unde...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #99035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99035/testReport)** for PR 23054 at commit [`0ffdb4b`](https://github.com/apache/spark/commit/0ffdb4b9f443849a85f43be689712f7ec4805a2b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
