[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455979038 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455979038 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455979045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7265/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455979045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7265/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455978766 **[Test build #101469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101469/testReport)** for PR 23559 at commit [`03f59e9`](https://github.com/apache/spark/commit/03f59e93945329f849f5f9e43ab9521a542572df). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix
ClassCastException in TableReader while creating HadoopRDD
URL: https://github.com/apache/spark/pull/23559#discussion_r249349044
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala
##
@@ -192,4 +192,44 @@ class HiveTableScanSuite extends HiveComparisonTest with
SQLTestUtils with TestH
case p: HiveTableScanExec => p
}.get
}
+
+ test("[SPARK-26630] Fix ClassCastException in TableReader while creating
HadoopRDD") {
Review comment:
Added more test cases? like CTAS and then SELECt? INSERT and then SELECT?
Move it to `HiveCatalogedDDLSuite`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] Deegue commented on a change in pull request #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
Deegue commented on a change in pull request #23559: [SPARK-26630][SQL] Fix
ClassCastException in TableReader while creating HadoopRDD
URL: https://github.com/apache/spark/pull/23559#discussion_r249348817
##
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##
@@ -243,7 +239,8 @@ class HadoopTableReader(
// Create local references so that the outer object isn't serialized.
val localTableDesc = tableDesc
- createHadoopRdd(localTableDesc, inputPathStr, ifc).mapPartitions { iter
=>
+ createHadoopRDD(localTableDesc, inputPathStr)
+.mapPartitions { iter =>
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] Deegue commented on a change in pull request #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
Deegue commented on a change in pull request #23559: [SPARK-26630][SQL] Fix
ClassCastException in TableReader while creating HadoopRDD
URL: https://github.com/apache/spark/pull/23559#discussion_r249348806
##
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##
@@ -288,16 +285,28 @@ class HadoopTableReader(
}
}
+ /**
+ * The entry of creating a RDD.
+ */
+ private def createHadoopRDD(localTableDesc: TableDesc, inputPathStr:
String): RDD[Writable] = {
+localTableDesc.getInputFileFormatClass match {
+ case c: Class[_]
Review comment:
Reverted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix
ClassCastException in TableReader while creating HadoopRDD
URL: https://github.com/apache/spark/pull/23559#discussion_r249346152
##
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##
@@ -288,16 +285,28 @@ class HadoopTableReader(
}
}
+ /**
+ * The entry of creating a RDD.
+ */
+ private def createHadoopRDD(localTableDesc: TableDesc, inputPathStr:
String): RDD[Writable] = {
+localTableDesc.getInputFileFormatClass match {
+ case c: Class[_]
Review comment:
```Scala
val inputFormatClazz = localTableDesc.getInputFileFormatClass
if (classOf[org.apache.hadoop.mapreduce.InputFormat[_,
_]].isAssignableFrom(inputFormatClazz)) {
createNewHadoopRdd(localTableDesc, inputPathStr)
} else {
createOldHadoopRdd(localTableDesc, inputPathStr)
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
gatorsmile commented on a change in pull request #23559: [SPARK-26630][SQL] Fix
ClassCastException in TableReader while creating HadoopRDD
URL: https://github.com/apache/spark/pull/23559#discussion_r249346208
##
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##
@@ -243,7 +239,8 @@ class HadoopTableReader(
// Create local references so that the outer object isn't serialized.
val localTableDesc = tableDesc
- createHadoopRdd(localTableDesc, inputPathStr, ifc).mapPartitions { iter
=>
+ createHadoopRDD(localTableDesc, inputPathStr)
+.mapPartitions { iter =>
Review comment:
combine line 242 and 243?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] purijatin removed a comment on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel
purijatin removed a comment on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel URL: https://github.com/apache/spark/pull/23549#issuecomment-455974241 Jenkins, retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] purijatin commented on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel
purijatin commented on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel URL: https://github.com/apache/spark/pull/23549#issuecomment-455974241 Jenkins, retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] purijatin commented on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel
purijatin commented on issue #23549: [SPARK-26616][MLlib] Expose document frequency in IDFModel URL: https://github.com/apache/spark/pull/23549#issuecomment-455973958 It is weird: when I run the failed test locally with `python/run-tests --testnames pyspark.ml.feature`: It passes for `3L` (and fails for `3`). `3L` was the original case. So I dont know why it failed on jenkins. I am unable to replicate the original issue locally with `3L`. I have modified the test case as: ```python >>> model.numDocs == df.count() True ``` This passes the test. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455972884 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455972891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101464/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455973041 Oops, I didn't know deleting comments logs. Sorry for noise. I removed test failures unrelated with the current update changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455972891 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101464/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455972884 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455971903 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455971908 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101466/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455940842 **[Test build #101464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101464/testReport)** for PR 23117 at commit [`297f45b`](https://github.com/apache/spark/commit/297f45b599aef0d61c41471d25646946338cd2c4). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455971908 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101466/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455971903 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455593812 Hmmm running it with coverage itself might be related .. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455474378 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455443520 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455941541 **[Test build #101465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101465/testReport)** for PR 23117 at commit [`0dcc98a`](https://github.com/apache/spark/commit/0dcc98ad26d775400603d7a9169ad9be39f4f0b0). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455891873 **[Test build #101442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101442/testReport)** for PR 23117 at commit [`65053e9`](https://github.com/apache/spark/commit/65053e98cc7295c34e153d46395e2ec531cea1c9). * This patch **fails from timeout after a configured wait of `400m`**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455436351 **[Test build #101386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101386/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455363868 **[Test build #101376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101376/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails from timeout after a configured wait of `400m`**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455460034 **[Test build #101392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101392/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-442165870 **[Test build #99332 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99332/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455593929 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455710621 **[Test build #101405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101405/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails from timeout after a configured wait of `400m`**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455078241 **[Test build #101348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101348/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
SparkQA removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455947669 **[Test build #101466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101466/testReport)** for PR 23559 at commit [`7029aa7`](https://github.com/apache/spark/commit/7029aa764acf3656bc2ef5dc1fcfd866193823a5). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455573266 **[Test build #101395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101395/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455035769 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455399224 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455230929 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455150126 **[Test build #101356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101356/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-441061601 **[Test build #99171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99171/testReport)** for PR 23117 at commit [`e591dc6`](https://github.com/apache/spark/commit/e591dc644d4d59b37367dd0e6c97a6b2990bdad2). * This patch **fails from timeout after a configured wait of `400m`**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971484 **[Test build #101468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101468/testReport)** for PR 23117 at commit [`82732ea`](https://github.com/apache/spark/commit/82732eaded312b0cae6ec4876d0b5791dd4faa54). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455971600 **[Test build #101466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101466/testReport)** for PR 23559 at commit [`7029aa7`](https://github.com/apache/spark/commit/7029aa764acf3656bc2ef5dc1fcfd866193823a5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455152669 **[Test build #101358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101358/testReport)** for PR 23117 at commit [`18c40d9`](https://github.com/apache/spark/commit/18c40d925d7e93cc9221f3358d5028bf5ba007bd). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-441068754 **[Test build #99182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99182/testReport)** for PR 23117 at commit [`e591dc6`](https://github.com/apache/spark/commit/e591dc644d4d59b37367dd0e6c97a6b2990bdad2). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-441007227 **[Test build #99170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99170/testReport)** for PR 23117 at commit [`d88d5aa`](https://github.com/apache/spark/commit/d88d5aa73db636f8c73ace9f83f339781ea50531). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455078829 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455970860 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101465/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-441014288 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971444 FWIW, @shaneknapp, I added an empty commit authored by you at 82732eaded312b0cae6ec4876d0b5791dd4faa54 so .. this PR will be committed with you (for instance like https://github.com/apache/spark/commit/51bee7aca13451167fa3e701fcd60f023eae5e61) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971283 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7264/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971276 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971283 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7264/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455971276 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455970855 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455970860 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101465/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455970855 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455968095 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins removed a comment on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455968089 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455968095 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7263/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
AmplabJenkins commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455968089 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
SparkQA commented on issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#issuecomment-455967872 **[Test build #101467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101467/testReport)** for PR 23117 at commit [`e2ed235`](https://github.com/apache/spark/commit/e2ed2356c09b2128629383d4d9ae90e4b3ad5ef5). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins
HyukjinKwon commented on a change in pull request #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage report from Python via Jenkins URL: https://github.com/apache/spark/pull/23117#discussion_r249339865 ## File path: python/pyspark/streaming/tests/test_dstream.py ## @@ -22,12 +22,16 @@ import unittest from functools import reduce from itertools import chain +import platform from pyspark import SparkConf, SparkContext, RDD from pyspark.streaming import StreamingContext from pyspark.testing.streamingutils import PySparkStreamingTestCase +@unittest.skipIf( +"pypy" in platform.python_implementation().lower() and "COVERAGE_PROCESS_START" in os.environ, +"PyPy implementation causes to hang DStream tests forever when Coverage report is used.") Review comment: Hm, I am not sure but those tests hang forever when coverage is used. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] viirya commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
viirya commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455967011 @caneGuy Retested and the failed test is passed now. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] viirya commented on a change in pull request #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation
viirya commented on a change in pull request #23263: [SPARK-23674][ML] Adds
Spark ML Events to Instrumentation
URL: https://github.com/apache/spark/pull/23263#discussion_r249339450
##
File path: mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
##
@@ -132,7 +133,8 @@ class Pipeline @Since("1.4.0") (
* @return fitted pipeline
Review comment:
Ok. I see.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
AmplabJenkins commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455963712 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101463/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
AmplabJenkins removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455963711 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
AmplabJenkins removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455963712 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101463/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] liupc commented on issue #23580: [SPARK-26660]Add warning logs for large taskBinary size
liupc commented on issue #23580: [SPARK-26660]Add warning logs for large taskBinary size URL: https://github.com/apache/spark/pull/23580#issuecomment-455963772 The Logs looks like below after changes: Driver: ``` val c = (0 to 100).map(_ => r.nextLong().toString).toSeq c: scala.collection.immutable.Seq[String] = Vector(-8824816803907433122, 676576201675227827, -4450800462103535332, 4941742638328912331, -629826214489158275, 3599102463125026709, -1565579140513476623, -4953176692696234676, -7337015974401685398, 6861152894389600367, 4204365614405597947, -5631388062093635828, 952349730665646011, -1737760484135545791, -3818941051984249617, 4577662835073288447, -1043227373757634713, -2626518287151915759, -8712872867573643992, -7649246031751633828, 1964631192671370827, -720939716164610968, -1113194824007301084, -5476000723857254882, 8672676133212526245, 8119817434619293651, -7264818629918867452, -2796934762986648699, -804542345832804, -7249714536899879581, 7106267389367680272, 1322500019139194514, 5502357868702972085, -6484503789... scala> sc.parallelize(Seq(1)).map(_ => c.length).collect() 19/01/21 14:18:09 WARN scheduler.DAGScheduler: Broadcasting large task binary with size 21.1 MiB res1: Array[Int] = Array(101) ``` Executor: ``` 19/01/21 14:18:09 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 1 with 5 pieces (estimated total size 20.0 MiB) 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 4.0 MiB, free 260.0 MiB) 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1_piece1 stored as bytes in memory (estimated size 4.0 MiB, free 256.0 MiB) 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1_piece3 stored as bytes in memory (estimated size 4.0 MiB, free 252.0 MiB) 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1_piece4 stored as bytes in memory (estimated size 920.1 KiB, free 251.1 MiB) 19/01/21 14:18:09 INFO client.TransportClientFactory: Successfully created connection to /10.132.4.49:36947 after 1 ms (0 ms spent in bootstraps) 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1_piece2 stored as bytes in memory (estimated size 4.0 MiB, free 247.1 MiB) 19/01/21 14:18:09 INFO broadcast.TorrentBroadcast: Reading broadcast variable 1 took 353 ms 19/01/21 14:18:09 INFO memory.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 21.7 MiB, free 225.4 MiB) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
AmplabJenkins commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455963711 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
SparkQA removed a comment on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455940054 **[Test build #101463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101463/testReport)** for PR 23519 at commit [`421227a`](https://github.com/apache/spark/commit/421227a1b9c66f8ac9572de34b1b50e73bfa2a1e). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
SparkQA commented on issue #23519: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable URL: https://github.com/apache/spark/pull/23519#issuecomment-455963559 **[Test build #101463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101463/testReport)** for PR 23519 at commit [`421227a`](https://github.com/apache/spark/commit/421227a1b9c66f8ac9572de34b1b50e73bfa2a1e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] liupc commented on a change in pull request #23580: [SPARK-26660]Add warning logs for large taskBinary size
liupc commented on a change in pull request #23580: [SPARK-26660]Add warning
logs for large taskBinary size
URL: https://github.com/apache/spark/pull/23580#discussion_r249335813
##
File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
##
@@ -1162,6 +1162,12 @@ private[spark] class DAGScheduler(
partitions = stage.rdd.partitions
}
+ val taskBinarySizeKb =
Utils.byteStringAsKb(s"${taskBinaryBytes.length}b")
+
+ if (taskBinarySizeKb > TaskSetManager.TASK_SIZE_TO_WARN_KB) {
Review comment:
Should I make this Threshold another constant variarble?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] liupc commented on issue #23580: [SPARK-26660]Add warning logs for large taskBinary size
liupc commented on issue #23580: [SPARK-26660]Add warning logs for large taskBinary size URL: https://github.com/apache/spark/pull/23580#issuecomment-455963082 @maropu I don't understand what you mean by "another better place to detect both cases simultaneously: whole closure size and task size you described?". As my point of view, broadcasting large task binary (this broadcast is shared among all tasks) and the serialized task in TaskSetManager are two different things. I think is'ok to print a warning log in before broadcasting large task binary. I also made some log changes in the reading of broadcast to make the reading size more clear. @srowen @jerryshao any suggestions? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation
HyukjinKwon commented on a change in pull request #23263: [SPARK-23674][ML]
Adds Spark ML Events to Instrumentation
URL: https://github.com/apache/spark/pull/23263#discussion_r249335236
##
File path: mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
##
@@ -132,7 +133,8 @@ class Pipeline @Since("1.4.0") (
* @return fitted pipeline
Review comment:
I am not fully sure if we should expose this and make it public ... I think
API doc at `MLEvents` might be good enough for now. Wanted to leave this
unstable and not quite public for now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL] Cache manager locks
gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL]
Cache manager locks
URL: https://github.com/apache/spark/pull/23539#discussion_r249334726
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala
##
@@ -160,34 +167,48 @@ class CacheManager extends Logging {
/**
* Tries to re-cache all the cache entries that refer to the given plan.
*/
- def recacheByPlan(spark: SparkSession, plan: LogicalPlan): Unit = writeLock {
+ def recacheByPlan(spark: SparkSession, plan: LogicalPlan): Unit = {
recacheByCondition(spark, _.find(_.sameResult(plan)).isDefined)
}
private def recacheByCondition(
spark: SparkSession,
condition: LogicalPlan => Boolean,
clearCache: Boolean = true): Unit = {
-val it = cachedData.iterator()
val needToRecache = scala.collection.mutable.ArrayBuffer.empty[CachedData]
-while (it.hasNext) {
- val cd = it.next()
- if (condition(cd.plan)) {
-if (clearCache) {
- cd.cachedRepresentation.cacheBuilder.clearCache()
+readLock {
+ val it = cachedData.iterator()
+ while (it.hasNext) {
+val cd = it.next()
+if (condition(cd.plan)) {
+ if (clearCache) {
+cd.cachedRepresentation.cacheBuilder.clearCache()
+ }
+ // Remove the cache entry before we create a new one, so that we can
have a different
+ // physical plan.
+ needToRecache += cd
+}
+ }
+}
+writeLock {
+ needToRecache.foreach(cachedData.remove(_))
+}
+val recomputedPlans = needToRecache.map { cd =>
+ val plan = spark.sessionState.executePlan(cd.plan).executedPlan
+ val newCache = InMemoryRelation(
+cacheBuilder =
cd.cachedRepresentation.cacheBuilder.withCachedPlan(plan),
+logicalPlan = cd.plan)
+ cd.copy(cachedRepresentation = newCache)
+}
+writeLock {
Review comment:
We can embed writeLock to the traversal of needToRecache
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on a change in pull request #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation
HyukjinKwon commented on a change in pull request #23263: [SPARK-23674][ML]
Adds Spark ML Events to Instrumentation
URL: https://github.com/apache/spark/pull/23263#discussion_r249334748
##
File path: mllib/src/test/scala/org/apache/spark/ml/MLEventsSuite.scala
##
@@ -0,0 +1,255 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml
+
+import scala.collection.mutable
+import scala.concurrent.duration._
+import scala.language.postfixOps
+
+import org.apache.hadoop.fs.Path
+import org.mockito.ArgumentMatchers.{any, eq => meq}
+import org.mockito.Mockito.when
+import org.scalatest.BeforeAndAfterEach
+import org.scalatest.concurrent.Eventually
+import org.scalatest.mockito.MockitoSugar.mock
+
+import org.apache.spark.SparkFunSuite
+import org.apache.spark.ml.param.ParamMap
+import org.apache.spark.ml.util.{DefaultParamsReader, DefaultParamsWriter,
MLWriter}
+import org.apache.spark.mllib.util.MLlibTestSparkContext
+import org.apache.spark.scheduler.{SparkListener, SparkListenerEvent}
+import org.apache.spark.sql._
+
+
+class MLEventsSuite
+ extends SparkFunSuite with BeforeAndAfterEach with MLlibTestSparkContext
with Eventually {
+
+ private val events = mutable.ArrayBuffer.empty[MLEvent]
+ private val listener: SparkListener = new SparkListener {
+override def onOtherEvent(event: SparkListenerEvent): Unit = event match {
+ case e: MLEvent => events.append(e)
+ case _ =>
+}
+ }
+
+ override def beforeAll(): Unit = {
+super.beforeAll()
+spark.sparkContext.addSparkListener(listener)
+ }
+
+ override def afterEach(): Unit = {
+try {
+ events.clear()
+} finally {
+ super.afterEach()
+}
+ }
+
+ override def afterAll(): Unit = {
+try {
+ if (spark != null) {
+spark.sparkContext.removeSparkListener(listener)
+ }
+} finally {
+ super.afterAll()
+}
+ }
+
+ abstract class MyModel extends Model[MyModel]
+
+ test("pipeline fit events") {
+val estimator1 = mock[Estimator[MyModel]]
+val model1 = mock[MyModel]
+val transformer1 = mock[Transformer]
+val estimator2 = mock[Estimator[MyModel]]
+val model2 = mock[MyModel]
+
+when(estimator1.copy(any[ParamMap])).thenReturn(estimator1)
+when(model1.copy(any[ParamMap])).thenReturn(model1)
+when(transformer1.copy(any[ParamMap])).thenReturn(transformer1)
+when(estimator2.copy(any[ParamMap])).thenReturn(estimator2)
+when(model2.copy(any[ParamMap])).thenReturn(model2)
+
+val dataset1 = mock[DataFrame]
+val dataset2 = mock[DataFrame]
+val dataset3 = mock[DataFrame]
+val dataset4 = mock[DataFrame]
+val dataset5 = mock[DataFrame]
+
+when(dataset1.toDF).thenReturn(dataset1)
+when(dataset2.toDF).thenReturn(dataset2)
+when(dataset3.toDF).thenReturn(dataset3)
+when(dataset4.toDF).thenReturn(dataset4)
+when(dataset5.toDF).thenReturn(dataset5)
+
+when(estimator1.fit(meq(dataset1))).thenReturn(model1)
+when(model1.transform(meq(dataset1))).thenReturn(dataset2)
+when(model1.parent).thenReturn(estimator1)
+when(transformer1.transform(meq(dataset2))).thenReturn(dataset3)
+when(estimator2.fit(meq(dataset3))).thenReturn(model2)
+
+val pipeline = new Pipeline()
+ .setStages(Array(estimator1, transformer1, estimator2))
+assert(events.isEmpty)
+val pipelineModel = pipeline.fit(dataset1)
+val expected =
+ FitStart(pipeline, dataset1) ::
+ FitStart(estimator1, dataset1) ::
+ FitEnd(estimator1, model1) ::
+ TransformStart(model1, dataset1) ::
+ TransformEnd(model1, dataset2) ::
+ TransformStart(transformer1, dataset2) ::
+ TransformEnd(transformer1, dataset3) ::
+ FitStart(estimator2, dataset3) ::
+ FitEnd(estimator2, model2) ::
+ FitEnd(pipeline, pipelineModel) :: Nil
+eventually(timeout(10 seconds), interval(1 second)) {
+ assert(events === expected)
+}
+ }
+
+ test("pipeline model transform events") {
+val dataset1 = mock[DataFrame]
+val dataset2 = mock[DataFrame]
+val dataset3 = mock[DataFrame]
+val dataset4 = mock[DataFrame]
+when(dataset1.toDF).thenReturn(dataset1)
+when(dataset2.toDF).thenRetur
[GitHub] gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL] Cache manager locks
gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL]
Cache manager locks
URL: https://github.com/apache/spark/pull/23539#discussion_r249334149
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala
##
@@ -160,34 +167,48 @@ class CacheManager extends Logging {
/**
* Tries to re-cache all the cache entries that refer to the given plan.
*/
- def recacheByPlan(spark: SparkSession, plan: LogicalPlan): Unit = writeLock {
+ def recacheByPlan(spark: SparkSession, plan: LogicalPlan): Unit = {
recacheByCondition(spark, _.find(_.sameResult(plan)).isDefined)
}
private def recacheByCondition(
spark: SparkSession,
condition: LogicalPlan => Boolean,
clearCache: Boolean = true): Unit = {
-val it = cachedData.iterator()
val needToRecache = scala.collection.mutable.ArrayBuffer.empty[CachedData]
-while (it.hasNext) {
- val cd = it.next()
- if (condition(cd.plan)) {
-if (clearCache) {
- cd.cachedRepresentation.cacheBuilder.clearCache()
+readLock {
Review comment:
The first writeLock block should be something like
```
writeLock {
val it = cachedData.iterator()
while (it.hasNext) {
val cd = it.next()
if (cd.sessionUuidOpt == sessionUuidOpt && condition(cd.plan)) {
needToRecache += cd
it.remove()
}
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL] Cache manager locks
gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL]
Cache manager locks
URL: https://github.com/apache/spark/pull/23539#discussion_r249334042
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala
##
@@ -136,21 +137,27 @@ class CacheManager extends Logging {
spark: SparkSession,
plan: LogicalPlan,
cascade: Boolean,
- blocking: Boolean): Unit = writeLock {
+ blocking: Boolean): Unit = {
val shouldRemove: LogicalPlan => Boolean =
if (cascade) {
_.find(_.sameResult(plan)).isDefined
} else {
_.sameResult(plan)
}
-val it = cachedData.iterator()
-while (it.hasNext) {
- val cd = it.next()
- if (shouldRemove(cd.plan)) {
-cd.cachedRepresentation.cacheBuilder.clearCache(blocking)
-it.remove()
+val toRemove = mutable.Buffer[CachedData]()
+readLock {
Review comment:
The first protected block should be something like:
```Scala
writeLock {
val it = cachedData.iterator()
while (it.hasNext) {
val cd = it.next()
if (cd.sessionUuidOpt == sessionUuidOpt && condition(cd.plan)) {
needToRecache += cd
it.remove()
}
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL] Cache manager locks
gatorsmile commented on a change in pull request #23539: [SPARK-26617][SQL]
Cache manager locks
URL: https://github.com/apache/spark/pull/23539#discussion_r249334042
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala
##
@@ -136,21 +137,27 @@ class CacheManager extends Logging {
spark: SparkSession,
plan: LogicalPlan,
cascade: Boolean,
- blocking: Boolean): Unit = writeLock {
+ blocking: Boolean): Unit = {
val shouldRemove: LogicalPlan => Boolean =
if (cascade) {
_.find(_.sameResult(plan)).isDefined
} else {
_.sameResult(plan)
}
-val it = cachedData.iterator()
-while (it.hasNext) {
- val cd = it.next()
- if (shouldRemove(cd.plan)) {
-cd.cachedRepresentation.cacheBuilder.clearCache(blocking)
-it.remove()
+val toRemove = mutable.Buffer[CachedData]()
+readLock {
Review comment:
The first protected block should be something like:
```Scala
writeLock {
val it = cachedData.iterator()
while (it.hasNext) {
val cd = it.next()
if (cd.sessionUuidOpt == sessionUuidOpt && condition(cd.plan)) {
needToRecache += cd
it.remove()
}
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon edited a comment on issue #23588: Diversity
HyukjinKwon edited a comment on issue #23588: Diversity URL: https://github.com/apache/spark/pull/23588#issuecomment-455952639 I think you brought it first https://github.com/apache/spark/pull/23588#issuecomment-455819321 and I said to read it. The cost of merging it - it gives an example that those changes are encouraged. This PR itself isn't well-formed. See the description, title and what it proposes. Second, it costs Jenkins resource which is limited. When it runs multiple builds, it causes a global test failure which blocks all other PRs. Lastly reviewing cost. I would need to check if there are similar instances to fix at least. Additionally merging it as is might encourage other contributors to form PR like this and seek similar instances to fix. Why do you think this PR is worth to fix? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] shahidki31 commented on issue #23589: [SPARK-26351][mllib]Update doc and minor correction in the mllib evaluation metrics
shahidki31 commented on issue #23589: [SPARK-26351][mllib]Update doc and minor correction in the mllib evaluation metrics URL: https://github.com/apache/spark/pull/23589#issuecomment-455953799 Thank you @srowen This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23588: Diversity
HyukjinKwon commented on issue #23588: Diversity URL: https://github.com/apache/spark/pull/23588#issuecomment-455953583 More specifically that's an example. I think the example jsons already contain what to show enough. That's not even test case that's previously not covered. What's the gain of this PR? And why are you wasting your time here? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455952970 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101462/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455952967 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455952967 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455952970 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101462/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
SparkQA removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455934246 **[Test build #101462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101462/testReport)** for PR 23559 at commit [`1c72825`](https://github.com/apache/spark/commit/1c72825f7545c41a91e788d673b718b7fe24b1ca). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
SparkQA commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455952748 **[Test build #101462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101462/testReport)** for PR 23559 at commit [`1c72825`](https://github.com/apache/spark/commit/1c72825f7545c41a91e788d673b718b7fe24b1ca). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] HyukjinKwon commented on issue #23588: Diversity
HyukjinKwon commented on issue #23588: Diversity URL: https://github.com/apache/spark/pull/23588#issuecomment-455952639 I think you bought it first https://github.com/apache/spark/pull/23588#issuecomment-455819321 and I said to read it. The cost of merging it - it gives an example that those changes are encouraged. This PR itself isn't welcomed. See the description, title and what it proposes. Second, it costs Jenkins resource which is limited. When it runs multiple builds, it causes a global test failure which blocks all other PRs. Lastly reviewing cost. I would need to check if there are similar instances to fix at least. Additionally merging it as is might encourage other contributors to form PR like this and seek similar instances to fix. Why do you think this PR is worth to fix? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] cloud-fan commented on a change in pull request #23430: [SPARK-26520][SQL] data source v2 API refactor (micro-batch read)
cloud-fan commented on a change in pull request #23430: [SPARK-26520][SQL] data
source v2 API refactor (micro-batch read)
URL: https://github.com/apache/spark/pull/23430#discussion_r249327956
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
##
@@ -90,36 +88,39 @@ class MicroBatchExecution(
logInfo(s"Using Source [$source] from DataSourceV1 named
'$sourceName' [$dataSourceV1]")
StreamingExecutionRelation(source, output)(sparkSession)
})
- case s @ StreamingRelationV2(
-dataSourceV2: MicroBatchReadSupportProvider, sourceName, options,
output, _) if
- !disabledSources.contains(dataSourceV2.getClass.getCanonicalName) =>
-v2ToExecutionRelationMap.getOrElseUpdate(s, {
+ case s @ StreamingRelationV2(ds, dsName, table: SupportsMicroBatchRead,
options, output, _)
+ if !disabledSources.contains(ds.getClass.getCanonicalName) =>
+v2ToRelationMap.getOrElseUpdate(s, {
// Materialize source to avoid creating it in every batch
val metadataPath = s"$resolvedCheckpointRoot/sources/$nextSourceId"
- val readSupport = dataSourceV2.createMicroBatchReadSupport(
-metadataPath,
-new DataSourceOptions(options.asJava))
nextSourceId += 1
- readSupportToDataSourceMap(readSupport) = dataSourceV2 -> options
- logInfo(s"Using MicroBatchReadSupport [$readSupport] from " +
-s"DataSourceV2 named '$sourceName' [$dataSourceV2]")
- StreamingExecutionRelation(readSupport, output)(sparkSession)
+ logInfo(s"Reading table [$table] from DataSourceV2 named '$dsName'
[$ds]")
+ val dsOptions = new DataSourceOptions(options.asJava)
+ // TODO: operator pushdown.
Review comment:
It's not urgent, as no builtin streaming source supports pushdown yet. The
ds v2 API can handle pushdown, and our batch sources do support it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] Deegue commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
Deegue commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455950022 > BTW, to understand this issue easily, can you put a simple example query to reproduce this in the PR description above? Added an example to the PR description. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] viirya commented on a change in pull request #23263: [SPARK-23674][ML] Adds Spark ML Events to Instrumentation
viirya commented on a change in pull request #23263: [SPARK-23674][ML] Adds
Spark ML Events to Instrumentation
URL: https://github.com/apache/spark/pull/23263#discussion_r249325583
##
File path: mllib/src/main/scala/org/apache/spark/ml/Pipeline.scala
##
@@ -132,7 +133,8 @@ class Pipeline @Since("1.4.0") (
* @return fitted pipeline
Review comment:
Where should we document the ml events? Should we add doc at places we add
ml events like `Pipeline.fit`? Or write some doc in ml document?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948567 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/7262/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948567 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins removed a comment on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101460/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD
AmplabJenkins commented on issue #23559: [SPARK-26630][SQL] Fix ClassCastException in TableReader while creating HadoopRDD URL: https://github.com/apache/spark/pull/23559#issuecomment-455948394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/101460/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
