[GitHub] [spark] AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-569004255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115790/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-569004218 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-569004255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115790/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-569004218 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
SparkQA removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568972915 **[Test build #115790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115790/testReport)** for PR 27010 at commit [`223c466`](https://github.com/apache/spark/commit/223c46688bee6c8d237ae88d9ef5cd56abf45c39). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
AmplabJenkins removed a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569003062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20591/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
AmplabJenkins removed a comment on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012#issuecomment-569003032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20590/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
AmplabJenkins removed a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569003057 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
AmplabJenkins removed a comment on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012#issuecomment-569003026 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
AmplabJenkins commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012#issuecomment-569003026 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
AmplabJenkins commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569003062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20591/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
AmplabJenkins commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569003057 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
AmplabJenkins commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012#issuecomment-569003032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20590/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
SparkQA commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-569003094 **[Test build #115790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115790/testReport)** for PR 27010 at commit [`223c466`](https://github.com/apache/spark/commit/223c46688bee6c8d237ae88d9ef5cd56abf45c39). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
SparkQA commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569002775 **[Test build #115798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115798/testReport)** for PR 26924 at commit [`3d7f435`](https://github.com/apache/spark/commit/3d7f435f8452faff71b98a9163cd8e86e77c0a79). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
SparkQA commented on issue #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012#issuecomment-569002776 **[Test build #115797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115797/testReport)** for PR 27012 at commit [`8860691`](https://github.com/apache/spark/commit/8860691786887ea0ce71ff58922a163cdf32aabc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
Ngone51 commented on a change in pull request #26924: [SPARK-30285][CORE] Fix
deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
URL: https://github.com/apache/spark/pull/26924#discussion_r361392351
##
File path:
core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala
##
@@ -529,6 +529,46 @@ class SparkListenerSuite extends SparkFunSuite with
LocalSparkContext with Match
}
}
+ Seq(true, false).foreach { throwInterruptedException =>
+val suffix = if (throwInterruptedException) "throw interrupt" else "set
Thread interrupted"
+test(s"SPARK-30285: Fix deadlock in AsyncEventQueue.removeListenerOnError:
$suffix") {
+ val conf = new SparkConf(false)
+.set(LISTENER_BUS_EVENT_QUEUE_CAPACITY, 5)
+ val bus = new LiveListenerBus(conf)
+ val counter1 = new BasicJobCounter()
+ val counter2 = new BasicJobCounter()
+ val interruptingListener = new
DelayInterruptingJobCounter(throwInterruptedException, 3)
+ bus.addToSharedQueue(counter1)
+ bus.addToSharedQueue(interruptingListener)
+ bus.addToEventLogQueue(counter2)
+ assert(bus.activeQueues() === Set(SHARED_QUEUE, EVENT_LOG_QUEUE))
+ assert(bus.findListenersByClass[BasicJobCounter]().size === 2)
+ assert(bus.findListenersByClass[DelayInterruptingJobCounter]().size ===
1)
+
+ bus.start(mockSparkContext, mockMetricsSystem)
+
+ (0 until 5).foreach { jobId =>
+bus.post(SparkListenerJobEnd(jobId, jobCompletionTime, JobSucceeded))
+ }
+
+ // Call bus.stop in a separate thread, otherwise we will block here
until bus is stopped
+ val stoppingThread = new Thread(() => {
+bus.stop()
+ })
+ stoppingThread.start()
+ // Notify interrupting listener starts to work
+ interruptingListener.sleep = false
Review comment:
Maybe, you should insert `CountDownLatch` after `bus.stop`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
Ngone51 commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569002486 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 opened a new pull request #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint
Ngone51 opened a new pull request #27012: [SPARK-30355][CORE] Unify isExecutorActive between CoarseGrainedSchedulerBackend and DriverEndpoint URL: https://github.com/apache/spark/pull/27012 ### What changes were proposed in this pull request? Unify `DriverEndpoint. executorIsAlive()` and `CoarseGrainedSchedulerBackend .isExecutorActive()`. ### Why are the changes needed? `DriverEndPoint` has method `executorIsAlive()` to check wether an executor is alive/active, while `CoarseGrainedSchedulerBackend` has method `isExecutorActive()` to do the same work. But, `isExecutorActive()` seems forget to consider `executorsPendingLossReason`. Unify these two methods makes behavior be consistent between `DriverEndPoint` and `CoarseGrainedSchedulerBackend` and make code more easier to maintain. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Pass Jenkins. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangshuo128 edited a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
wangshuo128 edited a comment on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569000267 @Ngone51 Would you please trigger the test again? I think the test failed due to a flaky test, see #27010 for details. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangshuo128 commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
wangshuo128 commented on issue #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError URL: https://github.com/apache/spark/pull/26924#issuecomment-569000267 @Ngone51 Would you please trigger the test again? I think the test failed due to #27010 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bmarcott edited a comment on issue #26696: [WIP][SPARK-18886][CORE] Make locality wait time be the time since a TSM's available slots were fully utilized
bmarcott edited a comment on issue #26696: [WIP][SPARK-18886][CORE] Make locality wait time be the time since a TSM's available slots were fully utilized URL: https://github.com/apache/spark/pull/26696#issuecomment-568617329 I think I came up with a much better approach [here](https://github.com/apache/spark/compare/master...bmarcott:nmarcott-fulfill-slots-2?expand=1). It avoids trying to simulate scheduling logic like the previous approach, which had a lot of discrepancies as well as high time complexity. This change makes the `TaskSetManager.resourceOffer` return an explicit boolean saying whether it rejected the resource due to delay scheduling or not An `isAllFreeResources` boolean parameter was also added to `TaskSchedulerImpl.resourceOffers` which tells the scheduler the offers represent all free resources as opposed to a single resource. Then, timers will be reset only if there were no resources rejected due to scheduling delay since the last offer which included all free resources. example case: offer 1 resource that was rejected - no timer reset offer all resources with no rejects - timer is reset offer 1 resource, no reject - timer is reset offer 1 resource that was rejected - no timer reset offer 1 resource, no reject - no timer reset because previous offer was rejected Here is a breakdown of when resources are offered (not changed): Single executors are offered when: - a task finishes - new executor launched All free resources are offered when: - continually every spark.scheduler.revive.interval seconds (default 1 second) - on taskset submit - when a task fails - speculationScheduler on fixed delay revives if there are any speculative tasks - executor lost One remaining case that isn't handled: Before any "all free resource" offer, all free resources are offered one by one and all not rejected. This case should reset the timer, but won't with current impl. Thoughts or know of any other issues with this approach? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568999519 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115791/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568999517 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
SparkQA removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974156 **[Test build #115791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115791/testReport)** for PR 26978 at commit [`06d2b80`](https://github.com/apache/spark/commit/06d2b80e0ab49bf3a673cc00cbb64cc95772e943). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
SparkQA commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568999445 **[Test build #115791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115791/testReport)** for PR 26978 at commit [`06d2b80`](https://github.com/apache/spark/commit/06d2b80e0ab49bf3a673cc00cbb64cc95772e943). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568999519 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115791/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568999517 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bmarcott edited a comment on issue #26696: [WIP][SPARK-18886][CORE] Make locality wait time be the time since a TSM's available slots were fully utilized
bmarcott edited a comment on issue #26696: [WIP][SPARK-18886][CORE] Make locality wait time be the time since a TSM's available slots were fully utilized URL: https://github.com/apache/spark/pull/26696#issuecomment-568617329 [Here is an approach](https://github.com/apache/spark/compare/master...bmarcott:nmarcott-fulfill-slots-2?expand=1) which avoids trying to simulate any type of scheduling logic. This change introduces a `isAllFreeResources` boolean parameter for `TaskSchedulerImpl.resourceOffers` which tells the scheduler the offers represent all free resources as opposed to a single resource. Timers will be reset only if there were no resources rejected due to scheduling delay since the last offer which included all free resources. example case: offer 1 resource that was rejected - no timer reset offer all resources with no rejects - timer is reset offer 1 resource, no reject - timer is reset offer 1 resource that was rejected - no timer reset offer 1 resource, no reject - no timer reset because previous offer was rejected Here is a breakdown of when resources are offered (not changed): Single executors are offered when: - a task finishes - new executor launched All free resources are offered when: - continually every spark.scheduler.revive.interval seconds (default 1 second) - on taskset submit - when a task fails - speculationScheduler on fixed delay revives if there are any speculative tasks - executor lost One remaining case that isn't handled: Before any "all free resource" offer, all free resources are offered one by one and all not rejected. This case should reset the timer, but won't with current impl. Thoughts or know of any other issues with this approach? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangshuo128 commented on a change in pull request #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
wangshuo128 commented on a change in pull request #26924: [SPARK-30285][CORE]
Fix deadlock between LiveListenerBus#stop and
AsyncEventQueue#removeListenerOnError
URL: https://github.com/apache/spark/pull/26924#discussion_r361388539
##
File path:
core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala
##
@@ -529,6 +529,46 @@ class SparkListenerSuite extends SparkFunSuite with
LocalSparkContext with Match
}
}
+ Seq(true, false).foreach { throwInterruptedException =>
+val suffix = if (throwInterruptedException) "throw interrupt" else "set
Thread interrupted"
+test(s"SPARK-30285: Fix deadlock in AsyncEventQueue.removeListenerOnError:
$suffix") {
+ val conf = new SparkConf(false)
+.set(LISTENER_BUS_EVENT_QUEUE_CAPACITY, 5)
+ val bus = new LiveListenerBus(conf)
+ val counter1 = new BasicJobCounter()
+ val counter2 = new BasicJobCounter()
+ val interruptingListener = new
DelayInterruptingJobCounter(throwInterruptedException, 3)
+ bus.addToSharedQueue(counter1)
+ bus.addToSharedQueue(interruptingListener)
+ bus.addToEventLogQueue(counter2)
+ assert(bus.activeQueues() === Set(SHARED_QUEUE, EVENT_LOG_QUEUE))
+ assert(bus.findListenersByClass[BasicJobCounter]().size === 2)
+ assert(bus.findListenersByClass[DelayInterruptingJobCounter]().size ===
1)
+
+ bus.start(mockSparkContext, mockMetricsSystem)
+
+ (0 until 5).foreach { jobId =>
+bus.post(SparkListenerJobEnd(jobId, jobCompletionTime, JobSucceeded))
+ }
+
+ // Call bus.stop in a separate thread, otherwise we will block here
until bus is stopped
+ val stoppingThread = new Thread(() => {
+bus.stop()
+ })
+ stoppingThread.start()
+ // Notify interrupting listener starts to work
+ interruptingListener.sleep = false
Review comment:
As the PR description, to reproduce the original issue, we have to make sure:
1. Holding the synchronized lock of `bus` in the stopping thread
2. Trying to acquire the synchronized lock of `bus` in the interrupting
listener thread
But signal the listener starts to interrupt just before `bus.stop` by a
`CountDownLatch` can't guarantee this 100%, right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
AmplabJenkins removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568996533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115783/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
SparkQA removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568957704 **[Test build #115783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115783/testReport)** for PR 26809 at commit [`40d0740`](https://github.com/apache/spark/commit/40d0740b80b2008f2520e6ccb69bd8faae9c4c06). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
AmplabJenkins removed a comment on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568996525 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
AmplabJenkins commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568996525 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
AmplabJenkins commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568996533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115783/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API
SparkQA commented on issue #26809: [SPARK-30185][SQL] Implement Dataset.tail API URL: https://github.com/apache/spark/pull/26809#issuecomment-568996251 **[Test build #115783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115783/testReport)** for PR 26809 at commit [`40d0740`](https://github.com/apache/spark/commit/40d0740b80b2008f2520e6ccb69bd8faae9c4c06). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
AmplabJenkins removed a comment on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011#issuecomment-568991892 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
AmplabJenkins removed a comment on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011#issuecomment-568991896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20589/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
AmplabJenkins commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011#issuecomment-568991896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20589/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
AmplabJenkins commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011#issuecomment-568991892 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #26924: [SPARK-30285][CORE] Fix deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
Ngone51 commented on a change in pull request #26924: [SPARK-30285][CORE] Fix
deadlock between LiveListenerBus#stop and AsyncEventQueue#removeListenerOnError
URL: https://github.com/apache/spark/pull/26924#discussion_r361383627
##
File path:
core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala
##
@@ -529,6 +529,46 @@ class SparkListenerSuite extends SparkFunSuite with
LocalSparkContext with Match
}
}
+ Seq(true, false).foreach { throwInterruptedException =>
+val suffix = if (throwInterruptedException) "throw interrupt" else "set
Thread interrupted"
+test(s"SPARK-30285: Fix deadlock in AsyncEventQueue.removeListenerOnError:
$suffix") {
+ val conf = new SparkConf(false)
+.set(LISTENER_BUS_EVENT_QUEUE_CAPACITY, 5)
+ val bus = new LiveListenerBus(conf)
+ val counter1 = new BasicJobCounter()
+ val counter2 = new BasicJobCounter()
+ val interruptingListener = new
DelayInterruptingJobCounter(throwInterruptedException, 3)
+ bus.addToSharedQueue(counter1)
+ bus.addToSharedQueue(interruptingListener)
+ bus.addToEventLogQueue(counter2)
+ assert(bus.activeQueues() === Set(SHARED_QUEUE, EVENT_LOG_QUEUE))
+ assert(bus.findListenersByClass[BasicJobCounter]().size === 2)
+ assert(bus.findListenersByClass[DelayInterruptingJobCounter]().size ===
1)
+
+ bus.start(mockSparkContext, mockMetricsSystem)
+
+ (0 until 5).foreach { jobId =>
+bus.post(SparkListenerJobEnd(jobId, jobCompletionTime, JobSucceeded))
+ }
+
+ // Call bus.stop in a separate thread, otherwise we will block here
until bus is stopped
+ val stoppingThread = new Thread(() => {
+bus.stop()
+ })
+ stoppingThread.start()
+ // Notify interrupting listener starts to work
+ interruptingListener.sleep = false
Review comment:
`CountDownLatch` always make things deterministic and it sounds better to me.
What do you mean by "it can't get rid of racing completely"?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
SparkQA commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011#issuecomment-568991555 **[Test build #115796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115796/testReport)** for PR 27011 at commit [`b04b17d`](https://github.com/apache/spark/commit/b04b17dcae703cd68c9f57fd95411d014b6f50b1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #26972: [SPARK-30321][Ml] Log weightSum in Algo that has weights support
zhengruifeng commented on issue #26972: [SPARK-30321][Ml] Log weightSum in Algo that has weights support URL: https://github.com/apache/spark/pull/26972#issuecomment-568991530 @huaxingao Can we pass the instr of GBTClassifier/GBTRegressor into the internal impl, and use it to log the metrics in the first tree building? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
zhengruifeng commented on issue #27011: [SPARK-30354][ML] GBT reuse
DecisionTreeMetadata among iterations
URL: https://github.com/apache/spark/pull/27011#issuecomment-568991035
testcode:
```scala
import org.apache.spark.ml.regression._
import org.apache.spark.storage.StorageLevel
var df = spark.read.format("libsvm").load("/data1/Datasets/a9a/a9a")
(0 until 8).foreach{ _ => df = df.union(df) }
df.persist(StorageLevel.MEMORY_AND_DISK)
df.count
df.count
df.count
val gbt = new GBTRegressor().setMaxIter(10)
val gbtm = gbt.fit(df)
val start = System.currentTimeMillis; val gbtm = gbt.fit(df); val end =
System.currentTimeMillis; end - start
```
reusing treeMetadata in this PR: 176969
master: 191657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng opened a new pull request #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations
zhengruifeng opened a new pull request #27011: [SPARK-30354][ML] GBT reuse DecisionTreeMetadata among iterations URL: https://github.com/apache/spark/pull/27011 ### What changes were proposed in this pull request? precompute the `DecisionTreeMetadata` and reuse it for all trees ### Why are the changes needed? Each `DecisionTreeRegressor` needs a pass on the whole dataset to calculate the same `DecisionTreeMetadata`. with default depth=5, it is about 8% faster then existing impl ### Does this PR introduce any user-facing change? No ### How was this patch tested? existing testsuites This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568983409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115793/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568983404 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
SparkQA removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977208 **[Test build #115793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115793/testReport)** for PR 26991 at commit [`d8f5b5c`](https://github.com/apache/spark/commit/d8f5b5c38bae57e6ee7ff3e3bf0c267650ea086d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568983404 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568983409 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115793/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
SparkQA commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568983352 **[Test build #115793 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115793/testReport)** for PR 26991 at commit [`d8f5b5c`](https://github.com/apache/spark/commit/d8f5b5c38bae57e6ee7ff3e3bf0c267650ea086d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart
AmplabJenkins commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart URL: https://github.com/apache/spark/pull/24173#issuecomment-568979461 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart
AmplabJenkins commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart URL: https://github.com/apache/spark/pull/24173#issuecomment-568979464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20588/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart
AmplabJenkins removed a comment on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart URL: https://github.com/apache/spark/pull/24173#issuecomment-568979461 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart
AmplabJenkins removed a comment on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart URL: https://github.com/apache/spark/pull/24173#issuecomment-568979464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20588/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart
SparkQA commented on issue #24173: [SPARK-27237][SS] Introduce State schema validation among query restart URL: https://github.com/apache/spark/pull/24173#issuecomment-568979141 **[Test build #115795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115795/testReport)** for PR 24173 at commit [`1fcfff5`](https://github.com/apache/spark/commit/1fcfff5c2ca78049eb38cf4ef7c041d0005ab9b3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568978313 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568978321 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115784/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568978313 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568978321 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115784/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
SparkQA removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568957706 **[Test build #115784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115784/testReport)** for PR 27004 at commit [`c4775fc`](https://github.com/apache/spark/commit/c4775fc0b6d843a725a370095750b2ef86ae6c2f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
SparkQA commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568977958 **[Test build #115784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115784/testReport)** for PR 27004 at commit [`c4775fc`](https://github.com/apache/spark/commit/c4775fc0b6d843a725a370095750b2ef86ae6c2f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20587/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977617 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins removed a comment on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977617 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
AmplabJenkins commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20587/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup
SparkQA commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#issuecomment-568977229 **[Test build #115794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115794/testReport)** for PR 26416 at commit [`e1a6e42`](https://github.com/apache/spark/commit/e1a6e42b73f8d58dbc0a04882f2288d2fae0dad8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package
SparkQA commented on issue #26991: [SPARK-30336][SQL][SS] Move Kafka consumer-related classes to its own package URL: https://github.com/apache/spark/pull/26991#issuecomment-568977208 **[Test build #115793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115793/testReport)** for PR 26991 at commit [`d8f5b5c`](https://github.com/apache/spark/commit/d8f5b5c38bae57e6ee7ff3e3bf0c267650ea086d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup
HeartSaVioR commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#issuecomment-568976301 retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup
HeartSaVioR commented on issue #26416: [SPARK-29779][CORE] Compact old event log files and cleanup URL: https://github.com/apache/spark/pull/26416#issuecomment-568976275 Test failure for 115781 was org.apache.spark.sql.kafka010.KafkaDelegationTokenSuite.(It is not a test it is a sbt.testing.SuiteSelector), not relevant This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568975998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568975994 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568975994 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
AmplabJenkins removed a comment on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568975998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20586/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
SparkQA commented on issue #27004: [SPARK-30348][CORE][TEST] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..." URL: https://github.com/apache/spark/pull/27004#issuecomment-568975518 **[Test build #115792 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115792/testReport)** for PR 27004 at commit [`8b7b1a1`](https://github.com/apache/spark/commit/8b7b1a1f7dae324aeff676340567c8c319604236). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ulysses-you commented on issue #26875: [SPARK-30245][SQL] Add cache for Like and RLike when pattern is not static
ulysses-you commented on issue #26875: [SPARK-30245][SQL] Add cache for Like and RLike when pattern is not static URL: https://github.com/apache/spark/pull/26875#issuecomment-568974622 > @ulysses-you can you show the exact steps and perf numbers in the PR description? Sure I will add it later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974420 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins removed a comment on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20585/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974420 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
AmplabJenkins commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20585/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project
SparkQA commented on issue #26978: [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without Project URL: https://github.com/apache/spark/pull/26978#issuecomment-568974156 **[Test build #115791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115791/testReport)** for PR 26978 at commit [`06d2b80`](https://github.com/apache/spark/commit/06d2b80e0ab49bf3a673cc00cbb64cc95772e943). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #27004: [SPARK-30348][CORE] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
HeartSaVioR commented on a change in pull request #27004: [SPARK-30348][CORE]
Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
URL: https://github.com/apache/spark/pull/27004#discussion_r361371157
##
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##
@@ -97,13 +97,40 @@ class MockWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf) extend
}
}
-class MockExecutorLaunchFailWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf)
- extends MockWorker(master, conf) {
+// This class is designed to handle the lifecycle of only one application.
+class MockExecutorLaunchFailWorker(master: Master, conf: SparkConf = new
SparkConf)
+ extends MockWorker(master.self, conf) with Eventually {
+
+ val appRegistered = new CountDownLatch(1)
+ val launchExecutorReceived = new CountDownLatch(1)
+ val appIdsToLaunchExecutor = new mutable.HashSet[String]
var failedCnt = 0
+
override def receive: PartialFunction[Any, Unit] = {
+case LaunchDriver(driverId, desc, resources_) =>
Review comment:
Good point. Will address.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
HeartSaVioR commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568973827 Hmm... Does it mean the comment in below is broken? The code comment says `self` will become valid when `onStart` is called, but that doesn't seem to be true - self will become valid "around" when onStart is called and there's no guarantee that self is valid in onStart. https://github.com/apache/spark/blob/481fb63f97d87d5b2e9e1f9b30bee466605b5a72/core/src/main/scala/org/apache/spark/rpc/RpcEndpoint.scala#L53-L63 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568973131 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20584/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins removed a comment on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568973127 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568973131 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20584/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
AmplabJenkins commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568973127 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
SparkQA commented on issue #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010#issuecomment-568972915 **[Test build #115790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115790/testReport)** for PR 27010 at commit [`223c466`](https://github.com/apache/spark/commit/223c46688bee6c8d237ae88d9ef5cd56abf45c39). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
HeartSaVioR commented on a change in pull request #27010: [SPARK-30313][CORE]
Ensure EndpointRef is available MasterWebUI/WorkerPage
URL: https://github.com/apache/spark/pull/27010#discussion_r361370210
##
File path:
core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala
##
@@ -33,7 +33,12 @@ import org.apache.spark.ui.{UIUtils, WebUIPage}
import org.apache.spark.util.Utils
private[ui] class WorkerPage(parent: WorkerWebUI) extends WebUIPage("") {
- private val workerEndpoint = parent.worker.self
+ private val workerEndpoint = {
Review comment:
Same here: if we don't feel comfortable adding infinite loop, we can just
change it from val to def, with adding comment it shouldn't be cached.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
HeartSaVioR commented on a change in pull request #27010: [SPARK-30313][CORE]
Ensure EndpointRef is available MasterWebUI/WorkerPage
URL: https://github.com/apache/spark/pull/27010#discussion_r361370176
##
File path:
core/src/main/scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala
##
@@ -34,7 +34,12 @@ class MasterWebUI(
extends WebUI(master.securityMgr,
master.securityMgr.getSSLOptions("standalone"),
requestedPort, master.conf, name = "MasterUI") with Logging {
- val masterEndpointRef = master.self
+ val masterEndpointRef = {
Review comment:
If we don't feel comfortable adding infinite loop, we can just change it
from val to def, with adding comment it shouldn't be cached.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR opened a new pull request #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage
HeartSaVioR opened a new pull request #27010: [SPARK-30313][CORE] Ensure EndpointRef is available MasterWebUI/WorkerPage URL: https://github.com/apache/spark/pull/27010 ### What changes were proposed in this pull request? This patch fixes flaky tests "master/worker web ui available" & "master/worker web ui available with reverseProxy" in MasterSuite. Tracking back from stack trace, I found there's possible race condition in `Dispatcher.registerRpcEndpoint()`: https://github.com/apache/spark/blob/481fb63f97d87d5b2e9e1f9b30bee466605b5a72/core/src/main/scala/org/apache/spark/rpc/netty/Dispatcher.scala#L64-L77 `getMessageLoop()` initializes a new Inbox for this endpoint for both DedicatedMessageLoop and SharedMessageLoop, which calls `onStart()` "asynchronously" and "eventually" via posting `OnStart` message. `onStart()` will initialize UI page instance(s), so the execution of `endpointRefs.put()` and initializing UI page instance(s) are "concurrent". MasterPage and WorkerPage retrieve endpoint ref and store it as "val" assuming endpoint ref is valid when they're initialized - so in bad case they could store "null" as endpoint ref, and don't change. https://github.com/apache/spark/blob/481fb63f97d87d5b2e9e1f9b30bee466605b5a72/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala#L33-L38 https://github.com/apache/spark/blob/481fb63f97d87d5b2e9e1f9b30bee466605b5a72/core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala#L35-L41 This patch simply adds loop to ensure endpoint ref is available when retrieving from MasterPage/WorkerPage, given the fact that endpoint ref assignment in Dispatcher won't take long. ### Why are the changes needed? We observed the test failures from Jenkins; below are the links: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115583/testReport/ https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115700/testReport/ ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Existing UTs. Unfortunately it's hard to reproduce as it's due to race condition and it barely fails. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27004: [SPARK-30348][CORE] Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
Ngone51 commented on a change in pull request #27004: [SPARK-30348][CORE] Fix
flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
URL: https://github.com/apache/spark/pull/27004#discussion_r361369613
##
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##
@@ -97,13 +97,40 @@ class MockWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf) extend
}
}
-class MockExecutorLaunchFailWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf)
- extends MockWorker(master, conf) {
+// This class is designed to handle the lifecycle of only one application.
+class MockExecutorLaunchFailWorker(master: Master, conf: SparkConf = new
SparkConf)
+ extends MockWorker(master.self, conf) with Eventually {
+
+ val appRegistered = new CountDownLatch(1)
+ val launchExecutorReceived = new CountDownLatch(1)
+ val appIdsToLaunchExecutor = new mutable.HashSet[String]
var failedCnt = 0
+
override def receive: PartialFunction[Any, Unit] = {
+case LaunchDriver(driverId, desc, resources_) =>
Review comment:
nit: Personally, I'd prefer `_` for those unused fields.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #26981: [SPARK-26389][SS][FOLLOW-UP]Format config name to follow the other boolean conf naming convention
Ngone51 commented on issue #26981: [SPARK-26389][SS][FOLLOW-UP]Format config name to follow the other boolean conf naming convention URL: https://github.com/apache/spark/pull/26981#issuecomment-568971426 Thanks all! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #26998: [SPARK-25855][CORE][FOLLOW-UP] Format config name to follow the other boolean conf naming convention
Ngone51 commented on issue #26998: [SPARK-25855][CORE][FOLLOW-UP] Format config name to follow the other boolean conf naming convention URL: https://github.com/apache/spark/pull/26998#issuecomment-568971473 Thanks all! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
AmplabJenkins commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568968461 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
AmplabJenkins commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568968469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115780/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
AmplabJenkins removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568968461 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
AmplabJenkins removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568968469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115780/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
SparkQA removed a comment on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568942102 **[Test build #115780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115780/testReport)** for PR 27005 at commit [`464255e`](https://github.com/apache/spark/commit/464255e906cffda44950364688b135a8ab4635a3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object
SparkQA commented on issue #27005: [SPARK-30350][SQL] Fix ScalaReflection to use an empty array for getting its class object URL: https://github.com/apache/spark/pull/27005#issuecomment-568968241 **[Test build #115780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115780/testReport)** for PR 27005 at commit [`464255e`](https://github.com/apache/spark/commit/464255e906cffda44950364688b135a8ab4635a3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
