[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63549732 Okay pulling this in - thanks davies! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-18 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63598077 merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-18 Thread davies
Github user davies closed the pull request at: https://github.com/apache/spark/pull/3029 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467516 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467643 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467709 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467763 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467790 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467830 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467888 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467846 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467949 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20467968 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20468037 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63382985 Super cool. I left mostly minor comments. Otherwise it LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20468318 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20468419 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63383666 @davies @pwendell The three-stage solution looks reasonable to me! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20468644 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63385857 The new thing for multiple stages is really nice. I also think the new architecture is great. I made some minor comments, but overall looks good. On thing, does

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63388317 [Test build #23506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23506/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63389665 @pwendell @andrewor14 I should had addressed you comments, please have another look, thanks! This should work in SQL Cli, also in Windows. --- If your project

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63389796 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63399151 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63399144 [Test build #23506 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23506/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20414049 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20414069 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20414182 --- Diff: core/src/main/scala/org/apache/spark/ui/ConsoleProgressBar.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63244719 Hey @davies - I played with this a bit and I actually found the behavior around concurrent stages might not be great. The reason is that the set of active stages will

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63244854 By the way, @squito, I don't think this can go through log4j, it needs to access the jline console interface directly. This feature will just be controlled by a flag

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20414302 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -231,6 +231,13 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20414811 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -231,6 +231,13 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63249917 [Test build #23442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23442/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63251889 @pwendell @kayousterhout It can show mutiple stages (at most 3) in one line in the same time now, it looks like ``` [Stage 0: (316 + 4) /

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63251878 [Test build #23448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23448/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63253165 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63253164 [Test build #23442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23442/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63254820 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63254818 [Test build #23448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23448/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63257525 [Test build #23454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23454/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63261422 [Test build #523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/523/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63261770 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63261768 [Test build #23454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23454/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204439 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204436 [Test build #23434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23434/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204432 [Test build #23434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23434/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204653 [Test build #521 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/521/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204768 @JoshRosen @pwendell @kayousterhout @squito I had re-implemented it using the new poll based progress api. The progress bar is much simplified as the original one,

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204747 [Test build #23436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23436/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204911 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63204979 [Test build #23437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23437/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206309 [Test build #521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/521/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206407 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206404 [Test build #23436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23436/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206787 [Test build #23438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23438/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206812 [Test build #23437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23437/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63206817 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63208927 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console

2014-11-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63208925 [Test build #23438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23438/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-14 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-63149772 Sorry for my delay in responding ... (a) I think this DOES add a lot of value over the std INFO logging. One log line per task completion is *much* noisier than

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20112947 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -554,12 +627,11 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62454596 This is a very nifty feature. However, it's not great to have modifications to the TaskSetManager and other scheduler internals for this presentation-related logic. It

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20112968 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -569,6 +641,11 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r20112906 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -528,7 +601,7 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62458897 This is definitely a cool/useful feature! A few things: (1) As Patrick alluded to, I think it could and should be able to implemented as a SparkListener,

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62473806 @pwendell @kayousterhout , Thanks for review this. Console is a character device, all the graph are implemented as an stream of characters. It's very hard to show

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62474857 I'm not sure if this is a good idea, but what if the progress bar printed a new line each time it advanced, and just used the normal logging infrastructure? So

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62475673 I was just about to suggest the same thing . So I admit it seemed a lot cooler to have the console keep updating, but I agree with their concerns. As a slight

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62476320 @squito that's a good idea with the minimum time interval to avoid unnecessary clutter! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62480890 @kayousterhout @squito The motivation of this PR is to allow user to see the progress without flushing away their input/output, so it's better to have a one-line progress

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62499404 I totally see the appeal of the one-progress bar (hence my initial excitement when I tried this out). But if it doesn't play nicely with logging multiple stages, this

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-10 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62503514 But if it doesn't play nicely with logging multiple stages, this seems like a very small improvement for the initial user experience, but a big headache for serious

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-09 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-62330615 this is awesome! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61581372 [Test build #22845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22845/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61581788 [Test build #22846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22846/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61583786 [Test build #22850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22850/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61587953 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61587948 [Test build #22846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22846/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61588063 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61588061 [Test build #22845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22845/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61589372 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61589367 [Test build #22850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22850/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61359587 [Test build #22685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22685/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61359713 @JoshRosen I had make some improvements: 1) finished the bar before print the result in scala shell 2) can interwave with logging better(will not overwrite each other) 3)

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61360811 [Test build #22685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22685/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61360812 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r19702287 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -521,6 +529,51 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r19702293 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -521,6 +529,51 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r19702297 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -521,6 +529,51 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r19709497 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -521,6 +529,51 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61393355 [Test build #22737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22737/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61393375 [Test build #22737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22737/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61393376 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61393558 [Test build #22739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22739/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61394840 [Test build #22739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22739/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-11-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61394842 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-10-31 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3029#discussion_r19697163 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -521,6 +528,47 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61340878 [Test build #22656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22656/consoleFull) for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-10-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61349714 **[Test build #22656 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22656/consoleFull)** for PR 3029 at commit

[GitHub] spark pull request: [SPARK-4017] show progress bar in console and ...

2014-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3029#issuecomment-61349718 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   >