[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16272 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16272 **[Test build #70221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70221/testReport)** for PR 16272 at commit

[GitHub] spark issue #16276: [SPARK-18855][CORE] Add RDD flatten function

2016-12-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16276 Thanks @linbojin ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r92740284 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -331,7 +331,15 @@ private[spark] class MemoryStore( var

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16301 Yeah, ```org.apache.spark.mllib.stat.test.KolmogorovSmirnovTest``` is the path, however, ```stat``` is the same level with ```classification, regression, clustering, etc``` in the directory

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-15 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r92739704 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -60,7 +60,7 @@ class MemoryStoreSuite SizeEstimator

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r92739413 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -60,7 +60,7 @@ class MemoryStoreSuite SizeEstimator

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-15 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r92739151 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -60,7 +60,7 @@ class MemoryStoreSuite SizeEstimator

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-15 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r92739178 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -331,7 +331,15 @@ private[spark] class MemoryStore( var

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16281 Thank you for confirming, @rdblue . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15335 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15335 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70217/ Test FAILed. ---

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15335 **[Test build #70217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70217/testReport)** for PR 15335 at commit

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16281 I think we should move to a 1.8.2 patch release. The reason is that 1.9.0 moved to ByteBuffer based reads and we've found at least one problem with it. ByteBuffer based reads also changes an

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16281 Thanks @dongjoon-hyun! Lets get a Parquet 1.8.2 out in January. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16301 Sure? I got that from the Scala API namespace. There might be more test stuff we could add in the future? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16281 Sure. We are going to wait for Parquet 1.8.2. Maybe, in a month for Spark 2.1.1 on January? Anyway, thank you all! I'm closing this PR happily. --- If your project is set up for it,

[GitHub] spark pull request #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/16281 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16142: [SPARK-18716][CORE] Restrict the disk usage of spark eve...

2016-12-15 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16142 Besides, the unit test has proved that the older file will be cleaned first. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16142: [SPARK-18716][CORE] Restrict the disk usage of spark eve...

2016-12-15 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16142 @vanzin > For the feature, it feels like it's trying to make the SHS more like a "log management system" than a history server. Sorry, I do not get it. I just add a new clean-up

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16301 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70224/ Test PASSed. ---

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16301 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16263: [SPARK-18281][SQL][PySpark] Consumes the returned local ...

2016-12-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16263 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16301 **[Test build #70224 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70224/testReport)** for PR 16301 at commit

[GitHub] spark issue #16263: [SPARK-18281][SQL][PySpark] Consumes the returned local ...

2016-12-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16263 @holdenk @dusenberrymw There are timeouts for connection and reading data. If we want to keep the connection timeout, which I think is reasonable, eagerly evaluating of the first element is

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16302 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16302 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70225/ Test FAILed. ---

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16302 **[Test build #70225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70225/testReport)** for PR 16302 at commit

[GitHub] spark pull request #16276: [SPARK-18855][CORE] Add RDD flatten function

2016-12-15 Thread linbojin
Github user linbojin closed the pull request at: https://github.com/apache/spark/pull/16276 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16276: [SPARK-18855][CORE] Add RDD flatten function

2016-12-15 Thread linbojin
Github user linbojin commented on the issue: https://github.com/apache/spark/pull/16276 @rxin Thx, I just deep dive into spark, I hope i can try to contribute more impactful things later 😄 . I will close this one. @srowen I tried out your idea that i created

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16281 As we understand the Parquet community is willing to put out patch releases, I don't see any major reason now to motivate the forking. That is a great news for us. So do we postpone the upgrading to

[GitHub] spark issue #16232: [SPARK-18800][SQL] Fix UnsafeKVExternalSorter by correct...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16232 **[Test build #70230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70230/testReport)** for PR 16232 at commit

[GitHub] spark issue #16282: [DO_NOT_MERGE]Try to fix kafka

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16282 **[Test build #70229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70229/testReport)** for PR 16282 at commit

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16281 @rdblue Thanks again! : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15326: [SPARK-17759] [CORE] Avoid adding duplicate schedulables

2016-12-15 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15326 @erenavsarogullari I looked at this again (sorry for the long delay), and it looks like you maintained the old behavior of last-added-wins. I thought from the discussion above, @markhamstra

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16281 Great! I'm glad it was just confusion. I completely agree with @srowen that forking should be a last resort. In the future, please reach out to the community, whether its Parquet or

[GitHub] spark issue #16282: [DO_NOT_MERGE]Try to fix kafka

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70210/ Test FAILed. ---

[GitHub] spark issue #16282: [DO_NOT_MERGE]Try to fix kafka

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16303: [SPARK-17807][core] Demote scalatest to "provided" in sp...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16303 **[Test build #70228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70228/testReport)** for PR 16303 at commit

[GitHub] spark issue #16282: [DO_NOT_MERGE]Try to fix kafka

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16282 **[Test build #70210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70210/testReport)** for PR 16282 at commit

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70218/ Test FAILed. ---

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70218/testReport)** for PR 16294 at commit

[GitHub] spark pull request #16303: [SPARK-17807][core] Demote scalatest to "provided...

2016-12-15 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/16303 [SPARK-17807][core] Demote scalatest to "provided" in spark-tags. This avoids exposing scalatest as a transitive dependencies to projects that depend on spark-core, but don't declare a direct

[GitHub] spark pull request #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final ch...

2016-12-15 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16301#discussion_r92735185 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -447,31 +447,43 @@ head(teenagers) SparkR supports the following machine learning models

[GitHub] spark issue #16249: [SPARK-18828][SPARKR] Refactor scripts for R

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #70226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70226/consoleFull)** for PR 16249 at commit

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #70227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70227/testReport)** for PR 16173 at commit

[GitHub] spark pull request #16302: [MINOR] Handle fact that mv is different on linux...

2016-12-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16302 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-15 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16173 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16281 @julienledem @rdblue wow, that is so great!!! It will be much easier for us! I never expected the Parquet community is willing to do the regular patch releases. Sorry, that statement is

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16302 whoa? :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16302 Merging into master/branch-2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14512: [WIP] Add SSL support to dispatcher and spark-sub...

2016-12-15 Thread mgummelt
Github user mgummelt closed the pull request at: https://github.com/apache/spark/pull/14512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16302 **[Test build #70225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70225/testReport)** for PR 16302 at commit

[GitHub] spark issue #16302: [MINOR] Handle fact that mv is different on linux, mac

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16302 FYI @felixcheung @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16302: [MINOR] Handle fact that mv is different on linux...

2016-12-15 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/16302 [MINOR] Handle fact that mv is different on linux, mac Follow up to https://github.com/apache/spark/commit/ae853e8f3bdbd16427e6f1ffade4f63abaf74abb as `mv` throws an error on the Jenkins

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16301 **[Test build #70224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70224/testReport)** for PR 16301 at commit

[GitHub] spark issue #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final check reo...

2016-12-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16301 @mengxr @shivaram --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16301: [SPARK-18849][ML][SPARKR][DOC] vignettes final ch...

2016-12-15 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/16301 [SPARK-18849][ML][SPARKR][DOC] vignettes final check reorg ## What changes were proposed in this pull request? Reorganizing content (copy/paste) ## How was this patch tested?

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16287 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70208/ Test FAILed. ---

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16287 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16300: [SPARK-18892][SQL] Alias percentile_approx approx_percen...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16300 **[Test build #70223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70223/testReport)** for PR 16300 at commit

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16287 **[Test build #70208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70208/testReport)** for PR 16287 at commit

[GitHub] spark pull request #16300: [SPARK-18892][SQL] Alias percentile_approx approx...

2016-12-15 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/16300 [SPARK-18892][SQL] Alias percentile_approx approx_percentile ## What changes were proposed in this pull request? percentile_approx is the name used in Hive, and approx_percentile is the name used

[GitHub] spark pull request #16253: [SPARK-18537][Web UI] Add a REST api to serve spa...

2016-12-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16253#discussion_r92731224 --- Diff: streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/AllReceiversResource.scala --- @@ -0,0 +1,77 @@ +/* + * Licensed to the

[GitHub] spark pull request #16253: [SPARK-18537][Web UI] Add a REST api to serve spa...

2016-12-15 Thread saturday-shi
Github user saturday-shi commented on a diff in the pull request: https://github.com/apache/spark/pull/16253#discussion_r92730696 --- Diff: streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/AllReceiversResource.scala --- @@ -0,0 +1,77 @@ +/* + * Licensed

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a schema inconsistent bug that ma...

2016-12-15 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16030 oh, I wrongly wrote in an opposite way..., okay, fixed cc: @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16272: [SPARK-18850][SS]Make StreamExecution and progres...

2016-12-15 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16272#discussion_r92729192 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala --- @@ -244,7 +244,7 @@ class StreamingQueryManagerSuite

[GitHub] spark pull request #16299: [MINOR] Only rename SparkR tar.gz if names mismat...

2016-12-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16299 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16253: [SPARK-18537][Web UI] Add a REST api to serve spa...

2016-12-15 Thread saturday-shi
Github user saturday-shi commented on a diff in the pull request: https://github.com/apache/spark/pull/16253#discussion_r92727999 --- Diff: project/MimaExcludes.scala --- @@ -116,7 +116,10 @@ object MimaExcludes {

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16299 **[Test build #70222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70222/testReport)** for PR 16299 at commit

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16299 Ah I see - yes - using `mv` is a simpler option - BTW when do we run `R CMD check` ? Is this for say when users download the tar.gz and run it ? (We can move this discussion to the JIRA) --- If

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16299 I found an issue with snapshot build just now - `R CMD check` doesn't handle the `-SNAPSHOT` part of the file name in SparkR_*tar.gz. Since it's only an issue with SNAPSHOT build I'll open a PR

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16299 can we `mv` instead? https://github.com/apache/spark/pull/16249/files#diff-ba2c046d92a1d2b5b417788bfb5cb5f8R232 --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16281 I'd love to see frequent, conservative patch releases. From my experience, parquet bugs cause significant trouble for downstream consumers. For example, we encountered a data corruption bug writing

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16272 LGTM. Just one minor question. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16287 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16287 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70212/ Test PASSed. ---

[GitHub] spark pull request #16272: [SPARK-18850][SS]Make StreamExecution and progres...

2016-12-15 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16272#discussion_r92726528 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala --- @@ -244,7 +244,7 @@ class StreamingQueryManagerSuite

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16287 **[Test build #70212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70212/testReport)** for PR 16287 at commit

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16299 **[Test build #70220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70220/testReport)** for PR 16299 at commit

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16272 **[Test build #70221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70221/testReport)** for PR 16272 at commit

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16272 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16299: [MINOR] Only rename SparkR tar.gz if names mismatch

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16299 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16299: [MINOR] Only rename SparkR tar.gz if names mismat...

2016-12-15 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/16299 [MINOR] Only rename SparkR tar.gz if names mismatch ## What changes were proposed in this pull request? For release builds the R_PACKAGE_VERSION and VERSION are the same (e.g., 2.1.0).

[GitHub] spark pull request #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryL...

2016-12-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16287 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16294: [WIP][SPARK-18669][SS][DOCS] Update Apache docs for Stru...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70218/testReport)** for PR 16294 at commit

[GitHub] spark issue #16282: [DO_NOT_MERGE]Try to fix kafka

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16282 **[Test build #70219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70219/testReport)** for PR 16282 at commit

[GitHub] spark issue #16287: [SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListener...

2016-12-15 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16287 LGTM. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16272 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70213/ Test FAILed. ---

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16272 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15335 **[Test build #70217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70217/testReport)** for PR 15335 at commit

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution and progress class...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16272 **[Test build #70213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70213/testReport)** for PR 16272 at commit

[GitHub] spark pull request #16298: [SPARK-8425][Scheduler][HOTFIX] fix scala 2.10 co...

2016-12-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16298 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15335 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16298: [SPARK-8425][Scheduler][HOTFIX] fix scala 2.10 compile e...

2016-12-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16298 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16281: [SPARK-13127][SQL] Update Parquet to 1.9.0

2016-12-15 Thread julienledem
Github user julienledem commented on the issue: https://github.com/apache/spark/pull/16281 Like @rdblue said, I don't recall people asking for a 1.8.2 on the parquet dev list. We are happy to help and if there is a 1.8.x patch release branch it's better to maintain it in the

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15335 ok, I'll rerun tests to make sure and take a look. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure refactor...

2016-12-15 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/15335 It's worth something, but not a lot. I think it's worth merging, but if someone thinks it's not, I'm not going to fight it. Read it and merge if you want to. --- If your project is set up

<    1   2   3   4   5   >