[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/17180 The test failure us caused by call method on the map after `destructiveIterator()` has been called. It is illegal by the definition.

[GitHub] spark issue #18723: [SPARK-21517][CORE] Avoid copying memory when transfer c...

2017-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18723 IIUC, this PR avoid data copy for consolidation in `netty.buffer.CompositeByteBuf.consolidateIfNeeded`. I think that it is reasonable. I have two questions to understand this change. 1.

[GitHub] spark pull request #18723: [SPARK-21517][CORE] Avoid copying memory when tra...

2017-07-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18723#discussion_r129211296 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -66,7 +66,7 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18388 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79923/ Test PASSed. ---

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18388 **[Test build #79923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79923/testReport)** for PR 18388 at commit

[GitHub] spark pull request #18540: [SPARK-19451][SQL] rangeBetween method should acc...

2017-07-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18540#discussion_r129209847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala --- @@ -43,58 +42,54 @@ case class

[GitHub] spark pull request #18722: [SPARK-21498][Examples] quick start -> one py dem...

2017-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18722#discussion_r129209256 --- Diff: docs/quick-start.md --- @@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \ Lines with a: 46, Lines with b: 23 {%

[GitHub] spark pull request #18722: [SPARK-21498][Examples] quick start -> one py dem...

2017-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18722#discussion_r129209310 --- Diff: docs/quick-start.md --- @@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \ Lines with a: 46, Lines with b: 23 {%

[GitHub] spark pull request #18722: [SPARK-21498][Examples] quick start -> one py dem...

2017-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18722#discussion_r129209368 --- Diff: docs/quick-start.md --- @@ -421,16 +421,15 @@ $ YOUR_SPARK_HOME/bin/spark-submit \ Lines with a: 46, Lines with b: 23 {%

[GitHub] spark issue #18722: [SPARK-21498][Examples] quick start -> one py demo have ...

2017-07-24 Thread lizhaoch
Github user lizhaoch commented on the issue: https://github.com/apache/spark/pull/18722 OK,I have done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18721: [SPARK-21516][SQL][Test] Overriding afterEach() i...

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18721 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18721: [SPARK-21516][SQL][Test] Overriding afterEach() in Datas...

2017-07-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18721 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16365: [SPARK-18950][SQL] Report conflicting fields when mergin...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16365 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79921/ Test PASSed. ---

[GitHub] spark issue #16365: [SPARK-18950][SQL] Report conflicting fields when mergin...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16365 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16365: [SPARK-18950][SQL] Report conflicting fields when mergin...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16365 **[Test build #79921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79921/testReport)** for PR 16365 at commit

[GitHub] spark issue #18723: [SPARK-21517][CORE] Avoid copying memory when transfer c...

2017-07-24 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/18723 Can anyone help verify this?Thank too much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18388 **[Test build #79923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79923/testReport)** for PR 18388 at commit

[GitHub] spark issue #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit problem i...

2017-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18641 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18649: [SPARK-21395][SQL] Spark SQL hive-thriftserver doesn't r...

2017-07-24 Thread debugger87
Github user debugger87 commented on the issue: https://github.com/apache/spark/pull/18649 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79922/ Test FAILed. ---

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #79922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79922/testReport)** for PR 18664 at commit

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-24 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18696 @srowen want to take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18726: [MINOR][CORE][TEST]Repeat stop SparkContext in ExecutorA...

2017-07-24 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18726 @srowen , Well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18706 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18706 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79917/ Test PASSed. ---

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18706 **[Test build #79917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79917/testReport)** for PR 18706 at commit

[GitHub] spark pull request #18722: [SPARK-21498][Examples] quick start -> one py dem...

2017-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18722#discussion_r129189476 --- Diff: docs/quick-start.md --- @@ -389,7 +389,7 @@ As an example, we'll create a simple Spark application, `SimpleApp.py`: from pyspark.sql

[GitHub] spark issue #18729: [SPARK-21526] [MLlib] Add support to ML LogisticRegressi...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18729 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18729: [SPARK-21526] [MLlib] Add support to ML LogisticRegressi...

2017-07-24 Thread JohnHBrock
Github user JohnHBrock commented on the issue: https://github.com/apache/spark/pull/18729 This contribution is my original work and I license the work to the project under the project’s open source license. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18729: [SPARK-21526] [MLlib] Add support to ML LogisticR...

2017-07-24 Thread JohnHBrock
GitHub user JohnHBrock opened a pull request: https://github.com/apache/spark/pull/18729 [SPARK-21526] [MLlib] Add support to ML LogisticRegression for setting initial model ## What changes were proposed in this pull request? * Change ML LogisticRegression setInitialModel

[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17180 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17180 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79918/ Test FAILed. ---

[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17180 **[Test build #79918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79918/testReport)** for PR 17180 at commit

[GitHub] spark pull request #18717: [SPARK-21510] [SQL] Add isMaterialized() and eage...

2017-07-24 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18717#discussion_r129185097 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -226,6 +216,21 @@ abstract class QueryTest extends PlanTest { }

[GitHub] spark issue #18313: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-07-24 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/18313 Current implementation of `CrossValidator` (with or without this PR) **NEVER** holds all the trained models in the driver memory at the same time. It collects models sequentially and allows GC to

[GitHub] spark issue #18728: [SPARK-21524] [ML] unit test fix: ValidatorParamsSuiteHe...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18728 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18313: [SPARK-21087] [ML] CrossValidator, TrainValidatio...

2017-07-24 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/18313 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18728: [SPARK-21524] [ML] unit test fix: ValidatorParamsSuiteHe...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18728 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79920/ Test PASSed. ---

[GitHub] spark issue #18728: [SPARK-21524] [ML] unit test fix: ValidatorParamsSuiteHe...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18728 **[Test build #79920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79920/testReport)** for PR 18728 at commit

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18664 Thanks @ueshin and @icexelloss. I like `SQLConf.SESSION_LOCAL_TIMEZONE` also, but it ended up being pretty messy trying to fit the `SQLConf` into these static functions - if there is an easier

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #79922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79922/testReport)** for PR 18664 at commit

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16158 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16158 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79919/ Test PASSed. ---

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16158 **[Test build #79919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79919/testReport)** for PR 16158 at commit

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79916/ Test PASSed. ---

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #79916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79916/testReport)** for PR 18727 at commit

[GitHub] spark issue #16365: [SPARK-18950][SQL] Report conflicting fields when mergin...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16365 **[Test build #79921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79921/testReport)** for PR 16365 at commit

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/18519 @vanzin all green, ready for a check. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18519 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79914/ Test PASSed. ---

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18519 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18519 **[Test build #79914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79914/testReport)** for PR 18519 at commit

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79915/ Test PASSed. ---

[GitHub] spark pull request #18313: [SPARK-21087] [ML] CrossValidator, TrainValidatio...

2017-07-24 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/18313#discussion_r129178830 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -113,15 +122,28 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #79915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79915/testReport)** for PR 18727 at commit

[GitHub] spark issue #18071: [SPARK-20855][Docs][DStream] Update the Spark kinesis do...

2017-07-24 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/18071 Thanks @budde @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18728: [SPARK-21524] [ML] unit test fix: ValidatorParamsSuiteHe...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18728 **[Test build #79920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79920/testReport)** for PR 18728 at commit

[GitHub] spark pull request #18098: [SPARK-16944][Mesos] Improve data locality when l...

2017-07-24 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/18098#discussion_r129176526 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -72,6 +72,10 @@ package object config { private[spark] val

[GitHub] spark pull request #18728: [SPARK-21524] [ML] fix temp dir

2017-07-24 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/18728 [SPARK-21524] [ML] fix temp dir ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-21524 ValidatorParamsSuiteHelpers.testFileMove() is

[GitHub] spark pull request #17419: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-07-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/17419#discussion_r129173570 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -0,0 +1,799 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16158 **[Test build #79919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79919/testReport)** for PR 16158 at commit

[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17180 **[Test build #79918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79918/testReport)** for PR 17180 at commit

[GitHub] spark issue #17180: [SPARK-19839][Core]release longArray in BytesToBytesMap

2017-07-24 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/17180 per review comments, release the longArray on destructive iterator creation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18706 +1 pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18706: [SPARK-21494][network] Use correct app id when au...

2017-07-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18706#discussion_r129164949 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/sasl/ShuffleSecretManager.java --- @@ -67,7 +67,7 @@ public void registerApp(String

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18706 **[Test build #79917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79917/testReport)** for PR 18706 at commit

[GitHub] spark pull request #18706: [SPARK-21494][network] Use correct app id when au...

2017-07-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18706#discussion_r129161633 --- Diff: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -243,7 +243,6 @@ public void

[GitHub] spark pull request #18706: [SPARK-21494][network] Use correct app id when au...

2017-07-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18706#discussion_r129159285 --- Diff: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -243,7 +243,6 @@ public void

[GitHub] spark pull request #18706: [SPARK-21494][network] Use correct app id when au...

2017-07-24 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/18706#discussion_r129157052 --- Diff: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java --- @@ -243,7 +243,6 @@ public void

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79913/ Test PASSed. ---

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18696 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18696 **[Test build #79913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79913/testReport)** for PR 18696 at commit

[GitHub] spark issue #18706: [SPARK-21494][network] Use correct app id when authentic...

2017-07-24 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18706 @zsxwing @tgravescs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #79916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79916/testReport)** for PR 18727 at commit

[GitHub] spark issue #18727: [SPARK-21522][core] Fix flakiness in LauncherServerSuite...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18727 **[Test build #79915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79915/testReport)** for PR 18727 at commit

[GitHub] spark pull request #18727: [SPARK-21522][core] Fix flakiness in LauncherServ...

2017-07-24 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/18727 [SPARK-21522][core] Fix flakiness in LauncherServerSuite. Handle the case where the server closes the socket before the full message has been written by the client. You can merge this pull

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18519 **[Test build #79914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79914/testReport)** for PR 18519 at commit

[GitHub] spark issue #18721: [SPARK-21516][SQL][Test] Overriding afterEach() in Datas...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18721 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18721: [SPARK-21516][SQL][Test] Overriding afterEach() in Datas...

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18721 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79912/ Test PASSed. ---

[GitHub] spark issue #18721: [SPARK-21516][SQL][Test] Overriding afterEach() in Datas...

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18721 **[Test build #79912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79912/testReport)** for PR 18721 at commit

[GitHub] spark issue #18708: [SPARK-21339] [CORE] spark-shell --packages option does ...

2017-07-24 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18708 > If you are sure that it breaks the other user cases, For one, `--jars file:/tmp/jars/*` will break with your change, since globs won't be resolved for `file` URLs. --- If your project

[GitHub] spark issue #18708: [SPARK-21339] [CORE] spark-shell --packages option does ...

2017-07-24 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/18708 Thanks @vanzin for checking this. ``` Then shouldn't the fix be in the code that transforms the URI list into an argument for -classpath? I'm pretty sure the code you're

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79911/ Test PASSed. ---

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18714 **[Test build #79911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79911/testReport)** for PR 18714 at commit

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79910/ Test PASSed. ---

[GitHub] spark issue #18714: [SPARK-20236][SQL] runtime partition overwrite

2017-07-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18714 **[Test build #79910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79910/testReport)** for PR 18714 at commit

[GitHub] spark pull request #18313: [SPARK-21087] [ML] CrossValidator, TrainValidatio...

2017-07-24 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18313#discussion_r129122995 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -113,15 +122,28 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark issue #12533: [SPARK-14760] [ML] Feature transformers should always in...

2017-07-24 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/12533 Thanks for the attention @MLnick. I'm closing most of my stale PRs. For this one, I found all of the transformers in the PR already have ` transformSchema(dataset.schema, logging = true)`,

[GitHub] spark issue #18692: [SPARK-21417][SQL] Detect joind conditions via filter ex...

2017-07-24 Thread aokolnychyi
Github user aokolnychyi commented on the issue: https://github.com/apache/spark/pull/18692 @gatorsmile thanks for the input. Let me check that I understood everything correctly. So, I keep it as a separate rule that is applied only if constraint propagation enabled. Inside the rule,

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-07-24 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/18664 Excited to see this being worked on. > SQLConf.SESSION_LOCAL_TIMEZONE I like this the best. This presents timestamp in local time which is compatible with the existing `toPandas()`

[GitHub] spark issue #18519: [SPARK-16742] Mesos Kerberos Support

2017-07-24 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/18519 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18722: [SPARK-21498][Examples] quick start -> one py dem...

2017-07-24 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18722#discussion_r129117923 --- Diff: docs/quick-start.md --- @@ -389,7 +389,7 @@ As an example, we'll create a simple Spark application, `SimpleApp.py`: from pyspark.sql

[GitHub] spark issue #12533: [SPARK-14760] [ML] Feature transformers should always in...

2017-07-24 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/12533 Hey @hhbyyh sorry for the lack of review. However, I think this is quite important. There are many instances where the schema validation is not necessarily checked, and many instances where it is

[GitHub] spark issue #12037: [SPARK-14239] [ML] Add load for LDAModel that supports b...

2017-07-24 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/12037 Closing stale PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #12037: [SPARK-14239] [ML] Add load for LDAModel that sup...

2017-07-24 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/12037 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

  1   2   3   >