[GitHub] spark issue #16903: [MLlib][TYPO] Replace LeastSquaresAggregator with Logist...

2017-02-18 Thread moutai
Github user moutai commented on the issue: https://github.com/apache/spark/pull/16903 Thanks, Yes I think the rest of the messages look fine on this file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909916 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -482,6 +482,15 @@ case class

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73123/testReport)** for PR 16944 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909598 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101909627 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +60,26 @@ object JacksonUtils {

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73121/ Test PASSed. ---

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73121/testReport)** for PR 16985 at commit

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101910249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -101,6 +101,11 @@ class CSVFileFormat

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73116/testReport)** for PR 16395 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73115/ Test FAILed. ---

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73118 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73118/testReport)** for PR 16395 at commit

[GitHub] spark issue #16985: [WIP] [SPARK-19122][SQL] Unnecessary shuffle+sort added ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73119/testReport)** for PR 16985 at commit

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16979 **[Test build #73117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73117/testReport)** for PR 16979 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73119/ Test PASSed. ---

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16987: [WIP][SPARK-][SS] FileSource read from FileSink

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73124/testReport)** for PR 16987 at commit

[GitHub] spark pull request #16988: [SPARK-19658] [SQL] Set NumPartitions of Repartit...

2017-02-18 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16988 [SPARK-19658] [SQL] Set NumPartitions of RepartitionByExpression In Analyzer ### What changes were proposed in this pull request? Currently, if `NumPartitions` is not set in

[GitHub] spark pull request #16933: [SPARK-19601] [SQL] Fix CollapseRepartition rule ...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16933#discussion_r101911156 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/CollapseRepartitionSuite.scala --- @@ -43,15 +55,44 @@ class

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added...

2017-02-18 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16985#discussion_r101905768 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -33,8 +33,8 @@ import

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73117/ Test PASSed. ---

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73118/testReport)** for PR 16395 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73118/ Test PASSed. ---

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16987: [WIP][SPARK-][SS] FileSource read from FileSink

2017-02-18 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/16987 [WIP][SPARK-][SS] FileSource read from FileSink ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested?

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73121/testReport)** for PR 16985 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73120/ Test FAILed. ---

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101908105 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16979 **[Test build #73117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73117/testReport)** for PR 16979 at commit

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16979 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/16985 @hvanhovell : This is as per our discussion in the jira : https://issues.apache.org/jira/browse/SPARK-19122 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73122/testReport)** for PR 16978 at commit

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101908155 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16938: [SPARK-19583][SQL]CTAS for data source table with a crea...

2017-02-18 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16938 @gatorsmile I have test all the cases above updated. The result shows that spark for datasource table with HiveExternalCatalog and InMemoryCatatlog have the same actions. spark for hive

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73120/testReport)** for PR 16978 at commit

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #73119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73119/testReport)** for PR 16985 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73122/testReport)** for PR 16978 at commit

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73122/ Test PASSed. ---

[GitHub] spark issue #16978: [SPARK-19652][UI] Do auth checks for REST API access.

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16978 **[Test build #73115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73115/testReport)** for PR 16978 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73116/testReport)** for PR 16395 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73116/ Test FAILed. ---

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73123/ Test FAILed. ---

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16981 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73099/ Test FAILed. ---

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73101/ Test FAILed. ---

[GitHub] spark issue #16979: [SPARK-19617][SS]Fix the race condition when starting an...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16979 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16898: [SPARK-19563][SQL] avoid unnecessary sort in File...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16898#discussion_r101887954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -119,23 +130,45 @@ object FileFormatWriter

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101888190 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r101888278 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSchemaInferenceSuite.scala --- @@ -0,0 +1,192 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16981 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73102/ Test PASSed. ---

[GitHub] spark pull request #16964: [SPARK-19534][TESTS] Convert Java tests to use la...

2017-02-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16964#discussion_r101891096 --- Diff: common/network-common/src/test/java/org/apache/spark/network/util/TransportFrameDecoderSuite.java --- @@ -153,7 +148,7 @@ public void

[GitHub] spark issue #16871: [SPARK-19550][BUILD][CORE][WIP] Remove Java 7 support

2017-02-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16871 @ericl @vanzin see https://github.com/apache/spark/pull/16983 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16983 **[Test build #73104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73104/testReport)** for PR 16983 at commit

[GitHub] spark pull request #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1...

2017-02-18 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/16983 [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for scalac 2.10, still ## What changes were proposed in this pull request? Go back to selecting source/target 1.7 for Scala 2.10 builds,

[GitHub] spark issue #16976: [SPARK-19610][SQL] Support parsing multiline CSV files

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16976 **[Test build #73105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73105/testReport)** for PR 16976 at commit

[GitHub] spark pull request #16898: [SPARK-19563][SQL] avoid unnecessary sort in File...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16898#discussion_r101888508 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -119,23 +130,45 @@ object FileFormatWriter

[GitHub] spark pull request #16968: [SPARK-19337] [ML] [Doc] Documentation and exampl...

2017-02-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16968#discussion_r101890081 --- Diff: docs/ml-classification-regression.md --- @@ -363,6 +363,44 @@ Refer to the [R API docs](api/R/spark.mlp.html) for more details.

[GitHub] spark issue #16903: [MLlib][TYPO] Replace LeastSquaresAggregator with Logist...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16903 **[Test build #3579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3579/testReport)** for PR 16903 at commit

[GitHub] spark pull request #16898: [SPARK-19563][SQL] avoid unnecessary sort in File...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16898#discussion_r101888059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -287,31 +320,16 @@ object FileFormatWriter

[GitHub] spark pull request #16898: [SPARK-19563][SQL] avoid unnecessary sort in File...

2017-02-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16898#discussion_r101888000 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -108,9 +107,21 @@ object FileFormatWriter

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73102/testReport)** for PR 16981 at commit

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73102/testReport)** for PR 16981 at commit

[GitHub] spark issue #16903: [MLlib][TYPO] Replace LeastSquaresAggregator with Logist...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16903 **[Test build #3579 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3579/testReport)** for PR 16903 at commit

[GitHub] spark issue #16964: [SPARK-19534][TESTS] Convert Java tests to use lambdas, ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16964 **[Test build #73103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73103/testReport)** for PR 16964 at commit

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101891903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -55,4 +62,32 @@ object JacksonUtils {

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101891758 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -18,7 +18,14 @@ package

[GitHub] spark pull request #16981: [SPARK-19637][SQL] Add from_json/to_json in Funct...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16981#discussion_r101891914 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -174,4 +174,44 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread lins05
Github user lins05 commented on the issue: https://github.com/apache/spark/pull/16984 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16984: [SPARK-19550] Follow-up: fixed a typo that fails ...

2017-02-18 Thread lins05
GitHub user lins05 opened a pull request: https://github.com/apache/spark/pull/16984 [SPARK-19550] Follow-up: fixed a typo that fails the dev/make-distribution.sh script. ## What changes were proposed in this pull request? Fixed a typo in `dev/make-distribution.sh` script

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16983 **[Test build #73104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73104/testReport)** for PR 16983 at commit

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16983 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73104/ Test PASSed. ---

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16983 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16983: [SPARK-19550][BUILD][WIP] Addendum: select Java 1.7 for ...

2017-02-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16983 This change does avoid the error at https://github.com/apache/spark/pull/16871#issuecomment-280791684 when I run the same commands that the Jenkins job does ```

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16981 **[Test build #73107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73107/testReport)** for PR 16981 at commit

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894380 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -101,6 +101,12 @@ class CSVFileFormat

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894427 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -222,12 +250,6 @@ private[csv] class

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894565 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala --- @@ -26,7 +26,7 @@ import

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-18 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16620 Thanks all for the work on this! I've merged this into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894462 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -45,24 +46,35 @@ private[csv] class

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894442 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -173,25 +185,41 @@ private[csv] class

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16620 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16928: [SPARK-18699][SQL] Put malformed tokens into a ne...

2017-02-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16928#discussion_r101894489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -45,24 +46,35 @@ private[csv] class

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16984 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16984 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73106/ Test PASSed. ---

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16984 **[Test build #73106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73106/testReport)** for PR 16984 at commit

[GitHub] spark issue #16982: [SPARK-19654][SPARKR] Structured Streaming API for R

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16982 **[Test build #73108 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73108/testReport)** for PR 16982 at commit

[GitHub] spark issue #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as values in...

2017-02-18 Thread paberline
Github user paberline commented on the issue: https://github.com/apache/spark/pull/8384 Closing as requested due to this feature being too narrow a use case for core Spark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #8384: [SPARK-8510] [CORE] [PYSPARK] NumPy matrices as va...

2017-02-18 Thread paberline
Github user paberline closed the pull request at: https://github.com/apache/spark/pull/8384 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16981: [SPARK-19637][SQL] Add from_json/to_json in FunctionRegi...

2017-02-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16981 @HyukjinKwon Thanks! I'll check. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread lins05
Github user lins05 commented on the issue: https://github.com/apache/spark/pull/16984 That's how the shell's "default value" works. FYI http://www.tldp.org/LDP/abs/html/parameter-substitution.html --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16984: [SPARK-19550] Follow-up: fixed a typo that fails the dev...

2017-02-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16984 **[Test build #73106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73106/testReport)** for PR 16984 at commit

  1   2   >