[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17929 **[Test build #76727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76727/testReport)** for PR 17929 at commit

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/17929 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recommendForA...

2017-05-09 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/17742 I don't think we should use BLAS 3 here, because no matter use output or not here, you need a big buffer to save the BLAS result. That still cause GC problem. I also want to test: build a

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17929 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17929 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76718/ Test FAILed. ---

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17929 **[Test build #76718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76718/testReport)** for PR 17929 at commit

[GitHub] spark pull request #17862: [SPARK-20602] [ML]Adding LBFGS as optimizer for L...

2017-05-09 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17862#discussion_r115657829 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -223,6 +229,25 @@ class LinearSVCSuite extends SparkFunSuite

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17916 Thanks everyone. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17916 thanks, merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17916: [SPARK-20590][SQL] Use Spark internal datasource ...

2017-05-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17916 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76721/ Test PASSed. ---

[GitHub] spark issue #17869: [SPARK-20609][CORE]Run the SortShuffleSuite unit tests h...

2017-05-09 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/17869 To run again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #76721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76721/testReport)** for PR 16989 at commit

[GitHub] spark issue #17909: [SPARK-20661][WIP] try to dump table names

2017-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17909 ah, you are right. I'm going to fix the python code --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17928 LGTM, cc @gatorsmile to take another look --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17931: [SPARK-12837][CORE][FOLLOWUP] getting name should not fa...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17931 **[Test build #76726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76726/testReport)** for PR 17931 at commit

[GitHub] spark pull request #17862: [SPARK-20602] [ML]Adding LBFGS as optimizer for L...

2017-05-09 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17862#discussion_r115656752 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LinearSVCSuite.scala --- @@ -154,22 +159,23 @@ class LinearSVCSuite extends SparkFunSuite

[GitHub] spark issue #17931: [SPARK-12837][CORE][FOLLOWUP] getting name should not fa...

2017-05-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17931 What's the issue with SQL metrics? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17931: [SPARK-12837][CORE][FOLLOWUP] getting name should not fa...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17931 cc @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #17931: [SPARK-12837][CORE][FOLLOWUP] getting name should...

2017-05-09 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17931 [SPARK-12837][CORE][FOLLOWUP] getting name should not fail if accumulator is garbage collected ## What changes were proposed in this pull request? After

[GitHub] spark issue #17930: [SPARK-20688][SQL] correctly check analysis for scalar s...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17930 **[Test build #76725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76725/testReport)** for PR 17930 at commit

[GitHub] spark issue #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recommendForA...

2017-05-09 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/17742 Thanks @mengxr , glad to meet you here. I am Meng Peng. I have tested different blockSize, see https://issues.apache.org/jira/browse/SPARK-20443 I will test the other methods you mentioned.

[GitHub] spark pull request #17930: [SPARK-20688][SQL] correctly check analysis for s...

2017-05-09 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17930 [SPARK-20688][SQL] correctly check analysis for scalar sub-queries ## What changes were proposed in this pull request? In `CheckAnalysis`, we should call `checkAnalysis` for

[GitHub] spark issue #17930: [SPARK-20688][SQL] correctly check analysis for scalar s...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17930 cc @rxin @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recommendForA...

2017-05-09 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/17742 I think the problem is not BLAS-3 ops, nor the 256MB total memory. The `val output = new Array[(Int, (Int, Double))](m * n)` is not specialized. Each element holds two references. If `m=4096` and

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76719/ Test FAILed. ---

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17770 **[Test build #76719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76719/testReport)** for PR 17770 at commit

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17929 **[Test build #76724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76724/testReport)** for PR 17929 at commit

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17924 Yep. Since this is an approach adding new dependency on Apache ORC, the non-vectorized PR also will need more supports(or approval) from the committers. I'll wait for more opinions at the

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17924 @dongjoon-hyun It is good for me. We can reduce the size of this PR too and mitigate review job. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17924 @cloud-fan and @viirya . Shall we remove the vectorized part from this PR? - The non-vectorized ORCFileFormat is mandatory and also the performance is better than the current one.

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115650693 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,415 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #17711: [SPARK-19951][SQL] Add string concatenate operator || to...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17711 **[Test build #76723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76723/testReport)** for PR 17711 at commit

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115650375 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,415 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #17911: [SPARK-20668][SQL] Modify ScalaUDF to handle nullability...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17911 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17911: [SPARK-20668][SQL] Modify ScalaUDF to handle nullability...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17911 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76717/ Test PASSed. ---

[GitHub] spark issue #17911: [SPARK-20668][SQL] Modify ScalaUDF to handle nullability...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17911 **[Test build #76717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76717/testReport)** for PR 17911 at commit

[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17887 **[Test build #76722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76722/testReport)** for PR 17887 at commit

[GitHub] spark pull request #17887: [SPARK-20399][SQL] Add a config to fallback strin...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17887#discussion_r115649694 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala --- @@ -413,38 +428,102 @@ class

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-05-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r115649586 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala --- @@ -290,4 +290,15 @@ class SparkSqlParserSuite extends

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17928 @cloud-fan ok, could you check again? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-05-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r115649532 --- Diff: sql/core/src/test/resources/sql-tests/inputs/operator.sql --- @@ -32,3 +32,11 @@ select 1 - 2; select 2 * 5; select 5 % 3; select

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13775 Hmm. It seems `Merge remote-tracking branch` makes rebasing confused. Let me think how to compare this. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115649175 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala --- @@ -0,0 +1,415 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #17911: [SPARK-20668][SQL] Modify ScalaUDF to handle nullability...

2017-05-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17911 LGTM pending test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17909: [SPARK-20661][WIP] try to dump table names

2017-05-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17909 Two tables `bucketed_table ` and `sorted_bucketed_table ` are from the same file `readwriter.py`. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76716/ Test PASSed. ---

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17928 **[Test build #76716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76716/testReport)** for PR 17928 at commit

[GitHub] spark issue #17909: [SPARK-20661][WIP] try to dump table names

2017-05-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17909 [Second one](https://github.com/apache/spark/blob/096df6d933c5326e5782aa8c5de842a0800eb369/sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedWriteSuite.scala#L188) should not create a

[GitHub] spark issue #17869: [SPARK-20609][CORE]Run the SortShuffleSuite unit tests h...

2017-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17869 (it looks the test failure is unrelated to me) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13775 Thank you! First, I'll try to rebase and run with my `OrcReadBenchmark` (which is similar with ParquetReadBenchmark). --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 @dongjoon-hyun No problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13775 Otherwise, may I implement this way in my PR as a Viirya's approach? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 @dongjoon-hyun Sure. Do you need any help? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13775 @viirya . If possible, I'd like to benchmark this PR in my laptop. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17924: [SPARK-20682][SQL] Support a new faster ORC data source ...

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17924 From the current benchmark, seems the performance has not obvious improvement, compared with the vectorized Hive ORC reader #13775. Maybe with more efficient batch approach as @cloud-fan

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76715/ Test PASSed. ---

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17916 **[Test build #76715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76715/testReport)** for PR 17916 at commit

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17929 LGTM except a trivial comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorre...

2017-05-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17929#discussion_r115647510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelation.scala --- @@ -68,8 +63,13 @@ object

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76713/ Test PASSed. ---

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17929 **[Test build #76713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76713/testReport)** for PR 17929 at commit

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115646941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,407 @@ +/* + *

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115646778 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,407 @@ +/* + *

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76714/ Test PASSed. ---

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17916 **[Test build #76714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76714/testReport)** for PR 17916 at commit

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115646321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,407 @@ +/* + *

[GitHub] spark pull request #17924: [SPARK-20682][SQL] Support a new faster ORC data ...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17924#discussion_r115646099 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.scala --- @@ -0,0 +1,407 @@ +/* + *

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #76721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76721/testReport)** for PR 16989 at commit

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #76720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76720/testReport)** for PR 16989 at commit

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76720/ Test FAILed. ---

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17920: [SPARK-17685][SQL] Make SortMergeJoinExec's curre...

2017-05-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17920 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17926: [MINOR][SQL][PYSPARK] Allow user to specify numSlices in...

2017-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17926 It seems adding a functionality and not a trivial fix. I think we need a JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17920: [SPARK-17685][SQL] Make SortMergeJoinExec's currentVars ...

2017-05-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17920 LGTM - merging to master/2.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #76720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76720/testReport)** for PR 16989 at commit

[GitHub] spark issue #17770: [SPARK-20392][SQL] Set barrier to prevent re-entering a ...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17770 **[Test build #76719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76719/testReport)** for PR 17770 at commit

[GitHub] spark pull request #17770: [SPARK-20392][SQL] Set barrier to prevent re-ente...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r115644214 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -47,36 +47,11 @@ abstract class LogicalPlan

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r115643056 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala --- @@ -290,4 +290,15 @@ class SparkSqlParserSuite extends

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r115642886 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -547,6 +547,10 @@ valueExpression |

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17928 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76712/ Test FAILed. ---

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-05-09 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r115642487 --- Diff: sql/core/src/test/resources/sql-tests/inputs/operator.sql --- @@ -32,3 +32,11 @@ select 1 - 2; select 2 * 5; select 5 % 3; select

[GitHub] spark issue #17928: [SPARK-20311][SQL] Support aliases for table value funct...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17928 **[Test build #76712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76712/testReport)** for PR 17928 at commit

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17916 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17923: [SPARK-20591][WEB UI] Succeeded tasks num not equal in a...

2017-05-09 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/17923 I see, but really it's not easy to find the speculative tasks, especially when the succeeded task num are inconsistent on the all jobs page and the job detail page. Shall we let them seems

[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size

2017-05-09 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/17739 Hi @MLnick , since #17742 is merged into branch-2.2, and this PR is related with PR17742. Should we close this PR and open a new PR based on branch-2.2. --- If your project is set up for it, you can

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17916 Sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-09 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115640830 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-09 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16989 Yes, I will refine :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17916 @HyukjinKwon Shall we also update the PR description? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #17633: [SPARK-20331][SQL] Enhanced Hive partition prunin...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17633#discussion_r115639892 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -589,18 +590,34 @@ private[client] class Shim_v0_13 extends

[GitHub] spark pull request #17633: [SPARK-20331][SQL] Enhanced Hive partition prunin...

2017-05-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17633#discussion_r115639777 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -589,18 +590,39 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #17929: [SPARK-20686][SQL] PropagateEmptyRelation incorrectly ha...

2017-05-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17929 **[Test build #76718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76718/testReport)** for PR 17929 at commit

  1   2   3   4   5   6   7   8   >