[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88612839 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -33,6 +33,79 @@ import

[GitHub] spark pull request #15889: [SPARK-18445][BUILD][DOCS] Fix the markdown for `...

2016-11-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15889#discussion_r88611836 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala --- @@ -234,6 +234,9 @@ class JavaPairRDD[K, V](val rdd: RDD[(K, V)])

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r88611024 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -574,7 +574,8 @@ object ScalaReflection extends

[GitHub] spark issue #15913: [SPARK-18481][ML] ML 2.1 QA: Remove deprecated methods f...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15913 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68829/ Test FAILed. ---

[GitHub] spark issue #15913: [SPARK-18481][ML] ML 2.1 QA: Remove deprecated methods f...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15913 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15913: [SPARK-18481][ML] ML 2.1 QA: Remove deprecated methods f...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15913 **[Test build #68829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68829/consoleFull)** for PR 15913 at commit

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15620 **[Test build #68832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68832/consoleFull)** for PR 15620 at commit

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15620 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88609776 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +317,188 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15852 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68827/ Test PASSed. ---

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15852 **[Test build #68827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68827/consoleFull)** for PR 15852 at commit

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-17 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88609206 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +317,188 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88608752 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +317,188 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #15927: [SPARK-18500][SQL] Make GenericStrategy be able to prune...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15927 **[Test build #68831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68831/consoleFull)** for PR 15927 at commit

[GitHub] spark issue #15927: [SPARK-18500][SQL] Make GenericStrategy be able to prune...

2016-11-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15927 @marmbrus Could you review this pr please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15927: [SPARK-18500][SQL] Make GenericStrategy be able t...

2016-11-17 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/15927 [SPARK-18500][SQL] Make GenericStrategy be able to prune plans by itself after placeholders are replaced. ## What changes were proposed in this pull request? This pr adds a functionality

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88607110 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +317,188 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #15880: [SPARK-17913][SQL] compare long and string type column m...

2016-11-17 Thread dasbipulkumar
Github user dasbipulkumar commented on the issue: https://github.com/apache/spark/pull/15880 yes. As a Dev we can handle these things, but as Spark Sql is moving towards more abstracted APIs, it can be painful for general users. I hope that handing these issues can be planned in

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-17 Thread kapilsingh5050
Github user kapilsingh5050 commented on the issue: https://github.com/apache/spark/pull/15620 I ran the tests on my machine and they passed. @cloud-fan Can you please trigger a re-run of test build? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #15907: [SPARK-18458][CORE] Fix signed integer overflow p...

2016-11-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15907#discussion_r88604890 --- Diff: core/src/test/scala/org/apache/spark/util/collection/unsafe/sort/RadixSortSuite.scala --- @@ -73,22 +73,22 @@ class RadixSortSuite extends

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #68830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68830/consoleFull)** for PR 15780 at commit

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15921 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68826/ Test FAILed. ---

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15921 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15921 **[Test build #68826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68826/consoleFull)** for PR 15921 at commit

[GitHub] spark issue #15913: [SPARK-18481][ML] ML 2.1 QA: Remove deprecated methods f...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15913 **[Test build #68829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68829/consoleFull)** for PR 15913 at commit

[GitHub] spark issue #15913: [SPARK-18481][ML] ML 2.1 QA: Remove deprecated methods f...

2016-11-17 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15913 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15925 yes, I think it is better they are consistent. And exception should be throwed before connect to the sql server. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15889: [SPARK-18445][BUILD][DOCS] Fix the markdown for `Note:`/...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15889 **[Test build #68828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68828/consoleFull)** for PR 15889 at commit

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-11-17 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r88602061 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -331,7 +331,12 @@ private[spark] class MemoryStore( var

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68824/ Test PASSed. ---

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15916 **[Test build #68824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68824/consoleFull)** for PR 15916 at commit

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68821/ Test PASSed. ---

[GitHub] spark pull request #15898: [SPARK-18457][SQL] ORC and other columnar formats...

2016-11-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15898#discussion_r88601375 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala --- @@ -577,4 +579,28 @@ class OrcQuerySuite extends QueryTest with

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15907 **[Test build #68821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68821/consoleFull)** for PR 15907 at commit

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15921 @marmbrus Why don't we want to throw exceptions? Wouldn't it help users catch errors early. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68820/ Test PASSed. ---

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15907 **[Test build #68820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68820/consoleFull)** for PR 15907 at commit

[GitHub] spark issue #15899: [SPARK-18466] added withFilter method to RDD

2016-11-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15899 I don't get it. The only thing you can do here is just a simple syntactic sugar, and the sugar doesn't even work in general. Isn't it more surprising to fail in some cases? --- If your project is

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15925 When we use Spark SQL interface, we issue an error. ``` spark-sql> select * from t1 where c1 in (); 16/11/17 21:08:08 INFO SparkSqlParser: Parsing command: select * from t1 where c1

[GitHub] spark issue #15926: [SPARK-16803] [SQL] SaveAsTable does not work when targe...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15926 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68822/ Test PASSed. ---

[GitHub] spark issue #15926: [SPARK-16803] [SQL] SaveAsTable does not work when targe...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15926 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15926: [SPARK-16803] [SQL] SaveAsTable does not work when targe...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15926 **[Test build #68822 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68822/consoleFull)** for PR 15926 at commit

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15852 **[Test build #68827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68827/consoleFull)** for PR 15852 at commit

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 Yeah, I see. As I said in previous comment, the memory is released at the end anyway. I would guess the default setting as true is to find potential memory leak during development. So turn

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68819/ Test PASSed. ---

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15907 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15907: [SPARK-18458][CORE] Fix signed integer overflow problem ...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15907 **[Test build #68819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68819/consoleFull)** for PR 15907 at commit

[GitHub] spark pull request #15889: [SPARK-18445][BUILD][DOCS] Fix the markdown for `...

2016-11-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15889#discussion_r88599858 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala --- @@ -396,7 +396,7 @@ class

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15916 I'd set it maybe to false. You are just adding a completion listener, which is the same as taskMemoryManager.cleanUpAllAllocatedMemory anyway ... --- If your project is set up for it, you can reply

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15925 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68817/ Test PASSed. ---

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15925 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15925 **[Test build #68817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68817/consoleFull)** for PR 15925 at commit

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68818/ Test PASSed. ---

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15916 **[Test build #68818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68818/consoleFull)** for PR 15916 at commit

[GitHub] spark pull request #15889: [SPARK-18445][BUILD][DOCS] Fix the markdown for `...

2016-11-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15889#discussion_r88598964 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -1014,7 +1015,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88598628 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -245,51 +311,35 @@ case class NewInstance(

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68825/ Test PASSed. ---

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68825/consoleFull)** for PR 15874 at commit

[GitHub] spark issue #15924: [Spark-18498] [SQL] Revise HDFSMetadataLog API for bette...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15924 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15924: [Spark-18498] [SQL] Revise HDFSMetadataLog API for bette...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68815/ Test PASSed. ---

[GitHub] spark issue #15924: [Spark-18498] [SQL] Revise HDFSMetadataLog API for bette...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15924 **[Test build #68815 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68815/consoleFull)** for PR 15924 at commit

[GitHub] spark pull request #15889: [SPARK-18445][BUILD][DOCS] Fix the markdown for `...

2016-11-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15889#discussion_r88597141 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -1014,7 +1015,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15921 **[Test build #68826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68826/consoleFull)** for PR 15921 at commit

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68823/ Test PASSed. ---

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68823/consoleFull)** for PR 15874 at commit

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88596091 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -245,51 +311,35 @@ case class NewInstance(

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15717 Will review this PR tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15852 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68816/ Test FAILed. ---

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15852 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15852: Spark-18187 [SQL] CompactibleFileStreamLog should not us...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15852 **[Test build #68816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68816/consoleFull)** for PR 15852 at commit

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-17 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15358 cc @rxin @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15868: [SPARK-18413][SQL] Add `maxConnections` JDBCOption

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15868 > We cannot call repartition in SQL language environment This is a good point, we need to provide a workaround for sql users, and this PR did. --- If your project is set up for it, you

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r88594560 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -603,7 +603,14 @@ case class

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r88593822 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -603,7 +603,14 @@ case class

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15901 LGTM except some minor comments, thanks for working on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88593388 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -164,23 +233,20 @@ case class Invoke(

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 Of course it is not actually a real memory leak because the memory is released at the end by calling `taskMemoryManager.cleanUpAllAllocatedMemory` in `Executor`. But with

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68825/consoleFull)** for PR 15874 at commit

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88593070 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -164,23 +233,20 @@ case class Invoke(

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15916 **[Test build #68824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68824/consoleFull)** for PR 15916 at commit

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15921 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15921 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68814/ Test FAILed. ---

[GitHub] spark issue #15921: [SPARK-18493] Add missing python APIs: withWatermark and...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15921 **[Test build #68814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68814/consoleFull)** for PR 15921 at commit

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88592721 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -164,23 +233,20 @@ case class Invoke(

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 @rxin An exception will be thrown, not just a warning message. This exception is thrown at `Executor` after it calls `taskMemoryManager.cleanUpAllAllocatedMemory` and finds there are memory not

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r88592405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -603,7 +603,14 @@ case class

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68823/consoleFull)** for PR 15874 at commit

[GitHub] spark issue #15925: [SPARK-18436][SQL]isin with a empty list throw exception

2016-11-17 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15925 not generate isin means that `select * from test where key in()` is equal to `select * from test` ? I think this is not we expected behavior ,we should not cover this situation by

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15916 What do you mean by "report"? A warning message was logged? If a warning message was logged, it is generated by the callback itself which just releases the memory. --- If your project is set up for

[GitHub] spark pull request #15922: [SPARK-18462] Fix ClassCastException in SparkList...

2016-11-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15922 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88591775 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -33,6 +33,79 @@ import

[GitHub] spark issue #15922: [SPARK-18462] Fix ClassCastException in SparkListenerDri...

2016-11-17 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15922 Merging in master/branch-2.1/branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-17 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 @rxin Yeah, the added test case will report memory leak failure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

  1   2   3   4   5   6   >