[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15695 **[Test build #3380 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3380/consoleFull)** for PR 15695 at commit

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67832/consoleFull)** for PR 15696 at commit

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67833/consoleFull)** for PR 15675 at commit

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67834/consoleFull)** for PR 15673 at commit

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-31 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15398 @hvanhovell, I just wanted to add that option 2 from my last comment won't work: if we replace `\\` with `` in the unescape function (as is done for `\%` and `\_`), it will be impossible for a

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15701 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85847969 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( }

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15695 **[Test build #3380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3380/consoleFull)** for PR 15695 at commit

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67840/ Test FAILed. ---

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67844/consoleFull)** for PR 15703 at commit

[GitHub] spark pull request #15701: [SPARK-18167] [SQL] Also log all partitions when ...

2016-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15701 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15302: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15302 @dongjoon-hyun I'll take a look tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67835/consoleFull)** for PR 15696 at commit

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r85830450 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -41,6 +41,8 @@ class

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #67825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67825/consoleFull)** for PR 11105 at commit

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67825/ Test PASSed. ---

[GitHub] spark pull request #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos cl...

2016-10-31 Thread susanxhuynh
GitHub user susanxhuynh opened a pull request: https://github.com/apache/spark/pull/15700 [SPARK-17964][SparkR] Enable SparkR with Mesos client mode and cluster mode ## What changes were proposed in this pull request? Enabled SparkR with Mesos client mode and cluster mode.

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67824/ Test PASSed. ---

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15698 **[Test build #67828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67828/consoleFull)** for PR 15698 at commit

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-31 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15669 If I understand the issue here, the problem is not `--files` but setting `spark.files` through `--conf` (or the config file). The former is translated into `spark.yarn.dist.files` by `SparkSubmit`

[GitHub] spark pull request #15702: [SPARK-18124] Observed-delay based Even Time Wate...

2016-10-31 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15702#discussion_r85845880 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/EventTimeWatermarkExec.scala --- @@ -0,0 +1,93 @@ +/* + * Licensed to

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-10-31 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/15704 [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should support comparators ## What changes were proposed in this pull request? This PR aims to support `comparators`, e.g. '<', '<=',

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15673 It looks like all the unit tests passed, however one of the forked test java processes exited with nonzero status for some unknown reason. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSui...

2016-10-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15699 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67838/consoleFull)** for PR 15696 at commit

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67838/ Test FAILed. ---

[GitHub] spark issue #15669: [SPARK-18160][CORE][YARN] SparkContext.addFile doesn't w...

2016-10-31 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/15669 that's correct, it is due to `spark.files`, jira has been updated. Will update the PR soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67832/ Test FAILed. ---

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67832/consoleFull)** for PR 15696 at commit

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85848357 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( }

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67836/ Test PASSed. ---

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15701 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15677 Alright, then will try to get rid of the arguments part. Thank you all very much sincerely and I apologise the noise I caused. @gatorsmile I will keep in mind you comments too. --- If your

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVER...

2016-10-31 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15705 [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables ## What changes were proposed in this pull request? There are a couple issues with the

[GitHub] spark issue #15705: [SPARK-18183] [SPARK-18184] Fix INSERT [INTO|OVERWRITE] ...

2016-10-31 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15705 cc @cloud-fan @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15701 **[Test build #67836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67836/consoleFull)** for PR 15701 at commit

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67829/consoleFull)** for PR 15626 at commit

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67835/consoleFull)** for PR 15696 at commit

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67838/consoleFull)** for PR 15696 at commit

[GitHub] spark issue #15702: [SPARK-18124] Observed-delay based Even Time Watermarks

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15702 **[Test build #67839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67839/consoleFull)** for PR 15702 at commit

[GitHub] spark pull request #15702: [SPARK-18124] Observed-delay based Even Time Wate...

2016-10-31 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/15702 [SPARK-18124] Observed-delay based Even Time Watermarks This PR adds a new method `withWatermark` to the `Dataset` API, which can be used specify an _event time watermark_. An event time

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15699 **[Test build #67830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67830/consoleFull)** for PR 15699 at commit

[GitHub] spark pull request #15694: [SPARK-18179][SQL] Throws analysis exception with...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15694#discussion_r85848749 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/MiscFunctionsSuite.scala --- @@ -31,6 +34,47 @@ class MiscFunctionsSuite extends QueryTest with

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67845/consoleFull)** for PR 15626 at commit

[GitHub] spark pull request #15675: [SPARK-18144][SQL] logging StreamingQueryListener...

2016-10-31 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15675#discussion_r85827662 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala --- @@ -41,7 +41,9 @@ class

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85829905 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1700,19 +1700,34 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #14653: [SPARK-10931][PYSPARK][ML] PySpark ML Models should cont...

2016-10-31 Thread evanyc15
Github user evanyc15 commented on the issue: https://github.com/apache/spark/pull/14653 Hey @jkbradley the checkParams method already exists in the Python side. It's defined in the tests.py DefaultValuesTests class and is being called by test_java_params. I'm removing the param

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15538 In fact, if no one else is working on the Parquet upgrade it probably makes more sense for me to contribute that then continue working on this PR. I'll check with the dev mailing list. --- If

[GitHub] spark issue #15688: [SPARK-18173][SQL] data source tables should support tru...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15688 If we try to truncate a partition that does not exist, we just do nothing? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15688: [SPARK-18173][SQL] data source tables should supp...

2016-10-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15688#discussion_r85834452 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1636,15 +1635,23 @@ class DDLSuite extends QueryTest

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #67837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67837/consoleFull)** for PR 1 at commit

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-10-31 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/1 @hvanhovell, can you please relaunch the build? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15696 **[Test build #67841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67841/consoleFull)** for PR 15696 at commit

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85829640 --- Diff: R/pkg/R/context.R --- @@ -290,6 +290,33 @@ spark.addFile <- function(path, recursive = FALSE) { invisible(callJMethod(sc, "addFile",

[GitHub] spark issue #15700: [SPARK-17964][SparkR] Enable SparkR with Mesos client mo...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15700 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14444: [SPARK-16839] [SQL] redundant aliases after clean...

2016-10-31 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r85839282 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -137,7 +137,7 @@ object ColumnStatStruct {

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15704 **[Test build #67843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67843/consoleFull)** for PR 15704 at commit

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15675 **[Test build #67833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67833/consoleFull)** for PR 15675 at commit

[GitHub] spark issue #15695: [SPARK-18143][SQL]Ignore Structured Streaming event logs...

2016-10-31 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15695 Merging in branch-2.0. Can you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-10-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15697 windows failure seems unrelated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15688: [SPARK-18173][SQL] data source tables should support tru...

2016-10-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15688 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15608: [SPARK-17838][SparkR] Check named arguments for options ...

2016-10-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15608 LGTM. I ran through a few other cases and I think the omitted names are handled properly with this. This should go to master then (handledCallJMethod is not in branch-2.0) --- If your

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2016-10-31 Thread mariusvniekerk
Github user mariusvniekerk commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r85833766 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1700,19 +1700,34 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15659 **[Test build #67824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67824/consoleFull)** for PR 15659 at commit

[GitHub] spark issue #15698: [SPARK-18182] Expose ReplayListenerBus.read() overload w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15698 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67828/ Test FAILed. ---

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/15659 We have an AppVeyor build now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15701: [SPARK-18167] [SQL] Also log all partitions when the SQL...

2016-10-31 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15701 @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15673 **[Test build #67834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67834/consoleFull)** for PR 15673 at commit

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-10-31 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15703 Will add more details in the PR description soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15302: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-31 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15302 Hi, @hvanhovell . I made another attempt #15704 by using 'Expression' as you commented. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15675: [SPARK-18144][SQL] logging StreamingQueryListener$QueryS...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15675 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15626 **[Test build #67831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67831/consoleFull)** for PR 15626 at commit

[GitHub] spark issue #15697: [SparkR][Test]:remove unnecessary suppressWarnings

2016-10-31 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15697 retest it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15673 @ericl I've pushed a commit with the changes you recommended. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15701: [SPARK-18167] [SQL] Also log all partitions when ...

2016-10-31 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15701 [SPARK-18167] [SQL] Also log all partitions when the SQLQuerySuite test flakes ## What changes were proposed in this pull request? One possibility for this test flaking is that we have

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67835/ Test FAILed. ---

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67829/ Test FAILed. ---

[GitHub] spark issue #15696: [SPARK-18024][SQL] Introduce an internal commit protocol...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15696 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15626: SPARK-17829 [SQL] Stable format for offset log

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15659 @nchammas Ah, yes it is. It is a known problem. AppVeyor triggers the build when it has the changes in `R` directory but when it's merged (instead of rebased) for example,

[GitHub] spark pull request #15696: [SPARK-18024][SQL] Introduce an internal commit p...

2016-10-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15696#discussion_r85848150 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala --- @@ -141,15 +139,14 @@ class SimpleTextOutputWriter( }

[GitHub] spark issue #15703: [SPARK-18186] Migrate HiveUDAFFunction to TypedImperativ...

2016-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15703 **[Test build #67842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67842/consoleFull)** for PR 15703 at commit

[GitHub] spark issue #15677: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15677 Okay, @gatorsmile could you maybe submit a small one first please? Will refer yours and then I can work together (we can divide the files or packages to deal with I guess). --- If your

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67834/ Test FAILed. ---

[GitHub] spark issue #15673: [SPARK-17992][SQL] Return all partitions from HiveShim w...

2016-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15673 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85848802 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85848940 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85847613 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/AesCipher.java --- @@ -0,0 +1,311 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85847089 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/SaslRpcHandler.java --- @@ -80,46 +84,71 @@ public void receive(TransportClient

[GitHub] spark issue #15699: [SPARK-18030][Tests]Fix flaky FileStreamSourceSuite by n...

2016-10-31 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15699 Thanks! Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85850906 --- Diff: common/network-common/src/test/java/org/apache/spark/network/sasl/SparkSaslSuite.java --- @@ -374,6 +375,69 @@ public void testDelegates() throws

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85846395 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/SaslRpcHandler.java --- @@ -80,46 +84,71 @@ public void receive(TransportClient

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-10-31 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r85849868 --- Diff: common/network-common/src/main/java/org/apache/spark/network/util/ByteArrayReadableChannel.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to

  1   2   3   4   5   6   >