[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16772 well...will merge after new tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16772 jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2017-02-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 ## Future Work The support of all Spark data types to include complex types will be a quick follow up to ensure full compatibility. With the added ability to convert a Dataset on the

[GitHub] spark pull request #16930: [SPARK-19597][CORE] test case for task deserializ...

2017-02-23 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16930#discussion_r102863614 --- Diff: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala --- @@ -133,6 +122,79 @@ class ExecutorSuite extends SparkFunSuite {

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16987 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17043 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17043 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73380/ Test PASSed. ---

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73380/testReport)** for PR 17043 at commit

[GitHub] spark pull request #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy metho...

2017-02-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16772 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16923: [SPARK-19038][YARN] Avoid overwriting keytab configurati...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16923 **[Test build #73386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73386/testReport)** for PR 16923 at commit

[GitHub] spark issue #17038: [SPARK-19707][Core] Improve the invalid path check for s...

2017-02-23 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17038 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102870703 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -408,7 +408,13 @@ private[spark] class

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102871210 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -408,7 +408,13 @@ private[spark] class

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-02-23 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16696 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102873675 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -408,7 +408,13 @@ private[spark] class

[GitHub] spark issue #17046: [SPARK-19691][SQL][BRANCH-2.1] Fix ClassCastException wh...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17046 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17046: [SPARK-19691][SQL][BRANCH-2.1] Fix ClassCastException wh...

2017-02-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17046 @hvanhovell okay, ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73368/testReport)** for PR 17043 at commit

[GitHub] spark issue #12524: [SPARK-12524][Core]DagScheduler may submit a task set fo...

2017-02-23 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/12524 Can you update the PR description here to have the JIRA number (SPARK-14658), not the PR number? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #17009: [SPARK-19674][SQL]Ignore driver accumulator updates don'...

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17009 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17044: [SPARK-19718][SS]Handle more interrupt cases properly fo...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17044 **[Test build #73369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73369/testReport)** for PR 17044 at commit

[GitHub] spark issue #17041: [SPARK-19459][SQL][BRANCH-2.1] Support for nested char/v...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17041 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73358/ Test PASSed. ---

[GitHub] spark issue #17041: [SPARK-19459][SQL][BRANCH-2.1] Support for nested char/v...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17041 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.

2017-02-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/17018 Yeah asking them for 1.5hr seems like a good first step. We can also see what fraction of time it takes for build vs. tests. I guess we can't make builds faster without some effort, but we could

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r102843676 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,264 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.

2017-02-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17018 I see. Sure. Just as a side note, if you download the log, it prints the times. In this case, it was so close assuming from the last log :). ``` [00:59:47] Windows-specific

[GitHub] spark issue #17045: [SPARK-19373][MESOS] fix spark.scheduler.minRegisteredRe...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17045 **[Test build #73370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73370/testReport)** for PR 17045 at commit

[GitHub] spark pull request #17045: [SPARK-19373] Base spark.scheduler.minRegisteredR...

2017-02-23 Thread mgummelt
GitHub user mgummelt opened a pull request: https://github.com/apache/spark/pull/17045 [SPARK-19373] Base spark.scheduler.minRegisteredResourceRatio on registered cores r… …ather than acquired cores in the Mesos Coarse Grained Scheduler ## What changes were proposed in

[GitHub] spark issue #17044: [SPARK-19718][SS]Handle more interrupt cases properly fo...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73369/ Test PASSed. ---

[GitHub] spark issue #17044: [SPARK-19718][SS]Handle more interrupt cases properly fo...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17044 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17032: [SPARK-19460][SparkR]:Update dataset used in R do...

2017-02-23 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/17032#discussion_r102856602 --- Diff: examples/src/main/r/ml/kmeans.R --- @@ -26,10 +26,12 @@ sparkR.session(appName = "SparkR-ML-kmeans-example") # $example on$ #

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16992 **[Test build #73367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73367/testReport)** for PR 16992 at commit

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17032 **[Test build #73375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73375/testReport)** for PR 17032 at commit

[GitHub] spark pull request #17032: [SPARK-19460][SparkR]:Update dataset used in R do...

2017-02-23 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/17032#discussion_r102857809 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -565,11 +565,10 @@ We use a simple example to demonstrate `spark.logit` usage. In general, there

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-23 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r102859179 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -161,22 +164,45 @@ private[hive] class

[GitHub] spark pull request #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-23 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/15415#discussion_r102860331 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -0,0 +1,346 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #17046: [SPARK-19691][SQL][BRANCH-2.1] Fix ClassCastException wh...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17046 **[Test build #73378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73378/testReport)** for PR 17046 at commit

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-23 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r102848520 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,63 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Stage can hang if an executor fails...

2017-02-23 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r102860963 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-23 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r102862437 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -839,6 +869,7 @@ object SparkSession { if

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73380/testReport)** for PR 17043 at commit

[GitHub] spark pull request #16996: [SPARK-19664][SQL]put hive.metastore.warehouse.di...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/16996#discussion_r102863165 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -61,6 +62,9 @@ private[sql] class SharedState(val sparkContext:

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73381/testReport)** for PR 17043 at commit

[GitHub] spark issue #12888: [SPARK-14772][ML,PySpark]Python ML Params.copy treats ui...

2017-02-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/12888 @hujy Thank you for sending this PR, and apologies for not seeing it earlier. Since the other PR for this JIRA is ready to merge, could you please close this issue? Thanks again! --- If your

[GitHub] spark issue #16534: [SPARK-19161][PYTHON][SQL] Improving UDF Docstrings

2017-02-23 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16534 Yes pydoc.help does depend on looking at the docstring on the type rather than the object :( Too bad the IPython magic isn't used in pydoc too. Sorry for all the back and forth, I'm just

[GitHub] spark issue #16996: [SPARK-19664][SQL]put hive.metastore.warehouse.dir in ha...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16996 **[Test build #73389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73389/testReport)** for PR 16996 at commit

[GitHub] spark issue #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15415 **[Test build #73384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73384/testReport)** for PR 15415 at commit

[GitHub] spark issue #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15415 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73384/ Test PASSed. ---

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102870072 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -408,7 +408,13 @@ private[spark] class

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15125 **[Test build #73390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73390/testReport)** for PR 15125 at commit

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102870494 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -193,7 +193,16 @@ class SessionCatalog(

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102870643 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -193,7 +193,16 @@ class SessionCatalog(

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73377/testReport)** for PR 16395 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16395 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16696 **[Test build #73391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73391/testReport)** for PR 16696 at commit

[GitHub] spark issue #16826: [SPARK-19540][SQL] Add ability to clone SparkSession whe...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16826 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16826: [SPARK-19540][SQL] Add ability to clone SparkSession whe...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73379/ Test PASSed. ---

[GitHub] spark issue #16923: [SPARK-19038][YARN] Avoid overwriting keytab configurati...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16923 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16923: [SPARK-19038][YARN] Avoid overwriting keytab configurati...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16923 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73386/ Test PASSed. ---

[GitHub] spark issue #17001: [SPARK-19667][SQL]create table with hiveenabled in defau...

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17001 In `SharedState`, we should create the default database without location. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16923: [SPARK-19038][YARN] Avoid overwriting keytab configurati...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16923 **[Test build #73386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73386/testReport)** for PR 16923 at commit

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73368/testReport)** for PR 17043 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73361/testReport)** for PR 16944 at commit

[GitHub] spark pull request #17044: [SPARK-19718][SS]Handle more interrupt cases prop...

2017-02-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17044#discussion_r102838808 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -709,12 +717,13 @@ class StreamExecution(

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-23 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r102843314 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -161,22 +164,45 @@ private[hive] class

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-23 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r102841719 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf

[GitHub] spark issue #17042: [CORE][MINOR] Fix scaladoc

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17042 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73362/ Test FAILed. ---

[GitHub] spark pull request #17043: [SPARK-19719][SS][WIP] Kafka writer for both stru...

2017-02-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17043#discussion_r102846556 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala --- @@ -0,0 +1,264 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-23 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r102850496 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -296,6 +296,25 @@ object SQLConf { .longConf

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-02-23 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r102850475 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -161,22 +164,45 @@ private[hive] class

[GitHub] spark issue #16696: [SPARK-19350] [SQL] Cardinality estimation of Limit and ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16696 **[Test build #73372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73372/testReport)** for PR 16696 at commit

[GitHub] spark pull request #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread lw-lin
GitHub user lw-lin reopened a pull request: https://github.com/apache/spark/pull/16987 [SPARK-19633][SS] FileSource read from FileSink ## What changes were proposed in this pull request? Right now file source always uses `InMemoryFileIndex` to scan files from a given path.

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/16987 Reopening :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16987: [SPARK-19633][SS] FileSource read from FileSink

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16987 **[Test build #73374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73374/testReport)** for PR 16987 at commit

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73376/testReport)** for PR 16395 at commit

[GitHub] spark pull request #17033: [SPARK-16122][DOCS] application environment rest ...

2017-02-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17033 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16395 **[Test build #73377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73377/testReport)** for PR 16395 at commit

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73375/ Test PASSed. ---

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2017-02-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 ## Benchmarks with Conversion done on Executor (updated) ## 1mm Longs _ | With Arrow | Without Arrow --||--- count | 50.00 |50.00

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17032 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17032 **[Test build #73375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73375/testReport)** for PR 17032 at commit

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-23 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r102861839 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1178,4 +1178,36 @@ class SessionCatalog(

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16772 LGTM Merging with master @BryanCutler would you mind sending a backport PR against branch-2.1 to run Jenkins tests? Thank you! --- If your project is set up for it, you can

[GitHub] spark pull request #17047: [SPARK-19720][SPARK SUBMIT] Redact sensitive info...

2017-02-23 Thread markgrover
GitHub user markgrover opened a pull request: https://github.com/apache/spark/pull/17047 [SPARK-19720][SPARK SUBMIT] Redact sensitive information from SparkSubmit console ## What changes were proposed in this pull request? This change redacts senstive information (based on

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17043 **[Test build #73381 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73381/testReport)** for PR 17043 at commit

[GitHub] spark issue #17043: [SPARK-19719][SS][WIP] Kafka writer for both structured ...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17043 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73381/ Test PASSed. ---

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16772 OK merging now. @BryanCutler do let me know if you don't have time to send a backport--thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16772 **[Test build #3581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3581/testReport)** for PR 16772 at commit

[GitHub] spark pull request #17001: [SPARK-19667][SQL]create table with hiveenabled i...

2017-02-23 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17001#discussion_r102865566 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -193,7 +193,16 @@ class SessionCatalog(

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102863871 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,511 @@ +/*

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102862889 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,511 @@ +/*

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16772 **[Test build #73383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73383/testReport)** for PR 16772 at commit

[GitHub] spark pull request #16907: [SPARK-19572][SPARKR] Allow to disable hive in sp...

2017-02-23 Thread zjffdu
Github user zjffdu commented on a diff in the pull request: https://github.com/apache/spark/pull/16907#discussion_r102865692 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -48,13 +48,14 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102860256 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,511 @@ +/*

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102864512 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,511 @@ +/*

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102860017 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -0,0 +1,511 @@ +/*

[GitHub] spark pull request #16395: [SPARK-17075][SQL] implemented filter estimation

2017-02-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16395#discussion_r102862837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/Range.scala --- @@ -57,6 +58,20 @@ object Range {

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73383/ Test PASSed. ---

  1   2   3   4   5   6   7   >