[GitHub] spark issue #18230: [SPARK-19688] [STREAMING] Not to read `spark.yarn.creden...

2017-06-19 Thread saturday-shi
Github user saturday-shi commented on the issue: https://github.com/apache/spark/pull/18230 @vanzin [Xing Shi (saturday_s)](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=saturday_s), thanks. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122866332 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -328,6 +333,9 @@ case class DataSource(

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122867252 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -328,6 +333,9 @@ case class DataSource(

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18025 haha. I like the `\emph` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18355 **[Test build #78270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78270/testReport)** for PR 18355 at commit

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17758 **[Test build #78267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78267/testReport)** for PR 17758 at commit

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17758 yea, I've already found the cause; to fix the issue, it's okay to check name duplication for partition columns in `getOrInferFileFormatSchema` as @gatorsmile suggested

[GitHub] spark pull request #18328: [SPARK-21121][SQL] Support changing storage level...

2017-06-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18328#discussion_r122872220 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -106,6 +105,11 @@ class CacheManager extends Logging {

[GitHub] spark pull request #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name ...

2017-06-19 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/18356 [SPARK-21144][SQL][BRANCH-2.2] Check column name duplication in read/write paths ## What changes were proposed in this pull request? This pr fixed unexpected results when the data schema and

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18356 cc: @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18320 **[Test build #78269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78269/testReport)** for PR 18320 at commit

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122860370 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark issue #18357: [SPARK-21146] [CORE] Worker should handle and shutdown w...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18357 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18343 thanks, merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18320 I also tested the current state on CentOS for sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122866890 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/util/SchemaUtils.scala --- @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122866830 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -355,12 +356,12 @@ object ViewHelper { analyzedPlan:

[GitHub] spark pull request #18288: [SPARK-21066][ML] LibSVM load just one input file

2017-06-19 Thread darionyaphet
Github user darionyaphet commented on a diff in the pull request: https://github.com/apache/spark/pull/18288#discussion_r122869702 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -91,12 +91,10 @@ private[libsvm] class LibSVMFileFormat

[GitHub] spark pull request #18025: [SPARK-20889][SparkR] Grouped documentation for A...

2017-06-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18025 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122855545 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -31,7 +31,15 @@ inputCon <- socketConnection( port = port, open = "rb", blocking = TRUE, timeout =

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18355 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18355 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78266/ Test FAILed. ---

[GitHub] spark issue #18351: [SPARK-21135][WEB UI] On history server page,duration ...

2017-06-19 Thread fjh100456
Github user fjh100456 commented on the issue: https://github.com/apache/spark/pull/18351 Yes,it should be. @ajbozarth The screenshot:@zhuoliu ![default](https://user-images.githubusercontent.com/26785576/27312007-89a3eca6-5597-11e7-81fe-7dcff2c2a861.png)

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18025 AppVeyor failure is unfortunate. but it passed before a doc only change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18140: [SPARK-20917][ML][SparkR] SparkR supports string encodin...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18140 you can close and re-open this PR on github here --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18140: [SPARK-20917][ML][SparkR] SparkR supports string encodin...

2017-06-19 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18140 How do I do that? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78267/ Test PASSed. ---

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18359: Update Word2Vec.scala

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18359 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122875289 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17758 **[Test build #78267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78267/testReport)** for PR 17758 at commit

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15821 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78265/ Test PASSed. ---

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15821 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122862830 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -248,6 +249,10 @@ private[hive] class

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18320 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78269/ Test PASSed. ---

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18320 **[Test build #78269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78269/testReport)** for PR 18320 at commit

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18320 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18358: [SPARK-21148] [CORE] Set SparkUncaughtExceptionHandler t...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18358 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18358: [SPARK-21148] [CORE] Set SparkUncaughtExceptionHa...

2017-06-19 Thread devaraj-kavali
GitHub user devaraj-kavali opened a pull request: https://github.com/apache/spark/pull/18358 [SPARK-21148] [CORE] Set SparkUncaughtExceptionHandler to the Master ## What changes were proposed in this pull request? Adding the default UncaughtExceptionHandler to the Master as

[GitHub] spark pull request #18359: Update Word2Vec.scala

2017-06-19 Thread lawlietAi
GitHub user lawlietAi opened a pull request: https://github.com/apache/spark/pull/18359 Update Word2Vec.scala ## What changes were proposed in this pull request? the word2vec model needs an independent function to calculate the cosine similarity.we also desire a function

[GitHub] spark pull request #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name ...

2017-06-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18356#discussion_r122874896 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -222,12 +223,10 @@ case class

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18356 To avoid potential issues, could you revert all the unrelated changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18355 **[Test build #78270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78270/testReport)** for PR 18355 at commit

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18114 **[Test build #78272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78272/testReport)** for PR 18114 at commit

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 Agreed. The `hugeBlockSizes` map is not supposed to have too many records but only few huge blocks. LGTM --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15821 **[Test build #78265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78265/testReport)** for PR 15821 at commit

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122860937 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122862386 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -687,4 +688,52 @@ class DataFrameReaderWriterSuite

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18356 **[Test build #78268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78268/testReport)** for PR 18356 at commit

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18356 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78268/ Test PASSed. ---

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18356 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18356 **[Test build #78268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78268/testReport)** for PR 18356 at commit

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122861395 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122863353 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -328,6 +333,9 @@ case class DataSource(

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122867659 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -62,13 +63,8 @@ case class

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18025 merged to master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18140: [SPARK-20917][ML][SparkR] SparkR supports string ...

2017-06-19 Thread actuaryzhang
GitHub user actuaryzhang reopened a pull request: https://github.com/apache/spark/pull/18140 [SPARK-20917][ML][SparkR] SparkR supports string encoding consistent with R ## What changes were proposed in this pull request? Add `stringIndexerOrderType` to `spark.glm` and

[GitHub] spark pull request #18140: [SPARK-20917][ML][SparkR] SparkR supports string ...

2017-06-19 Thread actuaryzhang
Github user actuaryzhang closed the pull request at: https://github.com/apache/spark/pull/18140 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18114 For the `column_datetime_diff_functions`: ![image](https://user-images.githubusercontent.com/11082368/27315654-9ba01c08-552f-11e7-973e-f8351cb50aae.png)

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18114 For the date time functions, I create two groups: one for arithmetic functions that work with two columns `column_datetime_diff_functions`, and the other for functions that work with only one

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122853060 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -182,6 +183,10 @@ case class DataSource(

[GitHub] spark issue #18348: [SPARK-21120][CORE] Increasing the master's metric is co...

2017-06-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/18348 @srowen Sorry, the last two or three days I did not deal with my jira in time. Please help to review the code, thanks. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18355 **[Test build #78266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78266/testReport)** for PR 18355 at commit

[GitHub] spark issue #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name duplica...

2017-06-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18356 @gatorsmile This pr included whole changes in #17758 though, you originally meant this pr should include a part of them to fix this issue only? --- If your project is set up for it, you can reply

[GitHub] spark pull request #18357: [SPARK-21146] [CORE] Worker should handle and shu...

2017-06-19 Thread devaraj-kavali
GitHub user devaraj-kavali opened a pull request: https://github.com/apache/spark/pull/18357 [SPARK-21146] [CORE] Worker should handle and shutdown when any thread gets UncaughtException ## What changes were proposed in this pull request? Adding the default

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122860310 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark issue #18140: [SPARK-20917][ML][SparkR] SparkR supports string encodin...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18140 can you kick AppVeyor? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18114 **[Test build #78271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78271/testReport)** for PR 18114 at commit

[GitHub] spark pull request #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name ...

2017-06-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18356#discussion_r122875121 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -181,6 +182,10 @@ case class DataSource(

[GitHub] spark pull request #18356: [SPARK-21144][SQL][BRANCH-2.2] Check column name ...

2017-06-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18356#discussion_r122874997 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -181,6 +182,10 @@ case class DataSource(

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18114 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18355 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18114 **[Test build #78271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78271/testReport)** for PR 18114 at commit

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78271/ Test FAILed. ---

[GitHub] spark issue #18355: [SPARK-21145][SS] Added StateStoreProviderId with queryR...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18355 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78270/ Test FAILed. ---

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r122879650 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,42 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18114 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18114 **[Test build #78272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78272/testReport)** for PR 18114 at commit

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18346 Btw, even we can evaluate all children expressions of `CodegenFallback` with codegen path, we still can't do wholestage codegen with the plans including `CodegenFallback` expressions. We just can do

[GitHub] spark issue #14085: [SPARK-16408][SQL] SparkSQL Added file get Exception: is...

2017-06-19 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/14085 @zenglinxi0615 This pr is about adding all files in a directory recursively, thus no need to enumerate all the filenames? I think this can be pretty useful especially in production env.

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18343 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...

2017-06-19 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/18029 @budde @brkyvz could you suggest if the current patch seems ok, or I should make something similar to the case class/ trait ? --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17758 I think we should figure out https://issues.apache.org/jira/browse/SPARK-21144 first. It doesn't make sense to have duplicated columns between partition columns and data columns. --- If your

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122868692 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -62,13 +63,8 @@ case class

[GitHub] spark issue #18320: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18320 > Also I'd suggest not committing this to branch-2.2 -- if we want to just fix the CentOS tests we can have a different change for the older branches agreed, this won't run as a part of

[GitHub] spark issue #18350: [MINOR] Fix some typo of the document

2017-06-19 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/18350 thanks @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122865721 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/util/SchemaUtils.scala --- @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17758: [SPARK-20460][SQL] Make it more consistent to han...

2017-06-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r122865863 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -355,12 +356,12 @@ object ViewHelper {

[GitHub] spark pull request #17395: [SPARK-20065][SS][WIP] Avoid to output empty parq...

2017-06-19 Thread uncleGen
Github user uncleGen closed the pull request at: https://github.com/apache/spark/pull/17395 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18359: Update Word2Vec.scala

2017-06-19 Thread lawlietAi
Github user lawlietAi commented on the issue: https://github.com/apache/spark/pull/18359 sorry i'm confused about to operate the github.what should i do --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78272/ Test PASSed. ---

[GitHub] spark pull request #18353: Corrected kafka dependencies

2017-06-19 Thread timvw
GitHub user timvw opened a pull request: https://github.com/apache/spark/pull/18353 Corrected kafka dependencies ## What changes were proposed in this pull request? Currently spark-streaming-kafka-0-10 has a dependency on the full kafka distribution (but only uses and

[GitHub] spark pull request #18353: Corrected kafka dependencies

2017-06-19 Thread timvw
Github user timvw commented on a diff in the pull request: https://github.com/apache/spark/pull/18353#discussion_r122722841 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaTestUtils.scala --- @@ -20,25 +20,24 @@ package

[GitHub] spark issue #14739: [SPARK-17176][WEB UI]set default task sort column to "St...

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/14739 @srowen should we close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/11994 Are you still working on this? @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13863: [SPARK-16158][CORE][WIP] Make dynamic allocation heurist...

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/13863 Are you still working on this? @nezihyigitbasi --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16803: [SPARK-19458][BUILD]load hive jars from local repo which...

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/16803 ping @cloud-fan @gatorsmile @dongjoon-hyun Any thoughts on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17074: [SPARK-18646][REPL] Set parent classloader as null for E...

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17074 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17620: [SPARK-20305][Spark Core]Master may keep in the state of...

2017-06-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17620 Should we move forward with this PR or should we close this? @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17074: [SPARK-18646][REPL] Set parent classloader as null for E...

2017-06-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17074 **[Test build #78258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78258/testReport)** for PR 17074 at commit

  1   2   3   4   >