[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104994473 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -51,6 +51,47 @@ object UnsupportedOpe

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104994403 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -116,9 +157,49 @@ object UnsupportedO

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104994169 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -116,9 +157,49 @@ object UnsupportedO

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104994150 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -138,29 +138,202 @@ class UnsupportedO

[GitHub] spark pull request #17204: [SPARK-19864][SQL][TEST]provide a makeQualifiedPa...

2017-03-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17204: [SPARK-19864][SQL][TEST]provide a makeQualifiedPath func...

2017-03-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17204 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11601 **[Test build #74216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74216/testReport)** for PR 11601 at commit [`c67afc1`](https://github.com/apache/spark/commit/c6

[GitHub] spark pull request #17172: [SPARK-19008][SQL] Improve performance of Dataset...

2017-03-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17172#discussion_r104991089 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/objects.scala --- @@ -219,7 +219,33 @@ case class MapElementsExec( override def do

[GitHub] spark issue #16826: [SPARK-19540][SQL] Add ability to clone SparkSession whe...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16826 **[Test build #74215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74215/testReport)** for PR 16826 at commit [`4c23e7a`](https://github.com/apache/spark/commit/4c

[GitHub] spark pull request #17172: [SPARK-19008][SQL] Improve performance of Dataset...

2017-03-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17172#discussion_r104990128 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/objects.scala --- @@ -219,7 +219,33 @@ case class MapElementsExec( override def do

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-03-08 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/17087 I don't think we need a complex refactoring. Why can't `newPredicate` catch the exception, log a warning and return an interpreted `Predicate`? --- If your project is set up for it, you can reply

[GitHub] spark issue #17196: [SPARK-19855][SQL] Create an internal FilePartitionStrat...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17196 **[Test build #3598 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3598/testReport)** for PR 17196 at commit [`ad5c7d1`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #17202: [SPARK-19861][SS] watermark should not be a negat...

2017-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17202#discussion_r104983300 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -576,6 +576,11 @@ class Dataset[T] private[sql]( val parsedDelay =

[GitHub] spark pull request #17202: [SPARK-19861][SS] watermark should not be a negat...

2017-03-08 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17202#discussion_r104983221 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -563,7 +563,7 @@ class Dataset[T] private[sql]( * @param eventTime the name

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104879474 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -116,9 +157,49 @@ object UnsupportedOper

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104880626 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -116,9 +157,49 @@ object UnsupportedOper

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104873805 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -51,6 +51,47 @@ object UnsupportedOperat

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104875573 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -51,6 +51,47 @@ object UnsupportedOperat

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104880840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -116,9 +157,49 @@ object UnsupportedOper

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104981881 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -138,29 +138,202 @@ class UnsupportedOper

[GitHub] spark pull request #17197: [SPARK-19858][SS]Add output mode to flatMapGroups...

2017-03-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17197#discussion_r104981745 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -138,29 +138,202 @@ class UnsupportedOper

[GitHub] spark issue #17091: [SPARK-19757][CORE] DriverEndpoint#makeOffers race again...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17091 **[Test build #74214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74214/testReport)** for PR 17091 at commit [`9ade0fd`](https://github.com/apache/spark/commit/9a

[GitHub] spark issue #17212: [SPARK-17979] Remove deprecated support for config SPARK...

2017-03-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17212 And please remove the template text from your PR description. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #17091: [SPARK-19757][CORE] DriverEndpoint#makeOffers race again...

2017-03-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17091 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17212: [SPARK-17979] Remove deprecated support for config SPARK...

2017-03-08 Thread yongtang
Github user yongtang commented on the issue: https://github.com/apache/spark/pull/17212 Thanks @srowen Will take a look at other envs and update the PR if relevant. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17213: [SPARK-13740] [PySpark][SQL] Improve error messag...

2017-03-08 Thread blrnw3
GitHub user blrnw3 opened a pull request: https://github.com/apache/spark/pull/17213 [SPARK-13740] [PySpark][SQL] Improve error message in verify_type to indicate the field https://issues.apache.org/jira/browse/SPARK-19871 ## What changes were proposed in this pull request

[GitHub] spark issue #17213: [SPARK-19871] [PySpark][SQL] Improve error message in ve...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17213 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-03-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16729 @actuaryzhang that's true, it's not ideal. This is somewhat an unusual case for R for several reasons. In my head the guiding principles are: - we avoid depending on another packa

[GitHub] spark issue #17212: [SPARK-17979] Remove deprecated support for config SPARK...

2017-03-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17212 I think it's indeed OK to remove at this point because it's not documented, and because of the JIRA discussion. However it's ideal if you can look for other SPARK_* env variables that are long since

[GitHub] spark issue #16592: [SPARK-19235] [SQL] [TESTS] Enable Test Cases in DDLSuit...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16592 **[Test build #74213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74213/testReport)** for PR 16592 at commit [`64a8f5a`](https://github.com/apache/spark/commit/64

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15363 https://github.com/apache/spark/pull/17138 has been merged. Maybe we can start reviewing this PR @wzhfy @hvanhovell @cloud-fan @sameeragarwal ? --- If your project is set up for it, you can repl

[GitHub] spark pull request #17205: [SPARK-19843] [SQL] [Followup] Classdoc for `IntW...

2017-03-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #16933: [SPARK-19601] [SQL] Fix CollapseRepartition rule ...

2017-03-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16933 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16592: [SPARK-19235] [SQL] [TESTS] Enable Test Cases in DDLSuit...

2017-03-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16592 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17205: [SPARK-19843] [SQL] [Followup] Classdoc for `IntWrapper`...

2017-03-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17205 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16933: [SPARK-19601] [SQL] Fix CollapseRepartition rule to pres...

2017-03-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16933 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #17212: [SPARK-17979] Remove deprecated support for config SPARK...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17212 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17212: [SPARK-17979] Remove deprecated support for confi...

2017-03-08 Thread yongtang
GitHub user yongtang opened a pull request: https://github.com/apache/spark/pull/17212 [SPARK-17979] Remove deprecated support for config SPARK_YARN_USER_ENV ## What changes were proposed in this pull request? This fix removes deprecated support for config `SPARK_YARN_USER_E

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17202 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74211/ Test PASSed. ---

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17202 **[Test build #74211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74211/testReport)** for PR 17202 at commit [`b0c8f12`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-03-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17087 @marmbrus thank you for your comments. For feedback mechanism, I imagine that you are talking about [this](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/

[GitHub] spark pull request #17178: [SPARK-19828][R] Support array type in from_json ...

2017-03-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17178#discussion_r104971290 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1342,28 +1342,52 @@ test_that("column functions", { df <- read.json(mapTypeJsonPath)

[GitHub] spark issue #17207: [SPARK-19282][ML][SparkR] Expose model param "maxDepth" ...

2017-03-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17207 Can you clarify? this is on the model summary, not a param? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/17208 This looks like the right change. In fact, I could have sworn we had recently merged in something like this -- maybe there is another pr still in flight which includes this? @jinxing64 perhaps this

[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17208 **[Test build #74212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74212/testReport)** for PR 17208 at commit [`6c40b9f`](https://github.com/apache/spark/commit/6c

[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/17208 Jenkins, ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-08 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/17170 @felixcheung I think i addressed all the issues excluding `inputCol` and `predictionCol`. In general: - Using `formula`as an input doesn't make sense in my opinion. - Personally I would

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17211 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74210/ Test PASSed. ---

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17211 **[Test build #74210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74210/testReport)** for PR 17211 at commit [`ac79b05`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #17126: [SPARK-19789][SS]Add the shortcut of .format("parquet")....

2017-03-08 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/17126 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #17087: [SPARK-19372][SQL] Fix throwing a Java exception ...

2017-03-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17087#discussion_r104955336 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -213,12 +217,30 @@ case class FilterExec(condition: Expr

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-03-08 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r104950862 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,243 @@ +/* + * Licensed

[GitHub] spark pull request #17202: [SPARK-19861][SS] watermark should not be a negat...

2017-03-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17202#discussion_r104944765 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -576,6 +576,11 @@ class Dataset[T] private[sql]( val parsedDelay =

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17202 **[Test build #74211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74211/testReport)** for PR 17202 at commit [`b0c8f12`](https://github.com/apache/spark/commit/b0

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17210 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-03-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r104940657 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,243 @@ +/* + * Licensed t

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17210 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #17210: [SPARK-19865][SQL] remove the view identifier in ...

2017-03-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17210 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-08 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-08 Thread kishorvpatil
Github user kishorvpatil commented on the issue: https://github.com/apache/spark/pull/15009 The build failure seems unrelated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-08 Thread kishorvpatil
Github user kishorvpatil commented on the issue: https://github.com/apache/spark/pull/15009 @vanzin @tgravescs, I have updates the patch with suggested changes. - Removed unused `connect` method from `LauncherBackend`. - Documented expected behavior for `SparkAppHandler#disc

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17210 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74208/ Test PASSed. ---

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17210 **[Test build #74208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74208/testReport)** for PR 17210 at commit [`7ee5405`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17138 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17138 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74207/ Test PASSed. ---

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17138 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17138 **[Test build #74207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74207/testReport)** for PR 17138 at commit [`757e3b6`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17052 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17211 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #17202: [SPARK-19861][SS] watermark should not be a negat...

2017-03-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17202#discussion_r104931873 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -576,6 +576,11 @@ class Dataset[T] private[sql]( val parsedDelay =

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17052 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74206/ Test PASSed. ---

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17052 **[Test build #74206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74206/testReport)** for PR 17052 at commit [`ff38db7`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #17202: [SPARK-19861][SS] watermark should not be a negat...

2017-03-08 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/17202#discussion_r104930545 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -576,6 +576,11 @@ class Dataset[T] private[sql]( val parsedDelay =

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17211 **[Test build #74210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74210/testReport)** for PR 17211 at commit [`ac79b05`](https://github.com/apache/spark/commit/ac

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17211 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17211 **[Test build #74209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74209/testReport)** for PR 17211 at commit [`c31ae7f`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74209/ Test FAILed. ---

[GitHub] spark issue #17211: [SPARK-19869][SQL]move table related ddl from ddl.scala ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17211 **[Test build #74209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74209/testReport)** for PR 17211 at commit [`c31ae7f`](https://github.com/apache/spark/commit/c3

[GitHub] spark pull request #17211: [SPARK-19869][SQL]move table related ddl from ddl...

2017-03-08 Thread windpiger
GitHub user windpiger opened a pull request: https://github.com/apache/spark/pull/17211 [SPARK-19869][SQL]move table related ddl from ddl.scala to tables.scala ## What changes were proposed in this pull request? move table related ddl from ddl.scala to tables.scala

[GitHub] spark pull request #13735: [SPARK-15328][MLLIB][ML] Word2Vec import for orig...

2017-03-08 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/13735 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104924893 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/KMeansSuite.scala --- @@ -152,6 +158,35 @@ class KMeansSuite extends SparkFunSuite with ML

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17210 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark pull request #17117: [SPARK-10780][ML] Support initial model for KMean...

2017-03-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/17117#discussion_r104922364 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala --- @@ -300,6 +318,10 @@ class KMeans @Since("1.5.0") ( @Since("1.5.0")

[GitHub] spark pull request #17200: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-03-08 Thread JasonMWhite
Github user JasonMWhite commented on a diff in the pull request: https://github.com/apache/spark/pull/17200#discussion_r104919956 --- Diff: python/pyspark/sql/types.py --- @@ -189,7 +189,7 @@ def toInternal(self, dt): if dt is not None: seconds = (calen

[GitHub] spark issue #17210: [SPARK-19865][SQL] remove the view identifier in Subquer...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17210 **[Test build #74208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74208/testReport)** for PR 17210 at commit [`7ee5405`](https://github.com/apache/spark/commit/7e

[GitHub] spark pull request #17210: [SPARK-19865][SQL] remove the view identifier in ...

2017-03-08 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/17210 [SPARK-19865][SQL] remove the view identifier in SubqueryAlias ## What changes were proposed in this pull request? Since we have a `View` node now, we can remove the view identifier in

[GitHub] spark issue #17189: [SPARK-19831][CORE] Use a separate thread to clean up th...

2017-03-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17189 I think you need to call `cleanupApplicationThreadExecutor.shutdownNow()` when `worker` is stopped. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17138 **[Test build #74207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74207/testReport)** for PR 17138 at commit [`757e3b6`](https://github.com/apache/spark/commit/75

[GitHub] spark issue #17203: [SPARK-19863][DStream] Whether or not use CachedKafkaCon...

2017-03-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17203 What is the purpose of this change, did you see any problem when using cached consumers? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17138: [SPARK-17080] [SQL] join reorder

2017-03-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17138 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17202 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74204/ Test PASSed. ---

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17202 **[Test build #74204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74204/testReport)** for PR 17202 at commit [`c5e2221`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #17192: [SPARK-19849][SQL] Support ArrayType in to_json to produ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74202/ Test PASSed. ---

[GitHub] spark issue #17192: [SPARK-19849][SQL] Support ArrayType in to_json to produ...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17192 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17192: [SPARK-19849][SQL] Support ArrayType in to_json to produ...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17192 **[Test build #74202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74202/testReport)** for PR 17192 at commit [`ed0bbae`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #17202: [SPARK-19861][SS] watermark should not be a negative tim...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74203/ Test PASSed. ---

<    1   2   3   4   5   6   >