[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15395 retest this please. This one has not yet passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request #14788: [SPARK-17174][SQL] Add the support for TimestampT...

2016-10-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14788#discussion_r82701507 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2548,16 +2548,20 @@ object functions { def to_date(e: Column): Colum

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15421 **[Test build #66682 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66682/consoleFull)** for PR 15421 at commit [`349a47b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66682/ Test PASSed. ---

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15386 **[Test build #66683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66683/consoleFull)** for PR 15386 at commit [`6259438`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15386 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66683/ Test PASSed. ---

[GitHub] spark pull request #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflic...

2016-10-10 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/15423 [SPARK-17860][SQL] SHOW COLUMN's database conflict check should respect case sensitivity configuration ## What changes were proposed in this pull request? SHOW COLUMNS command validates the

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15395 **[Test build #3320 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3320/consoleFull)** for PR 15395 at commit [`52b2b7d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-10 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15395 @zsxwing All tests passed in previous try: https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3317/consoleFull, I kicked another. --- If your project is set up for it, you can re

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15423 **[Test build #66692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66692/consoleFull)** for PR 15423 at commit [`3acd08f`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15190 **[Test build #66685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66685/consoleFull)** for PR 15190 at commit [`f2b93de`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15190 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66685/ Test PASSed. ---

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15190 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #7898: [SPARK-9560][MLlib] add lda data generator

2016-10-10 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/7898 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark issue #7898: [SPARK-9560][MLlib] add lda data generator

2016-10-10 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/7898 Close this as we've stopped development for MLlib. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #13675: [SPARK-15957] [ML] RFormula supports forcing to index la...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13675 **[Test build #66693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66693/consoleFull)** for PR 13675 at commit [`3079996`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #15072: [SPARK-17123][SQL] Use type-widened encoder for D...

2016-10-10 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/15072#discussion_r82704021 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -53,7 +53,15 @@ import org.apache.spark.util.Utils private[sql] objec

[GitHub] spark issue #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysis and O...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15316 **[Test build #66687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66687/consoleFull)** for PR 15316 at commit [`2c5de3c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysis and O...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15316 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysis and O...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15316 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66687/ Test PASSed. ---

[GitHub] spark issue #12374: [SPARK-14610][ML] Remove superfluous split for continuou...

2016-10-10 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/12374 LGTM Merging with master Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15421 **[Test build #66686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66686/consoleFull)** for PR 15421 at commit [`83726fc`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66686/ Test FAILed. ---

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15421 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14702 **[Test build #66689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66689/consoleFull)** for PR 14702 at commit [`c7741f9`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #12374: [SPARK-14610][ML] Remove superfluous split for co...

2016-10-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12374 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66689/ Test FAILed. ---

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14702 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12374: [SPARK-14610][ML] Remove superfluous split for continuou...

2016-10-10 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/12374 Thanks @jkbradley! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 cc @yhuai @cloud-fan Based on the above PR discussion, it sounds like this PR is ok to merge. What do you think? Thank you! --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #15307: [SPARK-17731][SQL][STREAMING] Metrics for structured str...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15307 **[Test build #66694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66694/consoleFull)** for PR 15307 at commit [`b747b23`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15388: [SPARK-17821][SQL] Support And and Or in Expression Cano...

2016-10-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15388 ping @rxin @hvanhovell @cloud-fan @gatorsmile any else need to address? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-level sta...

2016-10-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15360 Will review this tonight. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-10 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15285 ping. any updates? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-10 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r82706375 --- Diff: core/src/main/java/org/apache/spark/io/NioBasedBufferedFileInputStream.java --- @@ -0,0 +1,127 @@ +/* + * Licensed under the Apache Lice

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-10 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r82706401 --- Diff: core/src/main/java/org/apache/spark/io/NioBasedBufferedFileInputStream.java --- @@ -0,0 +1,127 @@ +/* + * Licensed under the Apache Lice

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/15408 >> BufferedInputStream supports mark/reset, while we are not doing so here - does deserialization and other codepaths depend on this ? (If not for functionality, then for performance ?)

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15408 **[Test build #66695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66695/consoleFull)** for PR 15408 at commit [`5eed34f`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15408 **[Test build #66695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66695/consoleFull)** for PR 15408 at commit [`5eed34f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15408 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66695/ Test FAILed. ---

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15408 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82706656 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82706767 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark issue #15408: [SPARK-17839][CORE] Use Nio's directbuffer instead of Bu...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15408 **[Test build #66696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66696/consoleFull)** for PR 15408 at commit [`681ff62`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82706830 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82706923 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82706974 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82707120 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15422 **[Test build #66691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66691/consoleFull)** for PR 15422 at commit [`f810937`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82707181 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15422 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66691/ Test FAILed. ---

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15422 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82707355 --- Diff: python/pyspark/sql/streaming.py --- @@ -189,6 +189,282 @@ def resetTerminated(self): self._jsqm.resetTerminated() +class

[GitHub] spark issue #15295: [SPARK-17720][SQL] introduce static SQL conf

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15295 **[Test build #66697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66697/consoleFull)** for PR 15295 at commit [`8d93c4a`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82707725 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulAggregate.scala --- @@ -56,7 +57,12 @@ case class StateStoreRestoreExec(

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15422 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #15422: [SPARK-17850][Core]HadoopRDD should not catch EOFExcepti...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15422 **[Test build #66698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66698/consoleFull)** for PR 15422 at commit [`f810937`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82708148 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -176,7 +184,9 @@ class StreamExecution( //

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82708289 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -176,7 +184,9 @@ class StreamExecution( //

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82708317 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -176,7 +184,9 @@ class StreamExecution( //

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82708346 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -221,8 +247,15 @@ class StreamExecution(

[GitHub] spark issue #13675: [SPARK-15957] [ML] RFormula supports forcing to index la...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13675 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13675: [SPARK-15957] [ML] RFormula supports forcing to index la...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13675 **[Test build #66693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66693/consoleFull)** for PR 13675 at commit [`3079996`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13675: [SPARK-15957] [ML] RFormula supports forcing to index la...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66693/ Test PASSed. ---

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82708496 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -530,7 +692,7 @@ class StreamExecution( case o

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14690 @mallman I think this will be a vital part of Spark 2.1. There are more work that depends on this one and I want to make sure those will be done. On the Databricks side, @ericl has been testing

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82709228 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamMetrics.scala --- @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82709451 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamMetrics.scala --- @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #15307: [SPARK-17731][SQL][STREAMING] Metrics for structu...

2016-10-10 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15307#discussion_r82709478 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamMetrics.scala --- @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82707347 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala --- @@ -477,6 +478,15 @@ class InMemoryCatalog( cat

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82707709 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82707684 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82708474 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/SessionFileCatalog.scala --- @@ -0,0 +1,120 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82709034 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -619,6 +620,44 @@ private[spark] class HiveExternalCatalog(conf:

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82708580 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitions.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-10 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14690 We should also flag protect this (I am thinking of resolving to a ListingFileCatalog when disabled), definitely until a directory metadata cache is added back. --- If your project is set up for it,

[GitHub] spark pull request #15416: [SPARK-17849] [SQL] Fix NPE problem when using gr...

2016-10-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15416#discussion_r82710433 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -298,10 +298,11 @@ class Analyzer( case other

[GitHub] spark issue #15375: [SPARK-17790][SPARKR] Support for parallelizing R data.f...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66699/consoleFull)** for PR 15375 at commit [`62ab47b`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14959 **[Test build #66700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66700/consoleFull)** for PR 14959 at commit [`d692e71`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-10-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14452 Adding an explicit cache call btw -- can actually slow things down due to bad memory management. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request #15424: [SPARK-17338][SQL][follow-up] add global temp vie...

2016-10-10 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/15424 [SPARK-17338][SQL][follow-up] add global temp view ## What changes were proposed in this pull request? address post hoc review comments for https://github.com/apache/spark/pull/14897

[GitHub] spark issue #15424: [SPARK-17338][SQL][follow-up] add global temp view

2016-10-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15424 cc @rxin @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #15395: [SPARK-17738] [TEST] Fix flaky test in ColumnTypeSuite

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15395 **[Test build #3320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3320/consoleFull)** for PR 15395 at commit [`52b2b7d`](https://github.com/apache/spark/commit

[GitHub] spark issue #15424: [SPARK-17338][SQL][follow-up] add global temp view

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15424 **[Test build #66701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66701/consoleFull)** for PR 15424 at commit [`b93841b`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-10-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 @rxin yeah, as I tried adding explicit cache call doesn't improve it. So I remove it then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82712965 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala --- @@ -477,6 +478,15 @@ class InMemoryCatalog( c

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82713068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15423 **[Test build #66692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66692/consoleFull)** for PR 15423 at commit [`3acd08f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15423 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66692/ Test PASSed. ---

[GitHub] spark issue #15423: [SPARK-17860][SQL] SHOW COLUMN's database conflict check...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15423 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14959 **[Test build #66700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66700/consoleFull)** for PR 14959 at commit [`d692e71`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-10 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r82713318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -225,13 +225,16 @@ case class FileSourceScanExec( }

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14690 yea, making the parquet reader support case-insensitive field resolution is an ultimate solution, but I think dropping the support for reading mixed-case parquet files for catalog tables is also a

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14959 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66700/ Test FAILed. ---

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15421 **[Test build #66702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66702/consoleFull)** for PR 15421 at commit [`9e621eb`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-10 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 @shivaram can I nominate this patch for 2.0 branch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

<    1   2   3   4   5   6   7   8   >