[GitHub] spark issue #13822: [SPARK-16115][SQL] Change output schema to be partition ...

2016-09-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13822 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #11293: [SPARK-13080] [SQL] Implement new Catalog API usi...

2016-09-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11293#discussion_r79246548 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -127,33 +166,30 @@ abstract class Catalog

[GitHub] spark issue #13896: [SPARK-16193] [TESTS] Address flaky ExternalAppendOnlyMa...

2016-06-24 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13896 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13899: [SPARK-16196][SQL] Codegen in-memory scan with ColumnarB...

2016-06-24 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13899 @rxin @sameeragarwal @ooq --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13899: [SPARK-16196][SQL] Codegen caching + store rows a...

2016-06-24 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/13899 [SPARK-16196][SQL] Codegen caching + store rows as ColumnarBatches ## What changes were proposed in this pull request? This patch makes `InMemoryRelation` faster by generating code

[GitHub] spark issue #9124: [SPARK-11078] Ensure spilling tests actually spill

2016-06-24 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/9124 I see, that's possible. The right thing to do here is to add a `wait` of some sort in the listener to block until we have received the stage completed event. We've done this in some other test

[GitHub] spark issue #13814: [SPARK-16003] SerializationDebugger runs into infinite l...

2016-06-22 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13814 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13742: [SPARK-16023][SQL] Move InMemoryRelation to its own file

2016-06-17 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13742 I didn't --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13742: [SPARK-16023][SQL] Move InMemoryRelation to its o...

2016-06-17 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/13742 [SPARK-16023][SQL] Move InMemoryRelation to its own file ## What changes were proposed in this pull request? Improve readability of `InMemoryTableScanExec.scala`, which has too much

[GitHub] spark issue #13719: [SPARK-15966][DOC] Add closing tag to fix rendering issu...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13719 Actually merged into master 2.0. @dhruve can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #13140: [SPARK-15230] [SQL] distinct() does not handle co...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13140#discussion_r67429987 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1769,7 +1769,10 @@ class Dataset[T] private[sql]( * @since 2.0.0

[GitHub] spark issue #13432: [CORE][DOC][MINOR] Remove incorrect scaladoc

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13432 @jaceklaskowski Can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13458: [SPARK-15717][GraphX] Cannot perform RDD operations on a...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13458 ok to test cc @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13482: [SPARK-15725][YARN] Ensure ApplicationMaster slee...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13482#discussion_r67429393 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -462,10 +464,23 @@ private[spark] class ApplicationMaster

[GitHub] spark pull request #13482: [SPARK-15725][YARN] Ensure ApplicationMaster slee...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13482#discussion_r67429072 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -462,10 +464,23 @@ private[spark] class ApplicationMaster

[GitHub] spark issue #13482: [SPARK-15725][YARN] Ensure ApplicationMaster sleeps for ...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13482 @rdblue can you address the comments? I would like to get this into the 2.0 rc1 if possible. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #13492: [SPARK-15749][SQL]make the error message more meaningful

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13492 Merging into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13654: [SPARK-15868] [Web UI] Executors table in Executors tab ...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13654 Merging into master 2.0 thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13674: [MINOR][DOCS][SQL] Fix some comments about types(TypeCoe...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13674 LGTM merging into maser 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13709: [SPARK-15782] [YARN] Fix spark.jars and spark.yar...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13709#discussion_r67426717 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2352,6 +2352,26 @@ private[spark] object Utils extends Logging { log.info

[GitHub] spark issue #13716: [SPARK-15998] [SQL] Verification of SQLConf HIVE_METASTO...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13716 LGTM merging into master 2.0 thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13710: [SQL] Minor HashAggregateExec string output fixes

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13710 Merging into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13692: [SPARK-15975] Fix improper Popen retcode code handling i...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13692 also into 1.6 and 1.5 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13692: [SPARK-15975] Fix improper Popen retcode code handling i...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13692 Test failure is clearly unrelated. Merging into m2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13695: [SPARK-15978] [SQL] improve 'show tables' command relate...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13695 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13695: [SPARK-15978] [SQL] improve 'show tables' command relate...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13695 Merging into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13692: [SPARK-15975] Fix improper Popen retcode code handling i...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13692 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13711: [SPARK-15991] SparkContext.hadoopConfiguration should be...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13711 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13710: [SQL] Minor HashAggregateExec string output fixes

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13710 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13711: [SPARK-15991] SparkContext.hadoopConfiguration sh...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13711#discussion_r67425283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -43,23 +43,17 @@ private[sql] class SharedState(val sparkContext

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13620 also cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13697: [SPARK-15977][SQL] Fix TRUNCATE TABLE for Spark specific...

2016-06-16 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13697 new changes also LGTM. It's weird that we just implicitly assume the location must be `None` for data source tables without any comments there, but that's a separate issue. --- If your project

[GitHub] spark issue #13697: [SPARK-15977][SQL] Fix TRUNCATE TABLE for Spark specific...

2016-06-15 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13697 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13689: [SPARK-12492][SQL]Add missing SQLExecution.withNewExecut...

2016-06-15 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13689 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13683: [SPARK-15518][Core][Follow-up] Rename LocalSchedulerBack...

2016-06-15 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13683 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13654: [SPARK-15868] [Web UI] Executors table in Executors tab ...

2016-06-14 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13654 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13671: [SPARK-15952] [SQL] fix "show databases" ordering issue

2016-06-14 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13671 ah I see, that's fine then. LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13671: [SPARK-15952] [SQL] fix "show databases" ordering issue

2016-06-14 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13671 Looks good. Can you add unit tests for each of the cases you mentioned? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13618: [SPARK-15796] [CORE] Reduce spark.memory.fraction defaul...

2016-06-14 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13618 cc @davies @JoshRosen for sanity check --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-14 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r67028036 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark pull request #13115: [SPARK-12492] Using spark-sql commond to run quer...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13115#discussion_r66886807 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -110,24 +110,29 @@ class QueryExecution(val sparkSession

[GitHub] spark issue #13492: [SPARK-15749][SQL]make the error message more meaningful

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13492 Looks good. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13558 Looks good. Jenks retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885556 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885396 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13620 @nblintao this looks pretty good. There were a few methods that don't seem to be used anywhere. Am I missing something? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885448 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885512 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13620 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13561: [SPARK-15824][SQL] Run 'with ... insert ... select' fail...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13561 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13637: [SPARK-15914][SQL] Add deprecated method back to SQLCont...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13637 This looks good, just minor commnts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884192 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884075 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884061 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession

[GitHub] spark issue #13618: [SPARK-15796] [CORE] Reduce spark.memory.fraction defaul...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13618 This looks OK but somewhat arbitrary. This whole thing depends a lot on the data structures the user allocates outside of Spark so no matter how low we make it there will be some applications

[GitHub] spark issue #13596: [SPARK-15870][SQL] DataFrame can't execute after uncache...

2016-06-10 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13596 @rxin @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13415: [SPARK-15676] [SQL] Disallow Column Names as Partition C...

2016-06-10 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13415 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13189: [SPARK-14670][SQL][WIP] allow updating driver side sql m...

2016-06-06 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13189 Also cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13482: SPARK-15725: Ensure ApplicationMaster sleeps for the min...

2016-06-06 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13482 I think this is important to fix for 2.0 but I personally found the changes in this patch rather confusing. If there's a simpler workaround we could do (such as the solution I suggested

[GitHub] spark issue #13482: SPARK-15725: Ensure ApplicationMaster sleeps for the min...

2016-06-06 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13482 @rdblue the reason for the hang is the `GetExecutorLossReason` right? AFAIK we send one to the AM every time an executor dies. What if we just keep a set of executor IDs we're waiting to kill

[GitHub] spark pull request #13482: SPARK-15725: Ensure ApplicationMaster sleeps for ...

2016-06-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13482#discussion_r65940450 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -462,10 +464,23 @@ private[spark] class ApplicationMaster

[GitHub] spark pull request #13482: SPARK-15725: Ensure ApplicationMaster sleeps for ...

2016-06-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13482#discussion_r65940352 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -462,10 +464,23 @@ private[spark] class ApplicationMaster

[GitHub] spark issue #13484: [SPARK-15742][SQL] Reduce temp collections allocations i...

2016-06-03 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13484 LGTM2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13490: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-03 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13490 Sounds good. I've updated the patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13490: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13490 @cloud-fan @clockfly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13490: [SPARK-15722][SQL] Disallow specifying schema in ...

2016-06-02 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/13490 [SPARK-15722][SQL] Disallow specifying schema in CTAS statement ## What changes were proposed in this pull request? As of this patch, this leads to an exception because the schemas may

[GitHub] spark issue #13485: [SPARK-15732][SQL] better error message when use java re...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13485 LGTM2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13485: [SPARK-15732][SQL] better error message when use ...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13485#discussion_r65644242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -582,6 +582,11 @@ object ScalaReflection extends

[GitHub] spark issue #13464: [Minor] Fix Java Lint errors introduced by #13286 and 13...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13464 I believe it is intended to be upper case to match scala; it's just that for java it must be a method so that's failing the style. @srowen is there anyway to turn off the lint checks for a part

[GitHub] spark issue #13479: [SPARK-15736][CORE][branch-1.6] Gracefully handle loss o...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13479 can you delete branch --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13479: [SPARK-15736][CORE][branch-1.6] Gracefully handle loss o...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13479 Merging into 1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13453: [SPARK-15715][SQL] Fix alter partition with storage info...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13453 Merging into master 2.0 thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13473: [SPARK-15736][CORE] Gracefully handle loss of DiskStore ...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13473 Merging into master 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13478: [SPARK-15740] [MLLIB] ignore big model load / save in Wo...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13478 Merging into master 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13452: [SPARK-15718][SQL] better error message for writing buck...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13452 This looks OK. Merging into master 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13457: [SPARK-15722][SQL][WIP] Disallow specifying schem...

2016-06-02 Thread andrewor14
Github user andrewor14 closed the pull request at: https://github.com/apache/spark/pull/13457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13473: [SPARK-15736][CORE] Gracefully handle loss of DiskStore ...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13473 LGTM. Have you double checked that these are all the places where we do this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #13464: [Minor] Fix Java Lint errors introduced by #13286 and 13...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13464 This is not the right fix. Please close this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13451: [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS S...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13451 Merging into amster 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13451: [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS S...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13451 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13415: [SPARK-15676] [SQL] Disallow Column Names as Partition C...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13415 We should still do it in the parser, but use the `SQLConf` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13415: [SPARK-15676] [SQL] Disallow Column Names as Part...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13415#discussion_r65617955 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -880,6 +880,23 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark pull request #13415: [SPARK-15676] [SQL] Disallow Column Names as Part...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13415#discussion_r65617886 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -880,6 +880,23 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark pull request #13415: [SPARK-15676] [SQL] Disallow Column Names as Part...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13415#discussion_r65617809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -880,6 +880,23 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark pull request #13453: [SPARK-15715][SQL] Fix alter partition with stora...

2016-06-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13453#discussion_r65612716 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -779,18 +780,29 @@ private[hive] class HiveClientImpl

[GitHub] spark issue #13457: [SPARK-15722][SQL][WIP] Disallow specifying schema in CT...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13457 (actually, this doesn't work yet! Please don't review) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13457: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13457 @cloud-fan @clockfly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #13457: [SPARK-15722][SQL] Disallow specifying schema in ...

2016-06-01 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/13457 [SPARK-15722][SQL] Disallow specifying schema in CTAS statement ## What changes were proposed in this pull request? This is no longer allowed: ``` sql("CREATE TABLE blocks

[GitHub] spark issue #13452: [SPARK-15718][SQL] better error message for writing buck...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13452 @cloud-fan actually since this is only used in 2 places maybe you can just inline it. It's probably not worth the trouble to traverse the call stack to find the method name. --- If your

[GitHub] spark issue #13434: [SPARK-15322][SQL][FOLLOWUP] Use the new long accumulato...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13434 LGTM2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13386: [SPARK-15646] [SQL] When spark.sql.hive.convertCTAS is t...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13386 Merging into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13452: [SPARK-15718][SQL] better error message for writi...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13452#discussion_r65469251 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -502,8 +503,18 @@ final class DataFrameWriter private[sql](df

[GitHub] spark pull request #13452: [SPARK-15718][SQL] better error message for writi...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13452#discussion_r65453530 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -500,10 +500,10 @@ final class DataFrameWriter private[sql](df

[GitHub] spark pull request #13453: [SPARK-15715][SQL] Fix alter partition with stora...

2016-06-01 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/13453 [SPARK-15715][SQL] Fix alter partition with storage information in Hive ## What changes were proposed in this pull request? This command didn't work. Now it does: ``` ALTER

[GitHub] spark issue #13451: [SPARK-15711][SQL][WIP]Ban CREATE TEMPORARY TABLE USING ...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13451 Looks good. Can you add a test for the new exception that you throw? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13441: [SPARK-15702][Documentation]Update document programming-...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13441 @cloud-fan @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13432: [CORE][DOC][MINOR] Remove incorrect scaladoc

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13432 Also I'm not sure if this is incorrect. Even though it's `@DeveloperApi` it's still mainly for internal use and we don't guarantee backward compatibility with these APIs. --- If your project

[GitHub] spark issue #13427: [Minor] Slightly better error message when attempting to...

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13427 Sorry I usually do but forgot for this one. This was merged into master 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #13416: [SPARK-15596][SPARK-15635][SQL] ALTER TABLE RENAME fixes

2016-06-01 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13416 @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request: [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults prope...

2016-05-31 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/13088 LGTM thanks for working on this. Merging into master 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

<    1   2   3   4   5   6   7   8   9   10   >