[jira] [Resolved] (SPARK-16400) Remove InSet filter pushdown from Parquet

2016-07-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16400. Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.1.0 Resolved by

[jira] [Commented] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365777#comment-15365777 ] Cheng Lian commented on SPARK-16381: For a specific release, usually, we only make a schedule for the

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365591#comment-15365591 ] Cheng Lian commented on SPARK-16380: [~wm624] Considering 2.0.0 RC2 has already been cut, it's

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365590#comment-15365590 ] Cheng Lian commented on SPARK-16303: [~aokolnychyi] Considering 2.0.0 RC2 has already been cut, it's

[jira] [Commented] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365583#comment-15365583 ] Cheng Lian commented on SPARK-16381: Thanks for volunteering! I've assigned this ticket to you.

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Assignee: Xin Ren > Update SQL examples and programming guide for R language binding >

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365579#comment-15365579 ] Cheng Lian commented on SPARK-16380: I just noticed that I put "Scala" into the JIRA ticket title by

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Summary: Update SQL examples and programming guide for Python language binding (was: Update SQL

[jira] [Resolved] (SPARK-16388) Remove spark.sql.nativeView and spark.sql.nativeView.canonical config

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16388. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14061

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Labels: (was: Starter) > Update SQL examples and programming guide for R language binding >

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Labels: Starter (was: ) > Update SQL examples and programming guide for R language binding >

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Description: Please follow guidelines listed in this SPARK-16303

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364057#comment-15364057 ] Cheng Lian commented on SPARK-16380: I've assigned this ticket to you. > Update SQL examples and

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Assignee: Miao Wang > Update SQL examples and programming guide for Scala Python language binding >

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364055#comment-15364055 ] Cheng Lian commented on SPARK-16380: Thanks for volunteering! Following guidelines listed in

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Description: Please follow guidelines listed in this SPARK-16303

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Description: Please follow guidelines listed > Update SQL examples and programming guide for Scala

[jira] [Resolved] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16330. Resolution: Invalid I'm resolving this issue as invalid since it's actually a spark-avro bug. >

[jira] [Commented] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362678#comment-15362678 ] Cheng Lian commented on SPARK-16330: Please find the root cause analysis in the comment area:

[jira] [Assigned] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-16330: -- Assignee: Cheng Lian > Null pointer getting count from avro file in mesos distributed >

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Assignee: Anton Okolnychyi (was: Cheng Lian) > Update SQL examples and programming guide for Scala

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362625#comment-15362625 ] Cheng Lian commented on SPARK-16303: Just assigned this ticket to you. > Update SQL examples and

[jira] [Created] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

2016-07-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16380: -- Summary: Update SQL examples and programming guide for Scala Python language binding Key: SPARK-16380 URL: https://issues.apache.org/jira/browse/SPARK-16380 Project:

[jira] [Created] (SPARK-16381) Update SQL examples and programming guide for R language binding

2016-07-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16381: -- Summary: Update SQL examples and programming guide for R language binding Key: SPARK-16381 URL: https://issues.apache.org/jira/browse/SPARK-16381 Project: Spark

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Summary: Update SQL examples and programming guide (was: Update SQL examples and programming

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362621#comment-15362621 ] Cheng Lian commented on SPARK-16303: Please feel free to split the task. I'm going to narrow scope of

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Summary: Update SQL examples and programming guide for Scala and Java language bindings (was:

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362579#comment-15362579 ] Cheng Lian commented on SPARK-16303: Here's the aforementioned WIP branch, which only contains

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362575#comment-15362575 ] Cheng Lian commented on SPARK-16303: Sure, thanks for volunteering! Actually, I've started working on

[jira] [Updated] (SPARK-16360) Speed up SQL query performance by removing redundant `executePlan` call in `Dataset`

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16360: --- Assignee: Dongjoon Hyun > Speed up SQL query performance by removing redundant `executePlan` call in

[jira] [Resolved] (SPARK-16360) Speed up SQL query performance by removing redundant `executePlan` call in `Dataset`

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16360. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14044

[jira] [Resolved] (SPARK-15198) Support for filter push down for boolean types in ORC

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15198. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 12972

[jira] [Updated] (SPARK-15198) Support for filter push down for boolean types in ORC

2016-07-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15198: --- Assignee: Hyukjin Kwon > Support for filter push down for boolean types in ORC >

[jira] [Updated] (SPARK-16208) Add `PropagateEmptyRelation` optimizer

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16208: --- Assignee: Dongjoon Hyun (was: Apache Spark) > Add `PropagateEmptyRelation` optimizer >

[jira] [Resolved] (SPARK-16208) Add `PropagateEmptyRelation` optimizer

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16208. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13906

[jira] [Commented] (SPARK-16317) Add file filtering interface for FileFormat

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358975#comment-15358975 ] Cheng Lian commented on SPARK-16317: The motivation is to filter out input data files so that

[jira] [Updated] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16344: --- Description: This is a weird corner case. Users may hit this issue if they have a schema that # has

[jira] [Updated] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16344: --- Description: Array of struct with a single field name "element" can't be decoded from Parquet files

[jira] [Created] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-01 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16344: -- Summary: Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+ Key: SPARK-16344 URL:

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > Add

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Assignee: Weichen Xu > Add Catalog.refreshTable into python API >

[jira] [Resolved] (SPARK-15820) Add Catalog.refreshTable into python API

2016-06-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15820. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13558

[jira] [Created] (SPARK-16317) Add file filtering interface for FileFormat

2016-06-30 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16317: -- Summary: Add file filtering interface for FileFormat Key: SPARK-16317 URL: https://issues.apache.org/jira/browse/SPARK-16317 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-16134) optimizer rules for typed filter

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16134. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13846

[jira] [Created] (SPARK-16303) Update SQL examples and programming guide

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16303: -- Summary: Update SQL examples and programming guide Key: SPARK-16303 URL: https://issues.apache.org/jira/browse/SPARK-16303 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16295: --- Description: Currently, all example snippets in the SQL programming guide are hard-coded, which can

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-16295 > Labelling support for the

[jira] [Created] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16295: -- Summary: Extract SQL programming guide example snippets from source files instead of hard code them Key: SPARK-16295 URL: https://issues.apache.org/jira/browse/SPARK-16295

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll

[jira] [Created] (SPARK-16294) Labelling support for the include_example Jekyll plugin

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16294: -- Summary: Labelling support for the include_example Jekyll plugin Key: SPARK-16294 URL: https://issues.apache.org/jira/browse/SPARK-16294 Project: Spark Issue

[jira] [Created] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

2016-06-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16291: -- Summary: Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis Key: SPARK-16291 URL: https://issues.apache.org/jira/browse/SPARK-16291

[jira] [Resolved] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

2016-06-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16100. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Resolved] (SPARK-16221) Redirect Parquet JUL logger via SLF4J for WRITE operations

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16221. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16221) Redirect Parquet JUL logger via SLF4J for WRITE operations

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16221: --- Assignee: Dongjoon Hyun > Redirect Parquet JUL logger via SLF4J for WRITE operations >

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351244#comment-15351244 ] Cheng Lian commented on SPARK-16164: I'm not saying that we should make this explicit, but our

[jira] [Updated] (SPARK-10591) False negative in QueryTest.checkAnswer

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10591: --- Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > False negative

[jira] [Updated] (SPARK-10591) False negative in QueryTest.checkAnswer

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10591: --- Assignee: Dongjoon Hyun > False negative in QueryTest.checkAnswer >

[jira] [Resolved] (SPARK-10591) False negative in QueryTest.checkAnswer

2016-06-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10591. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13913

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347701#comment-15347701 ] Cheng Lian commented on SPARK-16164: I'm posting a summary of our offline and GitHub discussion about

[jira] [Resolved] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16165. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16165: --- Assignee: Dongjoon Hyun > Fix the update logic for InMemoryTableScanExec.readBatches accumulator >

[jira] [Updated] (SPARK-13709) Spark unable to decode Avro when partitioned

2016-06-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13709: --- Assignee: Cheng Lian > Spark unable to decode Avro when partitioned >

[jira] [Updated] (SPARK-13572) HiveContext reads avro Hive tables incorrectly

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13572: --- Description: I am using PySpark to read avro-based tables from Hive and while the avro tables can

[jira] [Updated] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16100: --- Description: I get a similar error when using complex types in Aggregator. Not sure if this is the

[jira] [Updated] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16100: --- Assignee: Wenchen Fan > Aggregator fails with Tungsten error when complex types are used for results

[jira] [Resolved] (SPARK-16097) Encoders.tuple should handle null object correctly

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16097. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13807

[jira] [Updated] (SPARK-16121) ListingFileCatalog does not list in parallel anymore

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16121: --- Assignee: Yin Huai > ListingFileCatalog does not list in parallel anymore >

[jira] [Resolved] (SPARK-16121) ListingFileCatalog does not list in parallel anymore

2016-06-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16121. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13830

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341754#comment-15341754 ] Cheng Lian commented on SPARK-16032: [~rdblue], I also migrated some test cases from your PR so that

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341658#comment-15341658 ] Cheng Lian commented on SPARK-16032: Hey [~rdblue], [~yhuai] and [~cloud_fan] had already covered a

[jira] [Resolved] (SPARK-15894) Add doc to control #partition for input files

2016-06-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15894. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13797

[jira] [Updated] (SPARK-15894) Add doc to control #partition for input files

2016-06-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15894: --- Assignee: Takeshi Yamamuro > Add doc to control #partition for input files >

[jira] [Resolved] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16030. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13769

[jira] [Updated] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16033: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > DataFrameWriter.partitionBy() can't

[jira] [Created] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16033: -- Summary: DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto() Key: SPARK-16033 URL: https://issues.apache.org/jira/browse/SPARK-16033

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Assignee: Yin Huai > Allow specifying static partitions in an INSERT statement for data source >

[jira] [Created] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16032: -- Summary: Audit semantics of various insertion operations related to partitioned tables Key: SPARK-16032 URL: https://issues.apache.org/jira/browse/SPARK-16032 Project:

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > Allow specifying static partitions in

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Description: A table from SQL server Northwind database was registered as a JDBC dataframe. A query

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Assignee: Hyukjin Kwon > JDBC AND/OR operator push down does not respect lower OR operator

[jira] [Resolved] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15916. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13743

[jira] [Resolved] (SPARK-15862) Better Error Message When Having Database Name in CACHE TABLE AS SELECT

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15862. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13572

[jira] [Resolved] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15786. Resolution: Duplicate Fix Version/s: 2.0.0 > joinWith bytecode generation calling

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15786: --- Assignee: Sean Zhong > joinWith bytecode generation calling ByteBuffer.wrap with InternalRow >

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead()

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Summary: Remove FileFormat.prepareRead() (was: Remove FileFormat.prepareRead) > Remove

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Description: Interface method {{FileFormat.prepareRead()}} was added in [PR

[jira] [Created] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15983: -- Summary: Remove FileFormat.prepareRead Key: SPARK-15983 URL: https://issues.apache.org/jira/browse/SPARK-15983 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15901: --- Assignee: Xiao Li > Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET >

[jira] [Resolved] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15901. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13622

[jira] [Commented] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332560#comment-15332560 ] Cheng Lian commented on SPARK-14953: Marked this as won't fix since this isn't causing any actual

[jira] [Resolved] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14953. Resolution: Won't Fix > LocalBackend should revive offers periodically >

[jira] [Resolved] (SPARK-15929) DataFrameSuite path globbing error message tests are not fully portable

2016-06-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15929. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13649

[jira] [Commented] (SPARK-15931) SparkR tests failing on R 3.3.0

2016-06-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328403#comment-15328403 ] Cheng Lian commented on SPARK-15931: cc [~mengxr] > SparkR tests failing on R 3.3.0 >

[jira] [Created] (SPARK-15931) SparkR tests failing on R 3.3.0

2016-06-13 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15931: -- Summary: SparkR tests failing on R 3.3.0 Key: SPARK-15931 URL: https://issues.apache.org/jira/browse/SPARK-15931 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15925) Replaces registerTempTable with createOrReplaceTempView in SparkR

2016-06-13 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15925: -- Summary: Replaces registerTempTable with createOrReplaceTempView in SparkR Key: SPARK-15925 URL: https://issues.apache.org/jira/browse/SPARK-15925 Project: Spark

[jira] [Reopened] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reopened SPARK-15639: We've decided to revert the merged PR, so reopening it. > Try to push down filter at RowGroups level

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15639: --- Assignee: Liang-Chi Hsieh > Try to push down filter at RowGroups level for parquet reader >

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15639: --- Affects Version/s: 2.0.0 > Try to push down filter at RowGroups level for parquet reader >

[jira] [Resolved] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15639. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13371

<    1   2   3   4   5   6   7   8   9   10   >