[jira] [Updated] (SPARK-11164) Add InSet pushdown filter back for Parquet

[ https://issues.apache.org/jira/browse/SPARK-11164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11164: --- Assignee: Xiao Li > Add InSet pushdown filter back for Parquet >

[jira] [Updated] (SPARK-12478) Dataset fields of product types can't be null

[ https://issues.apache.org/jira/browse/SPARK-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12478: --- Summary: Dataset fields of product types can't be null (was: Dataset fields whose types are case

[jira] [Updated] (SPARK-12478) Dataset fields whose types are case classs can't be null

[ https://issues.apache.org/jira/browse/SPARK-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12478: --- Summary: Dataset fields whose types are case classs can't be null (was: Top level case class field

[jira] [Created] (SPARK-12478) Top level case class field of a Dataset can't be null

Cheng Lian created SPARK-12478: -- Summary: Top level case class field of a Dataset can't be null Key: SPARK-12478 URL: https://issues.apache.org/jira/browse/SPARK-12478 Project: Spark Issue

[jira] [Closed] (SPARK-12444) A lightweight Scala DSL for schema construction

[ https://issues.apache.org/jira/browse/SPARK-12444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-12444. -- Resolution: Won't Fix > A lightweight Scala DSL for schema construction >

[jira] [Resolved] (SPARK-12371) Make sure non-nullable arguments of NewInstance don't receive null input data

[ https://issues.apache.org/jira/browse/SPARK-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12371. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10331

[jira] [Commented] (SPARK-12478) Dataset fields of product types can't be null

[ https://issues.apache.org/jira/browse/SPARK-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068043#comment-15068043 ] Cheng Lian commented on SPARK-12478: [~marmbrus] I guess this issue probably doesn't block 1.6 since

[jira] [Resolved] (SPARK-9271) Concurrency bug triggered by partition predicate push-down

[ https://issues.apache.org/jira/browse/SPARK-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9271. --- Resolution: Cannot Reproduce > Concurrency bug triggered by partition predicate push-down >

[jira] [Commented] (SPARK-9271) Concurrency bug triggered by partition predicate push-down

[ https://issues.apache.org/jira/browse/SPARK-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068434#comment-15068434 ] Cheng Lian commented on SPARK-9271: --- I only observed this issue in the context of PR #7492. Since that

[jira] [Updated] (SPARK-12371) Make sure non-nullable arguments of NewInstance don't receive null input data

[ https://issues.apache.org/jira/browse/SPARK-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12371: --- Description: When extracting objects from underlying rows using a Dataset, it's possible that a

[jira] [Updated] (SPARK-12371) Make sure non-nullable arguments of NewInstance don't receive null input data

[ https://issues.apache.org/jira/browse/SPARK-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12371: --- Summary: Make sure non-nullable arguments of NewInstance don't receive null input data (was: Make

[jira] [Created] (SPARK-12444) A lightweight Scala DSL for schema construction

Cheng Lian created SPARK-12444: -- Summary: A lightweight Scala DSL for schema construction Key: SPARK-12444 URL: https://issues.apache.org/jira/browse/SPARK-12444 Project: Spark Issue Type:

[jira] [Updated] (SPARK-12369) DataFrameReader fails on globbing parquet paths that contain nonexistent path(s)

[ https://issues.apache.org/jira/browse/SPARK-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12369: --- Summary: DataFrameReader fails on globbing parquet paths that contain nonexistent path(s) (was:

[jira] [Updated] (SPARK-12369) DataFrameReader fails on globbing parquet paths that contain nonexistent path(s)

[ https://issues.apache.org/jira/browse/SPARK-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12369: --- Shepherd: Cheng Lian > DataFrameReader fails on globbing parquet paths that contain nonexistent >

[jira] [Commented] (SPARK-11941) JSON representation of nested StructTypes could be more uniform

[ https://issues.apache.org/jira/browse/SPARK-11941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065334#comment-15065334 ] Cheng Lian commented on SPARK-11941: One thing is that, the JSON representation of a DataFrame schema

[jira] [Commented] (SPARK-11148) Unable to create views

[ https://issues.apache.org/jira/browse/SPARK-11148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063721#comment-15063721 ] Cheng Lian commented on SPARK-11148: Did you mean the Windows ODBC driver provided by Simba? AFAIK

[jira] [Created] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

Cheng Lian created SPARK-12406: -- Summary: Simple join query throws exception that complains generated classes are not found Key: SPARK-12406 URL: https://issues.apache.org/jira/browse/SPARK-12406

[jira] [Updated] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Description: This should be a regression of [PR #9923|https://github.com/apache/spark/pull/9923].

[jira] [Commented] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061994#comment-15061994 ] Cheng Lian commented on SPARK-12406: cc [~vanzin] > Simple join query throws exception that

[jira] [Updated] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Description: This should be a regression of [PR #9923|https://github.com/apache/spark/pull/9923].

[jira] [Updated] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Description: This should be a regression of [PR #9923|https://github.com/apache/spark/pull/9923].

[jira] [Updated] (SPARK-12406) Codegen'd classes can't be found under REPL

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Summary: Codegen'd classes can't be found under REPL (was: Simple join query throws exception that

[jira] [Updated] (SPARK-12406) Simple join query throws exception that complains generated classes are not found

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Priority: Blocker (was: Major) > Simple join query throws exception that complains generated

[jira] [Updated] (SPARK-12406) Codegen'd classes can't be found under REPL

[ https://issues.apache.org/jira/browse/SPARK-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12406: --- Description: This should be a regression of [PR #9923|https://github.com/apache/spark/pull/9923].

[jira] [Created] (SPARK-12371) Make sure Dataset nullability conforms to its underlying logical plan

Cheng Lian created SPARK-12371: -- Summary: Make sure Dataset nullability conforms to its underlying logical plan Key: SPARK-12371 URL: https://issues.apache.org/jira/browse/SPARK-12371 Project: Spark

[jira] [Created] (SPARK-12336) Outer join using multiple columns results in wrong nullability

Cheng Lian created SPARK-12336: -- Summary: Outer join using multiple columns results in wrong nullability Key: SPARK-12336 URL: https://issues.apache.org/jira/browse/SPARK-12336 Project: Spark

[jira] [Created] (SPARK-12335) CentralMomentAgg should be nullable

Cheng Lian created SPARK-12335: -- Summary: CentralMomentAgg should be nullable Key: SPARK-12335 URL: https://issues.apache.org/jira/browse/SPARK-12335 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-12335) CentralMomentAgg should be nullable

[ https://issues.apache.org/jira/browse/SPARK-12335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12335: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-12323 > CentralMomentAgg should be nullable >

[jira] [Updated] (SPARK-12336) Outer join using multiple columns results in wrong nullability

[ https://issues.apache.org/jira/browse/SPARK-12336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12336: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-12323 > Outer join using multiple columns

[jira] [Created] (SPARK-12342) Corr (Pearson correlation) should be nullable

Cheng Lian created SPARK-12342: -- Summary: Corr (Pearson correlation) should be nullable Key: SPARK-12342 URL: https://issues.apache.org/jira/browse/SPARK-12342 Project: Spark Issue Type:

[jira] [Created] (SPARK-12341) The "comment" field of DESCRIBE result set should be nullable

Cheng Lian created SPARK-12341: -- Summary: The "comment" field of DESCRIBE result set should be nullable Key: SPARK-12341 URL: https://issues.apache.org/jira/browse/SPARK-12341 Project: Spark

[jira] [Created] (SPARK-12323) Don't assign default value for non-nullable columns of a Dataset

Cheng Lian created SPARK-12323: -- Summary: Don't assign default value for non-nullable columns of a Dataset Key: SPARK-12323 URL: https://issues.apache.org/jira/browse/SPARK-12323 Project: Spark

[jira] [Updated] (SPARK-10364) Support Parquet logical type TIMESTAMP_MILLIS

[ https://issues.apache.org/jira/browse/SPARK-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10364: --- Description: The {{TimestampType}} in Spark SQL is of microsecond precision. Ideally, we should

[jira] [Resolved] (SPARK-10366) Support Parquet logical type DATE

[ https://issues.apache.org/jira/browse/SPARK-10366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10366. Resolution: Fixed Actually this has already been implemented since at least 1.4. > Support

[jira] [Resolved] (SPARK-12131) Cannot create ExpressionEncoder for Array[T] where T is a nested class

[ https://issues.apache.org/jira/browse/SPARK-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12131. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by

[jira] [Resolved] (SPARK-12252) refactor MapObjects to make it less hacky

[ https://issues.apache.org/jira/browse/SPARK-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12252. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10239

[jira] [Resolved] (SPARK-11676) Parquet filter tests all pass if filters are not really pushed down

[ https://issues.apache.org/jira/browse/SPARK-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11676. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9659

[jira] [Created] (SPARK-12131) Cannot create ExpressionEncoder for Array[T] where T is a nested class

Cheng Lian created SPARK-12131: -- Summary: Cannot create ExpressionEncoder for Array[T] where T is a nested class Key: SPARK-12131 URL: https://issues.apache.org/jira/browse/SPARK-12131 Project: Spark

[jira] [Created] (SPARK-12094) Better format for query plan tree string

Cheng Lian created SPARK-12094: -- Summary: Better format for query plan tree string Key: SPARK-12094 URL: https://issues.apache.org/jira/browse/SPARK-12094 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-12094) Better format for query plan tree string

[ https://issues.apache.org/jira/browse/SPARK-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12094: --- Description: When examine plans of complex queries with multiple joins, a pain point of mine is

[jira] [Created] (SPARK-12046) Visibility and format issues in ScalaDoc/JavaDoc for branch-1.6

Cheng Lian created SPARK-12046: -- Summary: Visibility and format issues in ScalaDoc/JavaDoc for branch-1.6 Key: SPARK-12046 URL: https://issues.apache.org/jira/browse/SPARK-12046 Project: Spark

[jira] [Created] (SPARK-12047) Unhelpful error messages generated by JavaDoc while doing sbt unidoc

Cheng Lian created SPARK-12047: -- Summary: Unhelpful error messages generated by JavaDoc while doing sbt unidoc Key: SPARK-12047 URL: https://issues.apache.org/jira/browse/SPARK-12047 Project: Spark

[jira] [Commented] (SPARK-9182) filter and groupBy on DataFrames are not passed through to jdbc source

[ https://issues.apache.org/jira/browse/SPARK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15031526#comment-15031526 ] Cheng Lian commented on SPARK-9182: --- Converting all {{expressions.Filter}} to {{sources.Filter}}

[jira] [Commented] (SPARK-12000) `sbt publishLocal` hits a Scala compiler bug caused by `Since` annotation

[ https://issues.apache.org/jira/browse/SPARK-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15030453#comment-15030453 ] Cheng Lian commented on SPARK-12000: Yeah, also found that the error message IS harmful. By switching

[jira] [Commented] (SPARK-12000) `sbt publishLocal` hits a Scala compiler bug caused by `Since` annotation

[ https://issues.apache.org/jira/browse/SPARK-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15030428#comment-15030428 ] Cheng Lian commented on SPARK-12000: Hit exactly the same issue and worked it around by switching to

[jira] [Created] (SPARK-12012) Show more comprehensive PhysicalRDD metadata when visualizing SQL query plan

Cheng Lian created SPARK-12012: -- Summary: Show more comprehensive PhysicalRDD metadata when visualizing SQL query plan Key: SPARK-12012 URL: https://issues.apache.org/jira/browse/SPARK-12012 Project:

[jira] [Updated] (SPARK-12012) Show more comprehensive PhysicalRDD metadata when visualizing SQL query plan

[ https://issues.apache.org/jira/browse/SPARK-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12012: --- Description: Currently a {{PhysicalRDD}} operator is just visualized as a node with nothing but a

[jira] [Resolved] (SPARK-11043) Hive Thrift Server will log warn "Couldn't find log associated with operation handle"

[ https://issues.apache.org/jira/browse/SPARK-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11043. Resolution: Fixed Fix Version/s: 1.6.0 1.7.0 Issue resolved by pull

[jira] [Updated] (SPARK-11043) Hive Thrift Server will log warn "Couldn't find log associated with operation handle"

[ https://issues.apache.org/jira/browse/SPARK-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11043: --- Assignee: SaintBacchus > Hive Thrift Server will log warn "Couldn't find log associated with

[jira] [Updated] (SPARK-9141) DataFrame recomputed instead of using cached parent.

[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9141: -- Description: As I understand, DataFrame.cache() is supposed to work the same as RDD.cache(), so that

[jira] [Commented] (SPARK-10367) Support Parquet logical type INTERVAL

[ https://issues.apache.org/jira/browse/SPARK-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013199#comment-15013199 ] Cheng Lian commented on SPARK-10367: Dropped. > Support Parquet logical type INTERVAL >

[jira] [Updated] (SPARK-10367) Support Parquet logical type INTERVAL

[ https://issues.apache.org/jira/browse/SPARK-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10367: --- Target Version/s: (was: 1.6.0) > Support Parquet logical type INTERVAL >

[jira] [Updated] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9686: -- Priority: Blocker (was: Major) > Spark Thrift server doesn't return correct JDBC metadata >

[jira] [Updated] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11785: --- Priority: Blocker (was: Major) > When deployed against remote Hive metastore with lower versions,

[jira] [Assigned] (SPARK-11783) When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore

[ https://issues.apache.org/jira/browse/SPARK-11783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-11783: -- Assignee: Cheng Lian > When deployed against remote Hive metastore, HiveContext.executionHive

[jira] [Updated] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11785: --- Assignee: Cheng Lian > When deployed against remote Hive metastore with lower versions, JDBC >

[jira] [Commented] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15010792#comment-15010792 ] Cheng Lian commented on SPARK-9686: --- [~navis] Thanks for the information! I'll try you patch shortly. >

[jira] [Updated] (SPARK-11694) Parquet logical types are not being tested properly

[ https://issues.apache.org/jira/browse/SPARK-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11694: --- Fix Version/s: 1.7.0 > Parquet logical types are not being tested properly >

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008827#comment-15008827 ] Cheng Lian commented on SPARK-11153: You are right that Impala and Hive write timestamps as Parquet

[jira] [Created] (SPARK-11783) When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore

Cheng Lian created SPARK-11783: -- Summary: When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore Key: SPARK-11783 URL: https://issues.apache.org/jira/browse/SPARK-11783

[jira] [Commented] (SPARK-9686) Spark hive jdbc client cannot get table from metadata store

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009033#comment-15009033 ] Cheng Lian commented on SPARK-9686: --- Tested 1.7-SNAPSHOT

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009057#comment-15009057 ] Cheng Lian commented on SPARK-11153: Yes please file a ticket. One complication here is that,

[jira] [Commented] (SPARK-11783) When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore

[ https://issues.apache.org/jira/browse/SPARK-11783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009098#comment-15009098 ] Cheng Lian commented on SPARK-11783: This

[jira] [Updated] (SPARK-6776) Implement backwards-compatibility rules in Catalyst converters (which convert Parquet record to rows)

[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6776: -- Fix Version/s: 1.5.0 > Implement backwards-compatibility rules in Catalyst converters (which convert >

[jira] [Commented] (SPARK-6776) Implement backwards-compatibility rules in Catalyst converters (which convert Parquet record to rows)

[ https://issues.apache.org/jira/browse/SPARK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15009181#comment-15009181 ] Cheng Lian commented on SPARK-6776: --- It's 1.5.0, just added. > Implement backwards-compatibility rules

[jira] [Created] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

Cheng Lian created SPARK-11785: -- Summary: When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception Key: SPARK-11785 URL: https://issues.apache.org/jira/browse/SPARK-11785

[jira] [Resolved] (SPARK-11044) Parquet writer version fixed as version1

[ https://issues.apache.org/jira/browse/SPARK-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11044. Resolution: Fixed Fix Version/s: 1.7.0 Issue resolved by pull request 9060

[jira] [Resolved] (SPARK-11692) Support for Parquet logical types, JSON and BSON (embedded types)

[ https://issues.apache.org/jira/browse/SPARK-11692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11692. Resolution: Fixed Fix Version/s: 1.7.0 Issue resolved by pull request 9658

[jira] [Updated] (SPARK-11044) Parquet writer version fixed as version1

[ https://issues.apache.org/jira/browse/SPARK-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11044: --- Assignee: Hyukjin Kwon > Parquet writer version fixed as version1 >

[jira] [Updated] (SPARK-11692) Support for Parquet logical types, JSON and BSON (embedded types)

[ https://issues.apache.org/jira/browse/SPARK-11692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11692: --- Assignee: Hyukjin Kwon > Support for Parquet logical types, JSON and BSON (embedded types) >

[jira] [Updated] (SPARK-11044) Parquet writer version fixed as version1

[ https://issues.apache.org/jira/browse/SPARK-11044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11044: --- Fix Version/s: 1.6.0 > Parquet writer version fixed as version1 >

[jira] [Comment Edited] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005023#comment-15005023 ] Cheng Lian edited comment on SPARK-11153 at 11/15/15 11:43 AM: --- Good

[jira] [Resolved] (SPARK-11694) Parquet logical types are not being tested properly

[ https://issues.apache.org/jira/browse/SPARK-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11694. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9660

[jira] [Updated] (SPARK-11694) Parquet logical types are not being tested properly

[ https://issues.apache.org/jira/browse/SPARK-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11694: --- Assignee: Hyukjin Kwon > Parquet logical types are not being tested properly >

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005004#comment-15005004 ] Cheng Lian commented on SPARK-11153: Yes. > Turns off Parquet filter push-down for string and binary

[jira] [Issue Comment Deleted] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11153: --- Comment: was deleted (was: Yes.) > Turns off Parquet filter push-down for string and binary columns

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005005#comment-15005005 ] Cheng Lian commented on SPARK-11153: Yes. > Turns off Parquet filter push-down for string and binary

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005023#comment-15005023 ] Cheng Lian commented on SPARK-11153: Good question. We tried, see [PR

[jira] [Resolved] (SPARK-11678) Partition discovery fail if there is a _SUCCESS file in the table's root dir

[ https://issues.apache.org/jira/browse/SPARK-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11678. Resolution: Fixed Fix Version/s: 1.6.0 1.7.0 Issue resolved by pull

[jira] [Commented] (SPARK-5968) Parquet warning in spark-shell

[ https://issues.apache.org/jira/browse/SPARK-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001885#comment-15001885 ] Cheng Lian commented on SPARK-5968: --- As explained in the JIRA description, this issue shouldn't affect

[jira] [Resolved] (SPARK-11661) We should still pushdown filters returned by a data source's unhandledFilters

[ https://issues.apache.org/jira/browse/SPARK-11661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11661. Resolution: Fixed Fix Version/s: 1.6.0 1.7.0 Issue resolved by pull

[jira] [Commented] (SPARK-11191) [1.5] Can't create UDF's using hive thrift service

[ https://issues.apache.org/jira/browse/SPARK-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003549#comment-15003549 ] Cheng Lian commented on SPARK-11191: Sorry that I wasn't clear enough in my previous reply. So in

[jira] [Commented] (SPARK-11191) [1.5] Can't create UDF's using hive thrift service

[ https://issues.apache.org/jira/browse/SPARK-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002047#comment-15002047 ] Cheng Lian commented on SPARK-11191: This issue consists of two bugs. One of them is the ADD JAR

[jira] [Commented] (SPARK-11191) [1.5] Can't create UDF's using hive thrift service

[ https://issues.apache.org/jira/browse/SPARK-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003578#comment-15003578 ] Cheng Lian commented on SPARK-11191: What error message/exception stacktrace did you get when working

[jira] [Commented] (SPARK-11191) [1.5] Can't create UDF's using hive thrift service

[ https://issues.apache.org/jira/browse/SPARK-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002385#comment-15002385 ] Cheng Lian commented on SPARK-11191: Spark SQL hasn't supported persisted functions yet. > [1.5]

[jira] [Resolved] (SPARK-11500) Not deterministic order of columns when using merging schemas.

[ https://issues.apache.org/jira/browse/SPARK-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11500. Resolution: Fixed Fix Version/s: 1.7.0 Issue resolved by pull request 9517

[jira] [Commented] (SPARK-10113) Support for unsigned Parquet logical types

[ https://issues.apache.org/jira/browse/SPARK-10113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000203#comment-15000203 ] Cheng Lian commented on SPARK-10113: I think emitting a clear error message is more reasonable since

[jira] [Commented] (SPARK-11089) Add a option for thrift-server to share a single session across all connections

[ https://issues.apache.org/jira/browse/SPARK-11089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000195#comment-15000195 ] Cheng Lian commented on SPARK-11089: OK, I'm taking this. > Add a option for thrift-server to share

[jira] [Commented] (SPARK-9686) Spark hive jdbc client cannot get table from metadata store

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000215#comment-15000215 ] Cheng Lian commented on SPARK-9686: --- [~navis] [~bugg_tb] [~pin_zhang] May I ask were you all using

[jira] [Updated] (SPARK-11500) Not deterministic order of columns when using merging schemas.

[ https://issues.apache.org/jira/browse/SPARK-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11500: --- Fix Version/s: 1.6.0 > Not deterministic order of columns when using merging schemas. >

[jira] [Commented] (SPARK-5968) Parquet warning in spark-shell

[ https://issues.apache.org/jira/browse/SPARK-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000200#comment-15000200 ] Cheng Lian commented on SPARK-5968: --- It had once been fixed via a quite hacky trick. Unfortunately it

[jira] [Commented] (SPARK-10954) Parquet version in the "created_by" metadata field of Parquet files written by Spark 1.5 and 1.6 is wrong

[ https://issues.apache.org/jira/browse/SPARK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000222#comment-15000222 ] Cheng Lian commented on SPARK-10954: Figured out the reason why {{created_by}} is wrong in Spark

[jira] [Commented] (SPARK-11191) [1.5] Can't create UDF's using hive thrift service

[ https://issues.apache.org/jira/browse/SPARK-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996519#comment-14996519 ] Cheng Lian commented on SPARK-11191: One of the problem here is SPARK-11595. However, after fixing

[jira] [Created] (SPARK-11595) "ADD JAR" doesn't work if the given path contains URL scheme like "file:/" and "hdfs:/"

Cheng Lian created SPARK-11595: -- Summary: "ADD JAR" doesn't work if the given path contains URL scheme like "file:/" and "hdfs:/" Key: SPARK-11595 URL: https://issues.apache.org/jira/browse/SPARK-11595

[jira] [Updated] (SPARK-11595) "ADD JAR" doesn't work if the given path contains URL scheme like "file:/" and "hdfs:/"

[ https://issues.apache.org/jira/browse/SPARK-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11595: --- Description: When handling {{ADD JAR}}, Spark constructs a {{java.io.File}} first using the input

[jira] [Commented] (SPARK-11500) Not deterministic order of columns when using merging schemas.

[ https://issues.apache.org/jira/browse/SPARK-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992834#comment-14992834 ] Cheng Lian commented on SPARK-11500: [~hyukjin.kwon] Thanks for reporting. Would you like to work on

[jira] [Resolved] (SPARK-10533) DataFrame filter is not handling float/double with Scientific Notation 'e' / 'E'

[ https://issues.apache.org/jira/browse/SPARK-10533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10533. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9085

[jira] [Updated] (SPARK-10533) DataFrame filter is not handling float/double with Scientific Notation 'e' / 'E'

[ https://issues.apache.org/jira/browse/SPARK-10533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10533: --- Assignee: Adrian Wang > DataFrame filter is not handling float/double with Scientific Notation 'e' /

[jira] [Updated] (SPARK-10533) DataFrame filter is not handling float/double with Scientific Notation 'e' / 'E'

[ https://issues.apache.org/jira/browse/SPARK-10533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10533: --- Description: In DataFrames filter operation,when giving float comparison with e (2.0e2) it is not

[jira] [Resolved] (SPARK-10786) SparkSQLCLIDriver should take the whole statement to generate the CommandProcessor

[ https://issues.apache.org/jira/browse/SPARK-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10786. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8895

<    3   4   5   6   7   8   9   10   11   12   >