[jira] [Resolved] (SPARK-15884) Override stringArgs method in MapPartitionsInR case class in order to avoid Out Of Mermory exceptions when calling toString

[ https://issues.apache.org/jira/browse/SPARK-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15884. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13610 [https://github.

[jira] [Updated] (SPARK-15884) Override stringArgs method in MapPartitionsInR case class in order to avoid Out Of Mermory exceptions when calling toString

[ https://issues.apache.org/jira/browse/SPARK-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15884: --- Assignee: Narine Kokhlikyan > Override stringArgs method in MapPartitionsInR case class in order to a

[jira] [Updated] (SPARK-15862) Better Error Message When Having Database Name in CACHE TABLE AS SELECT

[ https://issues.apache.org/jira/browse/SPARK-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15862: --- Assignee: Xiao Li > Better Error Message When Having Database Name in CACHE TABLE AS SELECT > ---

[jira] [Resolved] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15753. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13496 [https://github.

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Assignee: Wenchen Fan > Revert API breaking changes made in DataFrameReader.text and SQLContext.range

[jira] [Updated] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15753: --- Assignee: Liang-Chi Hsieh > Move some Analyzer stuff to Analyzer from DataFrameWriter > -

[jira] [Created] (SPARK-15863) Update SQL programming guide for Spark 2.0

Cheng Lian created SPARK-15863: -- Summary: Update SQL programming guide for Spark 2.0 Key: SPARK-15863 URL: https://issues.apache.org/jira/browse/SPARK-15863 Project: Spark Issue Type: Documentat

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Description: In Spark 2.0, after unifying Datasets and DataFrames, we made two API breaking changes:

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Description: In Spark 2.0, after unifying Datasets and DataFrames, we made two API breaking changes:

[jira] [Created] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

Cheng Lian created SPARK-15856: -- Summary: Revert API breaking changes made in DataFrameReader.text and SQLContext.range Key: SPARK-15856 URL: https://issues.apache.org/jira/browse/SPARK-15856 Project: Sp

[jira] [Resolved] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15792. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13535 [https://github.

[jira] [Resolved] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15632. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13529 [https://github.

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317460#comment-15317460 ] Cheng Lian commented on SPARK-15632: The {{.map(identity)}} example is quite interest

[jira] [Updated] (SPARK-15654) Reading gzipped files results in duplicate rows

[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15654: --- Assignee: Takeshi Yamamuro > Reading gzipped files results in duplicate rows > --

[jira] [Resolved] (SPARK-15657) RowEncoder should validate the data type of input object

[ https://issues.apache.org/jira/browse/SPARK-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15657. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/spark/pull/1340

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Assignee: Sean Zhong > Dataset typed filter operation changes query plan schema > ---

[jira] [Commented] (SPARK-15140) encoder should make sure input object is not null

[ https://issues.apache.org/jira/browse/SPARK-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314853#comment-15314853 ] Cheng Lian commented on SPARK-15140: Issue resolved by pull request 13469 [https://gi

[jira] [Resolved] (SPARK-15140) encoder should make sure input object is not null

[ https://issues.apache.org/jira/browse/SPARK-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15140. Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > encoder should ma

[jira] [Resolved] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15547. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13474 [https://github.

[jira] [Resolved] (SPARK-15494) encoder code cleanup

[ https://issues.apache.org/jira/browse/SPARK-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15494. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13269 [https://github.

[jira] [Resolved] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14959. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13463 [https://github.

[jira] [Updated] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14959: --- Assignee: Xin Wu > ​Problem Reading partitioned ORC or Parquet files > --

[jira] [Resolved] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15733. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13470 [https://github.

[jira] [Updated] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15733: --- Assignee: Sean Zhong > Makes the explain output less verbose by hiding some verbose output like > No

[jira] [Resolved] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15732. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/spark/pull/1348

[jira] [Updated] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15732: --- Assignee: Wenchen Fan > Dataset generated code "generated.java" Fails with Certain Case Classes > ---

[jira] [Resolved] (SPARK-15734) Avoids printing internal row in explain output

[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15734. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13471 [https://github.

[jira] [Resolved] (SPARK-15719) Disable writing Parquet summary files by default

[ https://issues.apache.org/jira/browse/SPARK-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15719. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13455 [https://github.

[jira] [Updated] (SPARK-15734) Avoids printing internal row in explain output

[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15734: --- Assignee: Sean Zhong > Avoids printing internal row in explain output > -

[jira] [Updated] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13484: --- Assignee: Takeshi Yamamuro > Filter outer joined result using a non-nullable column from the right ta

[jira] [Resolved] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13484. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13290 [https://github.

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311363#comment-15311363 ] Cheng Lian commented on SPARK-11153: Yea, right. Can we do it later on master to mini

[jira] [Resolved] (SPARK-15441) dataset outer join seems to return incorrect result

[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15441. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13425 [https://github.

[jira] [Reopened] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reopened SPARK-9876: --- Re-opened this since we just reverted 1.8.1 upgrade for branch-2.0. https://github.com/apache/spark/pull/

[jira] [Resolved] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15269. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13270 [https://github.

[jira] [Updated] (SPARK-15712) Proper temp table support

[ https://issues.apache.org/jira/browse/SPARK-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15712: --- Description: For proper temp table support, I am proposing to create a temp dir for every {{SparkSes

[jira] [Created] (SPARK-15719) Disable writing Parquet summary files by default

Cheng Lian created SPARK-15719: -- Summary: Disable writing Parquet summary files by default Key: SPARK-15719 URL: https://issues.apache.org/jira/browse/SPARK-15719 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311231#comment-15311231 ] Cheng Lian commented on SPARK-11153: Unfortunately we just decided to revert Parquet

[jira] [Commented] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310700#comment-15310700 ] Cheng Lian commented on SPARK-13795: [~ganeshkrishnan] From the stack trace, I suspec

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14343. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13431 [https://github.

[jira] [Assigned] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14343: -- Assignee: Cheng Lian > Dataframe operations on a partitioned dataset (using partition discover

[jira] [Resolved] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6859. --- Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.0.0 Fixed by upgrading parquet-

[jira] [Commented] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308754#comment-15308754 ] Cheng Lian commented on SPARK-6859: --- Yea, thanks. I'm closing it. > Parquet File Binary

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308083#comment-15308083 ] Cheng Lian commented on SPARK-8118: --- Yea, unfortunately at last we found that due to a f

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15112. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13362 [https://github.

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14343: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataframe operations on a partitioned

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never change query plan schema. However, Dataset typed filter

[jira] [Updated] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9876: -- Assignee: Ryan Blue > Upgrade parquet-mr to 1.8.1 > --- > > Key:

[jira] [Resolved] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9876. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13280 [https://github.com

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304873#comment-15304873 ] Cheng Lian commented on SPARK-15632: cc [~cloud_fan] [~marmbrus] > Dataset typed fil

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never changes query plan schema. However, Dataset typed filter

[jira] [Created] (SPARK-15632) Dataset typed filter operation changes query plan schema

Cheng Lian created SPARK-15632: -- Summary: Dataset typed filter operation changes query plan schema Key: SPARK-15632 URL: https://issues.apache.org/jira/browse/SPARK-15632 Project: Spark Issue Ty

[jira] [Updated] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15112: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset filter returns garbage > -

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset.show() doesn't disply inner ne

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Encoder validation is too strict for i

[jira] [Created] (SPARK-15631) Dataset and encoder bug fixes

Cheng Lian created SPARK-15631: -- Summary: Dataset and encoder bug fixes Key: SPARK-15631 URL: https://issues.apache.org/jira/browse/SPARK-15631 Project: Spark Issue Type: Bug Component

[jira] [Resolved] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15550. Resolution: Fixed Issue resolved by pull request 13331 [https://github.com/apache/spark/pull/13331]

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: Say we have the following nested case class: {code} case class ClassData(a: String, b:

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Created] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

Cheng Lian created SPARK-15550: -- Summary: Dataset.show() doesn't disply inner nested structs properly Key: SPARK-15550 URL: https://issues.apache.org/jira/browse/SPARK-15550 Project: Spark Issu

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Created] (SPARK-15547) Encoder validation is too strict for inner nested structs

Cheng Lian created SPARK-15547: -- Summary: Encoder validation is too strict for inner nested structs Key: SPARK-15547 URL: https://issues.apache.org/jira/browse/SPARK-15547 Project: Spark Issue T

[jira] [Resolved] (SPARK-15498) fix slow tests

[ https://issues.apache.org/jira/browse/SPARK-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15498. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13273 [https://github.

[jira] [Updated] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15431: --- Assignee: Xin Wu > Support LIST FILE(s)|JAR(s) command natively > ---

[jira] [Resolved] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15431. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13212 [https://github.

[jira] [Commented] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297408#comment-15297408 ] Cheng Lian commented on SPARK-15269: Two facts make this issue pretty hard to be fixe

[jira] [Commented] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297282#comment-15297282 ] Cheng Lian commented on SPARK-14343: Seems that we were reading from the wrong column

[jira] [Resolved] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14031. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13229 [https://github.

[jira] [Updated] (SPARK-14543) SQL/Hive insertInto has unexpected results

[ https://issues.apache.org/jira/browse/SPARK-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14543: --- Assignee: Ryan Blue > SQL/Hive insertInto has unexpected results > --

[jira] [Resolved] (SPARK-15307) Super slow to load a partitioned table from local disks

[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15307. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13094 [https://github.

[jira] [Updated] (SPARK-15307) Super slow to load a partitioned table from local disks

[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15307: --- Assignee: Davies Liu > Super slow to load a partitioned table from local disks >

[jira] [Resolved] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15334. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13127 [https://github.

[jira] [Updated] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15334: --- Assignee: Sean Zhong > HiveClient facade not compatible with Hive 0.12 >

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Assignee: Xin Wu > Creating external table leaves empty directory under warehouse directory > ---

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Description: Adding the following test case in {{HiveDDLSuite}} may reproduce this issue: {code} t

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281657#comment-15281657 ] Cheng Lian commented on SPARK-15269: [~xwu0226] Thanks a lot for the detailed investi

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Summary: Creating external table leaves empty directory under warehouse directory (was: Creating ext

[jira] [Resolved] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15171. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12945 [https://github.

[jira] [Updated] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15171: --- Assignee: Sean Zhong > Deprecate registerTempTable and add dataset.createTempView > -

[jira] [Resolved] (SPARK-14933) Failed to create view out of a parquet or orc table

[ https://issues.apache.org/jira/browse/SPARK-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14933. Issue resolved by pull request 12716 [https://github.com/apache/spark/pull/12716] > Failed to create v

[jira] [Comment Edited] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279992#comment-15279992 ] Cheng Lian edited comment on SPARK-15269 at 5/11/16 11:44 AM: -

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279992#comment-15279992 ] Cheng Lian commented on SPARK-15269: Investigated this issue for a while, and observe

[jira] [Created] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

Cheng Lian created SPARK-15269: -- Summary: Creating external table in test code leaves empty directory under warehouse directory Key: SPARK-15269 URL: https://issues.apache.org/jira/browse/SPARK-15269 Pro

[jira] [Updated] (SPARK-15253) For a data source table, Describe table needs to handle spark.sql.sources.schema

[ https://issues.apache.org/jira/browse/SPARK-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15253: --- Assignee: Sean Zhong > For a data source table, Describe table needs to handle > spark.sql.sources.s

[jira] [Updated] (SPARK-15192) RowEncoder needs to verify nullability in a more explicit way

[ https://issues.apache.org/jira/browse/SPARK-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15192: --- Description: When we create a Dataset from an RDD of rows with a specific schema, if the nullability

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Assignee: Ryan Blue > SQL partitioning must match existing tables, but is not checked. >

[jira] [Resolved] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14459. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12239 [https://github.

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > SQL partitioning must match existing tables,

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 Description: It will cause failure wh

[jira] [Resolved] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15211. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12986 [https://github.

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Assignee: Liang-Chi Hsieh > Select features column from LibSVMRelation causes failure > -

[jira] [Resolved] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14962. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12777 [https://github.

[jira] [Updated] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14962: --- Assignee: Hyukjin Kwon > spark.sql.orc.filterPushdown=true breaks DataFrame where functionality > ---

[jira] [Comment Edited] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273837#comment-15273837 ] Cheng Lian edited comment on SPARK-15112 at 5/6/16 10:22 AM: -

<    1   2   3   4   5   6   7   8   9   10   >