[jira] [Commented] (SPARK-18559) Fix HLL++ with small relative error

[ https://issues.apache.org/jira/browse/SPARK-18559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828262#comment-15828262 ] Hyukjin Kwon commented on SPARK-18559: -- [~srowen], it seems mistakenly not resolved. > Fix HLL++

[jira] [Commented] (SPARK-15745) Use classloader's getResource() for reading resource files in HiveTests

[ https://issues.apache.org/jira/browse/SPARK-15745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828210#comment-15828210 ] Hyukjin Kwon commented on SPARK-15745: -- Hi [~rxin] and [~srowen], this seems mistakenly not

[jira] [Commented] (SPARK-17101) Provide consistent format identifiers for TextFileFormat and ParquetFileFormat

[ https://issues.apache.org/jira/browse/SPARK-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828253#comment-15828253 ] Hyukjin Kwon commented on SPARK-17101: -- I see. Thank you for correcting me. I will keep it in mind

[jira] [Commented] (SPARK-19183) Add deleteWithJob hook to internal commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-19183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828273#comment-15828273 ] Hyukjin Kwon commented on SPARK-19183: -- Hi [~cloud_fan], while looking though the JIRAs for my

[jira] [Commented] (SPARK-19296) Awkward changes for JdbcUtils.saveTable in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830915#comment-15830915 ] Hyukjin Kwon commented on SPARK-19296: --

[jira] [Comment Edited] (SPARK-19296) Awkward changes for JdbcUtils.saveTable in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830915#comment-15830915 ] Hyukjin Kwon edited comment on SPARK-19296 at 1/20/17 12:53 AM:

[jira] [Commented] (SPARK-19296) Awkward changes for JdbcUtils.saveTable in Spark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15830924#comment-15830924 ] Hyukjin Kwon commented on SPARK-19296: -- {quote} incompatible to previous versions {quote} If this

[jira] [Resolved] (SPARK-16626) Code duplication after SPARK-14906

[ https://issues.apache.org/jira/browse/SPARK-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16626. -- Resolution: Won't Fix > Code duplication after SPARK-14906 >

[jira] [Commented] (SPARK-7007) Add metrics source for ExecutorAllocationManager to expose internal status

[ https://issues.apache.org/jira/browse/SPARK-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831573#comment-15831573 ] Hyukjin Kwon commented on SPARK-7007: - Hi [~andrewor14], I just happened to look at this JIRA. It

[jira] [Resolved] (SPARK-14344) saveAsParquetFile creates _metadata file even when disabled

[ https://issues.apache.org/jira/browse/SPARK-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14344. -- Resolution: Cannot Reproduce I verified this in

[jira] [Commented] (SPARK-6802) User Defined Aggregate Function Refactoring

[ https://issues.apache.org/jira/browse/SPARK-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831570#comment-15831570 ] Hyukjin Kwon commented on SPARK-6802: - If this JIRA is about Python UDAF, there is a JIRA -

[jira] [Commented] (SPARK-8273) Driver hangs up when yarn shutdown in client mode

[ https://issues.apache.org/jira/browse/SPARK-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831577#comment-15831577 ] Hyukjin Kwon commented on SPARK-8273: - Oh, [~andrewor14], this one also looks not resolved. > Driver

[jira] [Commented] (SPARK-19134) Fix several sql, mllib and status api examples not working

[ https://issues.apache.org/jira/browse/SPARK-19134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831680#comment-15831680 ] Hyukjin Kwon commented on SPARK-19134: -- Oh [~yanboliang], it seems mistakenly not resolved.. :) >

[jira] [Commented] (SPARK-16896) Loading csv with duplicate column names

[ https://issues.apache.org/jira/browse/SPARK-16896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429950#comment-15429950 ] Hyukjin Kwon commented on SPARK-16896: -- Yup, then I will work on this and submit a PR within few

[jira] [Commented] (SPARK-17168) CSV with header is incorrectly read if file is partitioned

[ https://issues.apache.org/jira/browse/SPARK-17168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429947#comment-15429947 ] Hyukjin Kwon commented on SPARK-17168: -- Thanks for cc me [~maropu]! I also tend to agree CSV files

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon commented on SPARK-17024: -- I can't reproduce this against current master and suspect a

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 11:25 AM: I can't

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 10:23 AM: I can't

[jira] [Commented] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429672#comment-15429672 ] Hyukjin Kwon commented on SPARK-17024: -- It seems I can't reproduce the example in the JIRA

[jira] [Comment Edited] (SPARK-17024) Weird behaviour of the DataFrame when a column name contains dots.

[ https://issues.apache.org/jira/browse/SPARK-17024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429659#comment-15429659 ] Hyukjin Kwon edited comment on SPARK-17024 at 8/21/16 11:26 AM: I can't

[jira] [Created] (SPARK-16877) Add a rule for preventing use Java's Override annotation

Hyukjin Kwon created SPARK-16877: Summary: Add a rule for preventing use Java's Override annotation Key: SPARK-16877 URL: https://issues.apache.org/jira/browse/SPARK-16877 Project: Spark

[jira] [Commented] (SPARK-19713) saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885717#comment-15885717 ] Hyukjin Kwon commented on SPARK-19713: -- Could you update the JIRA title to be more meaningful and

[jira] [Commented] (SPARK-19729) Strange behaviour with reading csv with schema into dataframe

[ https://issues.apache.org/jira/browse/SPARK-19729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885758#comment-15885758 ] Hyukjin Kwon commented on SPARK-19729: -- I am sorry that I am a bit confused. {code} scala>

[jira] [Commented] (SPARK-19741) ClassCastException when using Dataset with type containing value types

[ https://issues.apache.org/jira/browse/SPARK-19741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891463#comment-15891463 ] Hyukjin Kwon commented on SPARK-19741: -- I just tried the code above in the current master and it

[jira] [Comment Edited] (SPARK-19741) ClassCastException when using Dataset with type containing value types

[ https://issues.apache.org/jira/browse/SPARK-19741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891463#comment-15891463 ] Hyukjin Kwon edited comment on SPARK-19741 at 3/2/17 2:00 AM: -- I just tried

[jira] [Commented] (SPARK-16102) Use Record API from Univocity rather than current data cast API.

[ https://issues.apache.org/jira/browse/SPARK-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889058#comment-15889058 ] Hyukjin Kwon commented on SPARK-16102: -- Yes, let me check out this API and other APIs too. Let me

[jira] [Commented] (SPARK-18579) spark-csv strips whitespace (pyspark)

[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889117#comment-15889117 ] Hyukjin Kwon commented on SPARK-18579: -- Oh, I overlooked. You meant it always strips the white

[jira] [Commented] (SPARK-19713) saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889114#comment-15889114 ] Hyukjin Kwon commented on SPARK-19713: -- Hi [~balaramraju] Could you update the title? > saveAsTable

[jira] [Resolved] (SPARK-16103) Share a single Row for CSV data source rather than creating every time

[ https://issues.apache.org/jira/browse/SPARK-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16103. -- Resolution: Duplicate yup, fixed in https://github.com/apache/spark/pull/16669 > Share a

[jira] [Resolved] (SPARK-16512) No way to load CSV data without dropping whole rows when some of data is not matched with given schema

[ https://issues.apache.org/jira/browse/SPARK-16512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16512. -- Resolution: Duplicate > No way to load CSV data without dropping whole rows when some of data

[jira] [Resolved] (SPARK-17225) Support multiple null values in csv files

[ https://issues.apache.org/jira/browse/SPARK-17225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17225. -- Resolution: Duplicate I am resolving this as a duplicate because that JIRA has a PR. >

[jira] [Resolved] (SPARK-14194) spark csv reader not working properly if CSV content contains CRLF character (newline) in the intermediate cell

[ https://issues.apache.org/jira/browse/SPARK-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14194. -- Resolution: Duplicate I proposed to solve this via {{wholeFile}} option and it seems merged. I

[jira] [Resolved] (SPARK-17224) Support skipping multiple header rows in csv

[ https://issues.apache.org/jira/browse/SPARK-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17224. -- Resolution: Duplicate Now multiple header line can be dealt with by {{wholeFile}} option. Let

[jira] [Resolved] (SPARK-19729) Strange behaviour with reading csv with schema into dataframe

[ https://issues.apache.org/jira/browse/SPARK-19729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19729. -- Resolution: Invalid I am resolving this as {{Invalid}}. Please reopen this if I was wrong with

[jira] [Resolved] (SPARK-19521) Error with embedded line break (multi-line record) in csv file.

[ https://issues.apache.org/jira/browse/SPARK-19521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19521. -- Resolution: Duplicate I am resolving this as a duplicate of SPARK-19610 as that one has a PR

[jira] [Resolved] (SPARK-16846) read.csv() option: "inferSchema" don't work

[ https://issues.apache.org/jira/browse/SPARK-16846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16846. -- Resolution: Not A Problem If the schema is given, it does not infer the schema. > read.csv()

[jira] [Resolved] (SPARK-19778) alais cannot use in group by

[ https://issues.apache.org/jira/browse/SPARK-19778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19778. -- Resolution: Duplicate I am resolving this as a duplicate of SPARK-14471 Please reopen this if

[jira] [Resolved] (SPARK-15224) Can not delete jar and list jar in spark Thrift server

[ https://issues.apache.org/jira/browse/SPARK-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15224. -- Resolution: Duplicate I am resolving this as a duplicate of SPARK-15763 Please reopen this if

[jira] [Commented] (SPARK-14471) The alias created in SELECT could be used in GROUP BY and followed expressions

[ https://issues.apache.org/jira/browse/SPARK-14471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885116#comment-15885116 ] Hyukjin Kwon commented on SPARK-14471: -- Still happens in the current master -

[jira] [Commented] (SPARK-14273) Add FileFormat.isSplittable to indicate whether a format is splittable

[ https://issues.apache.org/jira/browse/SPARK-14273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885086#comment-15885086 ] Hyukjin Kwon commented on SPARK-14273: -- Hi [~yhuai] and [~liancheng], it seems there is

[jira] [Commented] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890249#comment-15890249 ] Hyukjin Kwon commented on SPARK-19754: -- Thank you for cc'ing me. It seems it returns as below in the

[jira] [Resolved] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19754. -- Resolution: Cannot Reproduce > Casting to int from a JSON-parsed float rounds instead of

[jira] [Commented] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890272#comment-15890272 ] Hyukjin Kwon commented on SPARK-19754: -- I see. It'd be great if anyone can identify the JIRA fixing

[jira] [Created] (SPARK-17212) TypeCoercion support widening conversion between DateType and TimestampType

Hyukjin Kwon created SPARK-17212: Summary: TypeCoercion support widening conversion between DateType and TimestampType Key: SPARK-17212 URL: https://issues.apache.org/jira/browse/SPARK-17212 Project:

[jira] [Commented] (SPARK-17174) Provide support for Timestamp type Column in add_months function to return HH:mm:ss

[ https://issues.apache.org/jira/browse/SPARK-17174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434746#comment-15434746 ] Hyukjin Kwon commented on SPARK-17174: -- I see. Thank you for your advise! I will submit a PR for

[jira] [Updated] (SPARK-17212) TypeCoercion support widening conversion between DateType and TimestampType

[ https://issues.apache.org/jira/browse/SPARK-17212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17212: - Description: Currently, type-widening does not work between {{TimestampType}} and {{DateType}}.

[jira] [Commented] (SPARK-17227) Allow configuring record delimiter in csv

[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436140#comment-15436140 ] Hyukjin Kwon commented on SPARK-17227: -- Also, it would be great if the JIRA has an example and

[jira] [Commented] (SPARK-17222) Support multline csv records

[ https://issues.apache.org/jira/browse/SPARK-17222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436154#comment-15436154 ] Hyukjin Kwon commented on SPARK-17222: -- Here is *related* PR

[jira] [Commented] (SPARK-17227) Allow configuring record delimiter in csv

[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436150#comment-15436150 ] Hyukjin Kwon commented on SPARK-17227: -- Ah, SPARK-17222 is about miltiple-lines but IMHO it might

[jira] [Commented] (SPARK-17227) Allow configuring record delimiter in csv

[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436139#comment-15436139 ] Hyukjin Kwon commented on SPARK-17227: -- If I remember this correctly, we are not using that

[jira] [Commented] (SPARK-17226) Allow defining multiple date formats per column in csv

[ https://issues.apache.org/jira/browse/SPARK-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436158#comment-15436158 ] Hyukjin Kwon commented on SPARK-17226: -- Codes to reproduce and suggestion maybe rather than just

[jira] [Commented] (SPARK-17227) Allow configuring record delimiter in csv

[ https://issues.apache.org/jira/browse/SPARK-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436147#comment-15436147 ] Hyukjin Kwon commented on SPARK-17227: -- We may have to open a JIRA to deal with multiple-lines

[jira] [Commented] (SPARK-17341) Can't read Parquet data with fields containing periods "."

[ https://issues.apache.org/jira/browse/SPARK-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453775#comment-15453775 ] Hyukjin Kwon commented on SPARK-17341: -- Ah, the issue itself seems not duplicated but the fix should

[jira] [Commented] (SPARK-17349) Update testthat package on Jenkins

[ https://issues.apache.org/jira/browse/SPARK-17349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453784#comment-15453784 ] Hyukjin Kwon commented on SPARK-17349: -- Cool! > Update testthat package on Jenkins >

[jira] [Commented] (SPARK-17341) Can't read Parquet data with fields containing periods "."

[ https://issues.apache.org/jira/browse/SPARK-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453767#comment-15453767 ] Hyukjin Kwon commented on SPARK-17341: -- Is this a duplicate of SPARK-16698? I believe this does not

[jira] [Created] (SPARK-17388) Support for inferring type date/timestamp for partition column

Hyukjin Kwon created SPARK-17388: Summary: Support for inferring type date/timestamp for partition column Key: SPARK-17388 URL: https://issues.apache.org/jira/browse/SPARK-17388 Project: Spark

[jira] [Updated] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17388: - Summary: Support for inferring type date/timestamp/decimal for partition column (was: Support

[jira] [Updated] (SPARK-17388) Support for inferring type date/timestamp/decimal for partition column

[ https://issues.apache.org/jira/browse/SPARK-17388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17388: - Priority: Major (was: Minor) > Support for inferring type date/timestamp/decimal for partition

[jira] [Commented] (SPARK-17354) java.lang.ClassCastException: java.lang.Integer cannot be cast to java.sql.Date

[ https://issues.apache.org/jira/browse/SPARK-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455123#comment-15455123 ] Hyukjin Kwon commented on SPARK-17354: -- I see. This seems a bug in `ColumnVectorUtils`. IIUC, the

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464236#comment-15464236 ] Hyukjin Kwon commented on SPARK-17339: -- [~sarutak] [~shivaram] Please cc me if any of you submit a

[jira] [Commented] (SPARK-17339) Fix SparkR tests on Windows

[ https://issues.apache.org/jira/browse/SPARK-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464250#comment-15464250 ] Hyukjin Kwon commented on SPARK-17339: -- Yeap, I totally agree. Thank you both! Will submit a PR

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447755#comment-15447755 ] Hyukjin Kwon commented on SPARK-17290: -- This should be a duplicate of SPARK-16462, SPARK-16460,

[jira] [Commented] (SPARK-17290) Spark CSVInferSchema does not always respect nullValue settings

[ https://issues.apache.org/jira/browse/SPARK-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447758#comment-15447758 ] Hyukjin Kwon commented on SPARK-17290: -- BTW, there is a related PR here,

[jira] [Created] (SPARK-17326) Tests with HiveContext in SparkR being skipped always

Hyukjin Kwon created SPARK-17326: Summary: Tests with HiveContext in SparkR being skipped always Key: SPARK-17326 URL: https://issues.apache.org/jira/browse/SPARK-17326 Project: Spark Issue

[jira] [Created] (SPARK-17310) Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side

Hyukjin Kwon created SPARK-17310: Summary: Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side Key: SPARK-17310 URL: https://issues.apache.org/jira/browse/SPARK-17310

[jira] [Commented] (SPARK-17310) Disable Parquet's record-by-record filter in normal parquet reader and do it in Spark-side

[ https://issues.apache.org/jira/browse/SPARK-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15479530#comment-15479530 ] Hyukjin Kwon commented on SPARK-17310: -- [~andrew_duffy] Thanks Andrew. I will work on this. >

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15479917#comment-15479917 ] Hyukjin Kwon commented on SPARK-17477: -- Is this subset of SPARK-16544? Also, I remember I was told

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485505#comment-15485505 ] Hyukjin Kwon commented on SPARK-17477: -- I left a related commnet

[jira] [Commented] (SPARK-17545) Spark SQL Catalyst doesn't handle ISO 8601 date without colon in offset

[ https://issues.apache.org/jira/browse/SPARK-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495410#comment-15495410 ] Hyukjin Kwon commented on SPARK-17545: -- Therefore, IMHO, this is not an issue as we can workaround

[jira] [Comment Edited] (SPARK-17545) Spark SQL Catalyst doesn't handle ISO 8601 date without colon in offset

[ https://issues.apache.org/jira/browse/SPARK-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495408#comment-15495408 ] Hyukjin Kwon edited comment on SPARK-17545 at 9/16/16 5:20 AM: --- Hi

[jira] [Commented] (SPARK-17545) Spark SQL Catalyst doesn't handle ISO 8601 date without colon in offset

[ https://issues.apache.org/jira/browse/SPARK-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495413#comment-15495413 ] Hyukjin Kwon commented on SPARK-17545: -- FYI - this is related with

[jira] [Commented] (SPARK-17545) Spark SQL Catalyst doesn't handle ISO 8601 date without colon in offset

[ https://issues.apache.org/jira/browse/SPARK-17545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495408#comment-15495408 ] Hyukjin Kwon commented on SPARK-17545: -- Hi [~nbeyer], the basic ISO format currently follows

[jira] [Commented] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15495431#comment-15495431 ] Hyukjin Kwon commented on SPARK-17557: -- Do you mind if I ask a simple file so that I can reproduce

[jira] [Created] (SPARK-17658) write.df API requires path which is not actually always nessasary in SparkR

Hyukjin Kwon created SPARK-17658: Summary: write.df API requires path which is not actually always nessasary in SparkR Key: SPARK-17658 URL: https://issues.apache.org/jira/browse/SPARK-17658 Project:

[jira] [Commented] (SPARK-17665) SparkR does not support options in other types consistently other APIs

[ https://issues.apache.org/jira/browse/SPARK-17665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522372#comment-15522372 ] Hyukjin Kwon commented on SPARK-17665: -- Let me please open a PR and then talk there as it would be a

[jira] [Created] (SPARK-17665) SparkR supports options in other types consistently other APIs

Hyukjin Kwon created SPARK-17665: Summary: SparkR supports options in other types consistently other APIs Key: SPARK-17665 URL: https://issues.apache.org/jira/browse/SPARK-17665 Project: Spark

[jira] [Updated] (SPARK-17665) SparkR supports options in other types consistently other APIs

[ https://issues.apache.org/jira/browse/SPARK-17665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17665: - Component/s: SparkR > SparkR supports options in other types consistently other APIs >

[jira] [Commented] (SPARK-17765) org.apache.spark.mllib.linalg.VectorUDT cannot be cast to org.apache.spark.sql.types.StructType

[ https://issues.apache.org/jira/browse/SPARK-17765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548308#comment-15548308 ] Hyukjin Kwon commented on SPARK-17765: -- It seems this one can be quickly fixed. Let me please submit

[jira] [Created] (SPARK-17776) Potentially duplicated names which might have conflicts between JDBC options and properties instance

Hyukjin Kwon created SPARK-17776: Summary: Potentially duplicated names which might have conflicts between JDBC options and properties instance Key: SPARK-17776 URL:

[jira] [Commented] (SPARK-8128) Schema Merging Broken: Dataframe Fails to Recognize Column in Schema

[ https://issues.apache.org/jira/browse/SPARK-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15558088#comment-15558088 ] Hyukjin Kwon commented on SPARK-8128: - I am not 100% sure but I recall I saw similar issue was

[jira] [Resolved] (SPARK-16903) nullValue in first field is not respected by CSV source when read

[ https://issues.apache.org/jira/browse/SPARK-16903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16903. -- Resolution: Duplicate [~falaki] I am going to make this as a duplicate because the PR was

[jira] [Commented] (SPARK-11428) Schema Merging Broken for Some Queries

[ https://issues.apache.org/jira/browse/SPARK-11428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15558095#comment-15558095 ] Hyukjin Kwon commented on SPARK-11428: -- How about https://issues.apache.org/jira/browse/SPARK-8128 ?

[jira] [Comment Edited] (SPARK-5511) [SQL] Possible optimisations for predicate pushdowns from Spark SQL to Parquet

[ https://issues.apache.org/jira/browse/SPARK-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557667#comment-15557667 ] Hyukjin Kwon edited comment on SPARK-5511 at 10/8/16 4:52 PM: -- 1. I agree it

[jira] [Commented] (SPARK-10501) support UUID as an atomic type

[ https://issues.apache.org/jira/browse/SPARK-10501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559405#comment-15559405 ] Hyukjin Kwon commented on SPARK-10501: -- Ah, it was type not the function. I just rushed the JIRA.

[jira] [Created] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

Hyukjin Kwon created SPARK-17838: Summary: Strict type checking for arguments with a better messages across APIs. Key: SPARK-17838 URL: https://issues.apache.org/jira/browse/SPARK-17838 Project:

[jira] [Closed] (SPARK-10744) parser error (constant * column is null interpreted as constant * boolean)

[ https://issues.apache.org/jira/browse/SPARK-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon closed SPARK-10744. Resolution: Cannot Reproduce I can't reproduce this in the current master. {code} scala> val df =

[jira] [Closed] (SPARK-10754) table and column name are case sensitive when json Dataframe was registered as tempTable using JavaSparkContext.

[ https://issues.apache.org/jira/browse/SPARK-10754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon closed SPARK-10754. Resolution: Cannot Reproduce I can't reproduce this in the current master. {code} scala> val df =

[jira] [Closed] (SPARK-10899) Support JDBC pushdown for additional commands

[ https://issues.apache.org/jira/browse/SPARK-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon closed SPARK-10899. Resolution: Duplicate I am closing this because I believe this is a subset of SPARK-12126 >

[jira] [Commented] (SPARK-10954) Parquet version in the "created_by" metadata field of Parquet files written by Spark 1.5 and 1.6 is wrong

[ https://issues.apache.org/jira/browse/SPARK-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557900#comment-15557900 ] Hyukjin Kwon commented on SPARK-10954: -- Hi [~lian cheng], I believe we upgraded the Parquet version.

[jira] [Resolved] (SPARK-11145) Cannot filter using a partition key and another column

[ https://issues.apache.org/jira/browse/SPARK-11145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11145. -- Resolution: Cannot Reproduce I can't reproduce this against current master {code} >>> from

[jira] [Commented] (SPARK-11186) Caseness inconsistency between SQLContext and HiveContext

[ https://issues.apache.org/jira/browse/SPARK-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557911#comment-15557911 ] Hyukjin Kwon commented on SPARK-11186: -- [~smolav] Could you confirm if this still happens in the

[jira] [Commented] (SPARK-11202) Unsupported dataType

[ https://issues.apache.org/jira/browse/SPARK-11202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557917#comment-15557917 ] Hyukjin Kwon commented on SPARK-11202: -- [~luckywhc] Can you fix the title of this JIRA? it looks

[jira] [Resolved] (SPARK-11356) Option to refresh information about parquet partitions

[ https://issues.apache.org/jira/browse/SPARK-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11356. -- Resolution: Fixed Please use structured streaming. If I misunderstood and you can't do this

[jira] [Closed] (SPARK-11660) Spark Thrift GetResultSetMetadata describes a VARCHAR as a STRING

[ https://issues.apache.org/jira/browse/SPARK-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon closed SPARK-11660. Resolution: Duplicate It seems a duplicate. Please reopen this if you feel strongly this is not a

[jira] [Commented] (SPARK-11620) parquet.hadoop.ParquetOutputCommitter.commitJob() throws parquet.io.ParquetEncodingException

[ https://issues.apache.org/jira/browse/SPARK-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557931#comment-15557931 ] Hyukjin Kwon commented on SPARK-11620: -- [~swethakasireddy] Could you please check if this still

[jira] [Commented] (SPARK-11784) enable Timestamp filter pushdown

[ https://issues.apache.org/jira/browse/SPARK-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557940#comment-15557940 ] Hyukjin Kwon commented on SPARK-11784: -- Could you feel up the description? it seems you referred the

[jira] [Commented] (SPARK-11868) wrong results returned from dataframe create from Rows without consistent schma on pyspark

[ https://issues.apache.org/jira/browse/SPARK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557944#comment-15557944 ] Hyukjin Kwon commented on SPARK-11868: -- FYI, it now prints differently: {code} >>> dicts =

[jira] [Comment Edited] (SPARK-11868) wrong results returned from dataframe create from Rows without consistent schma on pyspark

[ https://issues.apache.org/jira/browse/SPARK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557944#comment-15557944 ] Hyukjin Kwon edited comment on SPARK-11868 at 10/8/16 1:11 PM: --- FYI, it now

[jira] [Comment Edited] (SPARK-11868) wrong results returned from dataframe create from Rows without consistent schma on pyspark

[ https://issues.apache.org/jira/browse/SPARK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15557944#comment-15557944 ] Hyukjin Kwon edited comment on SPARK-11868 at 10/8/16 1:12 PM: --- FYI, it now

[jira] [Closed] (SPARK-12497) thriftServer does not support semicolon in sql

[ https://issues.apache.org/jira/browse/SPARK-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon closed SPARK-12497. Resolution: Duplicate It seems the duplicate is clearly linked. Please reopen this if you strongly

<    3   4   5   6   7   8   9   10   11   12   >