[jira] [Updated] (SPARK-48148) JSON objects should not be modified when read as STRING

2024-05-06 Thread Eric Maynard (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Maynard updated SPARK-48148: - Description: Currently, when reading a JSON like this: bq. {"a": {"b":

[jira] [Created] (SPARK-41995) schema_of_json only accepts foldable expressions

2023-01-11 Thread Eric Maynard (Jira)
Eric Maynard created SPARK-41995: Summary: schema_of_json only accepts foldable expressions Key: SPARK-41995 URL: https://issues.apache.org/jira/browse/SPARK-41995 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24936) Better error message when trying a shuffle fetch over 2 GB

2019-06-18 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Maynard updated SPARK-24936: - Description: *strong text*After SPARK-24297, spark will try to fetch shuffle blocks to disk if

[jira] [Updated] (SPARK-24936) Better error message when trying a shuffle fetch over 2 GB

2019-06-18 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Maynard updated SPARK-24936: - Description: After SPARK-24297, spark will try to fetch shuffle blocks to disk if their over

[jira] [Updated] (SPARK-27421) RuntimeException when querying a view on a partitioned parquet table

2019-04-10 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Maynard updated SPARK-27421: - Affects Version/s: 2.4.1 > RuntimeException when querying a view on a partitioned parquet table

[jira] [Commented] (SPARK-27421) RuntimeException when querying a view on a partitioned parquet table

2019-04-10 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814872#comment-16814872 ] Eric Maynard commented on SPARK-27421: -- [~shivuson...@gmail.com] Any hiccup confirming the issue

[jira] [Updated] (SPARK-27421) RuntimeException when querying a view on a partitioned parquet table

2019-04-09 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Maynard updated SPARK-27421: - Environment: Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_141) >

[jira] [Created] (SPARK-27421) RuntimeException when querying a view on a partitioned parquet table

2019-04-09 Thread Eric Maynard (JIRA)
Eric Maynard created SPARK-27421: Summary: RuntimeException when querying a view on a partitioned parquet table Key: SPARK-27421 URL: https://issues.apache.org/jira/browse/SPARK-27421 Project: Spark

[jira] [Commented] (SPARK-24469) Support collations in Spark SQL

2018-06-06 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503687#comment-16503687 ] Eric Maynard commented on SPARK-24469: -- Ah, I see, I was wrongly thinking of the second case where

[jira] [Commented] (SPARK-24469) Support collations in Spark SQL

2018-06-06 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503586#comment-16503586 ] Eric Maynard commented on SPARK-24469: -- bq. SELECT UPPER(text)GROUP BY UPPER(text) bq.

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

2018-04-26 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454365#comment-16454365 ] Eric Maynard commented on SPARK-23830: -- [~jerryshao] I agree we should not support using a

[jira] [Commented] (SPARK-23852) Parquet MR bug can lead to incorrect SQL results

2018-04-24 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450018#comment-16450018 ] Eric Maynard commented on SPARK-23852: -- {color:#33}>There is no upstream release of Parquet that

[jira] [Commented] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

2018-04-24 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449948#comment-16449948 ] Eric Maynard commented on SPARK-23519: -- Why is the fact that you dynamically generate the statement

[jira] [Commented] (SPARK-22541) Dataframes: applying multiple filters one after another using udfs and accumulators results in faulty accumulators

2017-11-17 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257004#comment-16257004 ] Eric Maynard commented on SPARK-22541: -- Yeah, this is a common problem when you have side effects in

[jira] [Commented] (SPARK-22436) New function strip() to remove all whitespace from string

2017-11-03 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238712#comment-16238712 ] Eric Maynard commented on SPARK-22436: -- [~asmaier] Wouldn't the right way to implement this be to

[jira] [Comment Edited] (SPARK-19713) saveAsTable

2017-03-16 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928778#comment-15928778 ] Eric Maynard edited comment on SPARK-19713 at 3/16/17 8:08 PM: --- Not really

[jira] [Commented] (SPARK-19713) saveAsTable

2017-03-16 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928778#comment-15928778 ] Eric Maynard commented on SPARK-19713: -- Not really relevant here, but to address: >2. Hive cannot

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

2017-03-05 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896194#comment-15896194 ] Eric Maynard edited comment on SPARK-19656 at 3/5/17 11:51 AM: --- Here is a

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

2017-03-05 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896194#comment-15896194 ] Eric Maynard commented on SPARK-19656: -- Here is a complete working example in Java:

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

2017-03-04 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895998#comment-15895998 ] Eric Maynard edited comment on SPARK-19656 at 3/5/17 12:58 AM: --- Normally

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

2017-03-04 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895998#comment-15895998 ] Eric Maynard commented on SPARK-19656: -- Normally after getting the `datum` you should call

[jira] [Commented] (SPARK-19713) saveAsTable

2017-03-04 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895991#comment-15895991 ] Eric Maynard commented on SPARK-19713: -- In general instead of using `DataFrameWriter.saveAsTable`, I

[jira] [Commented] (SPARK-18005) optional binary Dataframe Column throws (UTF8) is not a group while loading a Dataframe

2017-02-09 Thread Eric Maynard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859913#comment-15859913 ] Eric Maynard commented on SPARK-18005: -- This bug appears in the 1.6.x branch as well. > optional