[jira] [Created] (SPARK-14594) Improve error messages for RDD API

2016-04-13 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-14594: --- Summary: Improve error messages for RDD API Key: SPARK-14594 URL: https://issues.apache.org/jira/browse/SPARK-14594 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14594) Improve error messages for RDD API

2016-04-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253462#comment-15253462 ] Marco Gaido commented on SPARK-14594: - Yes, it works with few data. But if you put a lot of data

[jira] [Commented] (SPARK-14594) Improve error messages for RDD API

2016-04-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255186#comment-15255186 ] Marco Gaido commented on SPARK-14594: - Yes, I do believe that this is what is happening > Improve

[jira] [Updated] (SPARK-14594) Improve error messages for RDD API

2016-04-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-14594: Affects Version/s: (was: 1.6.0) 1.5.2 > Improve error messages for RDD

[jira] [Commented] (SPARK-14594) Improve error messages for RDD API

2016-04-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249521#comment-15249521 ] Marco Gaido commented on SPARK-14594: - I am using Spark1.5.2. Maybe the issue is resolved now.. >

[jira] [Commented] (SPARK-14594) Improve error messages for RDD API

2016-04-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251507#comment-15251507 ] Marco Gaido commented on SPARK-14594: - The code is quite simple, what I can't give you is the data

[jira] [Created] (SPARK-21738) Thriftserver doesn't cancel jobs when session is closed

2017-08-15 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-21738: --- Summary: Thriftserver doesn't cancel jobs when session is closed Key: SPARK-21738 URL: https://issues.apache.org/jira/browse/SPARK-21738 Project: Spark Issue

[jira] [Commented] (SPARK-21340) Bring PySpark MLLib evaluation metrics to parity with Scala API

2017-07-17 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089487#comment-16089487 ] Marco Gaido commented on SPARK-21340: - [~jake.charland] I submitted a PR but I am not sure it will be

[jira] [Commented] (SPARK-20990) Multi-line support for JSON

2017-07-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102975#comment-16102975 ] Marco Gaido commented on SPARK-20990: - A PR fixing it is ready:

[jira] [Commented] (SPARK-14516) Clustering evaluator

2017-06-30 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069815#comment-16069815 ] Marco Gaido commented on SPARK-14516: - Hello everybody, I have a proposal for a very efficient

[jira] [Commented] (SPARK-21658) Adds the default None for value in na.replace in PySpark to match

2017-08-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118097#comment-16118097 ] Marco Gaido commented on SPARK-21658: - [~viirya] ok, thanks. > Adds the default None for value in

[jira] [Commented] (SPARK-21658) Adds the default None for value in na.replace in PySpark to match

2017-08-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118040#comment-16118040 ] Marco Gaido commented on SPARK-21658: - Though, the documentation points out that there is no default

[jira] [Commented] (SPARK-21725) spark thriftserver insert overwrite table partition select

2017-08-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134424#comment-16134424 ] Marco Gaido commented on SPARK-21725: - [~zhangxin0112zx] I followed your instructions, but I am

[jira] [Commented] (SPARK-21772) HiveException unable to move results from srcf to destf in InsertIntoHiveTable

2017-08-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137260#comment-16137260 ] Marco Gaido commented on SPARK-21772: - can someone of the admin close this as 'Invalid' please? For

[jira] [Commented] (SPARK-21768) spark.csv.read Empty String Parsed as NULL when nullValue is Set

2017-08-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16132191#comment-16132191 ] Marco Gaido commented on SPARK-21768: - This is a duplicate of SPARK-17916. > spark.csv.read Empty

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

2017-06-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16051963#comment-16051963 ] Marco Gaido commented on SPARK-19909: - IMHO the best option to deal with this problem is to force the

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

2017-06-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049105#comment-16049105 ] Marco Gaido commented on SPARK-19909: - [~rvoyer] there is a workaround and it is easy: you have to

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

2017-09-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169023#comment-16169023 ] Marco Gaido commented on SPARK-22036: - Yes, it is only for multiplications. The reason is that for

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

2017-09-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168969#comment-16168969 ] Marco Gaido commented on SPARK-22036: - This happens because there is an overflow in the operation. I

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

2017-09-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169074#comment-16169074 ] Marco Gaido commented on SPARK-22036: - Maybe the "bad" part is that by default spark creates the

[jira] [Commented] (SPARK-22040) current_date function with timezone id

2017-09-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169124#comment-16169124 ] Marco Gaido commented on SPARK-22040: - May I work on this? > current_date function with timezone id

[jira] [Commented] (SPARK-22036) BigDecimal multiplication sometimes returns null

2017-09-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169059#comment-16169059 ] Marco Gaido commented on SPARK-22036: - Honestly I don't know, that is why I said that I don't know

[jira] [Created] (SPARK-22215) Add a configuration parameter to set max size for generated classes

2017-10-06 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22215: --- Summary: Add a configuration parameter to set max size for generated classes Key: SPARK-22215 URL: https://issues.apache.org/jira/browse/SPARK-22215 Project: Spark

[jira] [Commented] (SPARK-22226) Code generation fails for dataframes with 10000 columns

2017-10-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197111#comment-16197111 ] Marco Gaido commented on SPARK-6: - I am not sure about what the current open PR is going to

[jira] [Created] (SPARK-22226) Code generation fails for dataframes with 10000 columns

2017-10-09 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-6: --- Summary: Code generation fails for dataframes with 1 columns Key: SPARK-6 URL: https://issues.apache.org/jira/browse/SPARK-6 Project: Spark Issue

[jira] [Commented] (SPARK-22220) Spark SQL: LATERAL VIEW OUTER null pointer exception with GROUP BY

2017-10-10 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-0?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198295#comment-16198295 ] Marco Gaido commented on SPARK-0: - Please may you provide some sample data and easy code to

[jira] [Commented] (SPARK-22226) Code generation fails for dataframes with 10000 columns

2017-10-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197036#comment-16197036 ] Marco Gaido commented on SPARK-6: - [~srowen] I know that there are many ticket for this, but I

[jira] [Commented] (SPARK-22226) Code generation fails for dataframes with 10000 columns

2017-10-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197254#comment-16197254 ] Marco Gaido commented on SPARK-6: - [~kiszk] I am not sure that the PR you mentioned solves the

[jira] [Updated] (SPARK-22226) splitExpression can create too many method calls (generating a Constant Pool limit error)

2017-10-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-6: Summary: splitExpression can create too many method calls (generating a Constant Pool limit error)

[jira] [Commented] (SPARK-22226) splitExpression can create too many method calls (generating a Constant Pool limit error)

2017-10-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197286#comment-16197286 ] Marco Gaido commented on SPARK-6: - Exactly [~kiszk], sorry for the bad initial title of the JIRA.

[jira] [Reopened] (SPARK-22226) splitExpression can create too many method calls (generating a Constant Pool limit error)

2017-10-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido reopened SPARK-6: - > splitExpression can create too many method calls (generating a Constant Pool > limit error) >

[jira] [Comment Edited] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158406#comment-16158406 ] Marco Gaido edited comment on SPARK-21944 at 9/8/17 9:57 AM: - [~KevinZwx] you

[jira] [Comment Edited] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158406#comment-16158406 ] Marco Gaido edited comment on SPARK-21944 at 9/8/17 10:31 AM: -- [~KevinZwx]

[jira] [Commented] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158406#comment-16158406 ] Marco Gaido commented on SPARK-21944: - [~kevinzhang] you should define the watermark on the column

[jira] [Created] (SPARK-21957) Add current_user function

2017-09-08 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-21957: --- Summary: Add current_user function Key: SPARK-21957 URL: https://issues.apache.org/jira/browse/SPARK-21957 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-09-06 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155047#comment-16155047 ] Marco Gaido commented on SPARK-21918: - What I meant is that if we want to support doAs, we shouldn't

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-09-06 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155162#comment-16155162 ] Marco Gaido commented on SPARK-21918: - Yes, I think this would be great, thanks. > HiveClient

[jira] [Commented] (SPARK-21888) Cannot add stuff to Client Classpath for Yarn Cluster Mode

2017-09-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153888#comment-16153888 ] Marco Gaido commented on SPARK-21888: - [~tgraves] Sorry, I misread. Of course, this doesn't add it to

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-09-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154033#comment-16154033 ] Marco Gaido commented on SPARK-21918: - What do you mean by "works correctly"? Actually all the jobs

[jira] [Commented] (SPARK-21944) Watermark on window column is wrong

2017-09-07 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157091#comment-16157091 ] Marco Gaido commented on SPARK-21944: - May you please provide some sample data to reproduce the

[jira] [Commented] (SPARK-21938) Spark partial CSV write fails silently

2017-09-06 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156118#comment-16156118 ] Marco Gaido commented on SPARK-21938: - It would be helpful if you can post a sample code to reproduce

[jira] [Commented] (SPARK-21981) Python API for ClusteringEvaluator

2017-09-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162764#comment-16162764 ] Marco Gaido commented on SPARK-21981: - [~yanboliang] yes, thanks. I will post a PR asap, thank you.

[jira] [Created] (SPARK-22119) Add cosine distance to KMeans

2017-09-25 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22119: --- Summary: Add cosine distance to KMeans Key: SPARK-22119 URL: https://issues.apache.org/jira/browse/SPARK-22119 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-22040) current_date function with timezone id

2017-10-02 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-22040. - Resolution: Invalid > current_date function with timezone id >

[jira] [Commented] (SPARK-21905) ClassCastException when call sqlContext.sql on temp table

2017-09-04 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152955#comment-16152955 ] Marco Gaido commented on SPARK-21905: - This is likely to be caused by a bug in the Magellan package.

[jira] [Commented] (SPARK-21888) Cannot add stuff to Client Classpath for Yarn Cluster Mode

2017-09-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151128#comment-16151128 ] Marco Gaido commented on SPARK-21888: - It is enough to add {{hbase-site.xml}} using {{--files}} in

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-09-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153672#comment-16153672 ] Marco Gaido commented on SPARK-21918: - hive.server2.enable.doAs=true is currently not supported in

[jira] [Comment Edited] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2017-09-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153672#comment-16153672 ] Marco Gaido edited comment on SPARK-21918 at 9/5/17 1:54 PM: -

[jira] [Commented] (SPARK-22220) Spark SQL: LATERAL VIEW OUTER null pointer exception with GROUP BY

2017-10-07 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-0?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195630#comment-16195630 ] Marco Gaido commented on SPARK-0: - Your version is quite old and Spark 1.6 is no longer

[jira] [Created] (SPARK-22146) FileNotFoundException while reading ORC files containing '%'

2017-09-27 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22146: --- Summary: FileNotFoundException while reading ORC files containing '%' Key: SPARK-22146 URL: https://issues.apache.org/jira/browse/SPARK-22146 Project: Spark

[jira] [Commented] (SPARK-22146) FileNotFoundException while reading ORC files containing '%'

2017-09-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182650#comment-16182650 ] Marco Gaido commented on SPARK-22146: - If you look carefully at the file which Spark is looking for,

[jira] [Comment Edited] (SPARK-22146) FileNotFoundException while reading ORC files containing '%'

2017-09-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182686#comment-16182686 ] Marco Gaido edited comment on SPARK-22146 at 9/27/17 2:58 PM: -- Yes, that is

[jira] [Commented] (SPARK-22146) FileNotFoundException while reading ORC files containing '%'

2017-09-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182686#comment-16182686 ] Marco Gaido commented on SPARK-22146: - Yes, that is a local file and I am running `spark-shell`

[jira] [Comment Edited] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-10-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210898#comment-16210898 ] Marco Gaido edited comment on SPARK-20617 at 10/19/17 11:43 AM: This is

[jira] [Comment Edited] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-10-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210898#comment-16210898 ] Marco Gaido edited comment on SPARK-20617 at 10/19/17 11:43 AM: This is

[jira] [Commented] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-10-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210898#comment-16210898 ] Marco Gaido commented on SPARK-20617: - This is not a bug. This is the right and expected behavior

[jira] [Resolved] (SPARK-20617) pyspark.sql filtering fails when using ~isin when there are nulls in column

2017-10-19 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-20617. - Resolution: Not A Bug > pyspark.sql filtering fails when using ~isin when there are nulls in

[jira] [Created] (SPARK-22301) Add rule to Optimizer for In with empty list of values

2017-10-17 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22301: --- Summary: Add rule to Optimizer for In with empty list of values Key: SPARK-22301 URL: https://issues.apache.org/jira/browse/SPARK-22301 Project: Spark Issue

[jira] [Created] (SPARK-22520) Support code generation also for complex CASE WHEN

2017-11-14 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22520: --- Summary: Support code generation also for complex CASE WHEN Key: SPARK-22520 URL: https://issues.apache.org/jira/browse/SPARK-22520 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-11-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268602#comment-16268602 ] Marco Gaido commented on SPARK-19268: - In my case, deleting `_spark_metadata` solved the issue. Thus

[jira] [Created] (SPARK-22635) FileNotFoundException again while reading ORC files containing special characters

2017-11-28 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22635: --- Summary: FileNotFoundException again while reading ORC files containing special characters Key: SPARK-22635 URL: https://issues.apache.org/jira/browse/SPARK-22635

[jira] [Commented] (SPARK-22627) Fix formatting of headers in configuration.html page

2017-11-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268712#comment-16268712 ] Marco Gaido commented on SPARK-22627: - This should be fixed by SPARK-19106. I think it is a

[jira] [Resolved] (SPARK-22631) Consolidate all configuration properties into one page

2017-11-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-22631. - Resolution: Duplicate > Consolidate all configuration properties into one page >

[jira] [Commented] (SPARK-22627) Fix formatting of headers in configuration.html page

2017-11-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268869#comment-16268869 ] Marco Gaido commented on SPARK-22627: - [~srowen] the issue seems not present anymore on branch-2.2 (I

[jira] [Resolved] (SPARK-22609) Reuse CodeGeneration.nullSafeExec when possible

2017-11-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-22609. - Resolution: Invalid > Reuse CodeGeneration.nullSafeExec when possible >

[jira] [Closed] (SPARK-22609) Reuse CodeGeneration.nullSafeExec when possible

2017-11-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido closed SPARK-22609. --- > Reuse CodeGeneration.nullSafeExec when possible > --- > >

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

2017-11-22 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262395#comment-16262395 ] Marco Gaido commented on SPARK-22575: - You can use `UNCACHE TABLE` to remove them from cache if you

[jira] [Commented] (SPARK-22582) Spark SQL round throws error with negative precision

2017-11-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264582#comment-16264582 ] Marco Gaido commented on SPARK-22582: - I tried to run {code} spark.sql("select round(100.1 , 1) as

[jira] [Created] (SPARK-22609) Reuse CodeGeneration.nullSafeExec when possible

2017-11-26 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22609: --- Summary: Reuse CodeGeneration.nullSafeExec when possible Key: SPARK-22609 URL: https://issues.apache.org/jira/browse/SPARK-22609 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-11-27 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266544#comment-16266544 ] Marco Gaido commented on SPARK-19268: - [~zsxwing] I am hitting this too and I am running 2.2.0. My

[jira] [Created] (SPARK-22684) Avoid the generation of useless mutable states by datetime functions

2017-12-04 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22684: --- Summary: Avoid the generation of useless mutable states by datetime functions Key: SPARK-22684 URL: https://issues.apache.org/jira/browse/SPARK-22684 Project: Spark

[jira] [Created] (SPARK-22669) Avoid unnecessary function calls in code generation

2017-12-01 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22669: --- Summary: Avoid unnecessary function calls in code generation Key: SPARK-22669 URL: https://issues.apache.org/jira/browse/SPARK-22669 Project: Spark Issue

[jira] [Created] (SPARK-22693) Avoid the generation of useless mutable states in complexTypeCreator and predicates

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22693: --- Summary: Avoid the generation of useless mutable states in complexTypeCreator and predicates Key: SPARK-22693 URL: https://issues.apache.org/jira/browse/SPARK-22693

[jira] [Created] (SPARK-22698) Avoid the generation of useless mutable states by GenerateUnsafeProjection

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22698: --- Summary: Avoid the generation of useless mutable states by GenerateUnsafeProjection Key: SPARK-22698 URL: https://issues.apache.org/jira/browse/SPARK-22698 Project:

[jira] [Created] (SPARK-22699) Avoid the generation of useless mutable states by GenerateSafeProjection

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22699: --- Summary: Avoid the generation of useless mutable states by GenerateSafeProjection Key: SPARK-22699 URL: https://issues.apache.org/jira/browse/SPARK-22699 Project:

[jira] [Created] (SPARK-22697) Avoid the generation of useless mutable states by GenerateMutableProjection

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22697: --- Summary: Avoid the generation of useless mutable states by GenerateMutableProjection Key: SPARK-22697 URL: https://issues.apache.org/jira/browse/SPARK-22697 Project:

[jira] [Created] (SPARK-22694) Avoid the generation of useless mutable states by regexp functions

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22694: --- Summary: Avoid the generation of useless mutable states by regexp functions Key: SPARK-22694 URL: https://issues.apache.org/jira/browse/SPARK-22694 Project: Spark

[jira] [Created] (SPARK-22692) Reduce the number of generated mutable states

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22692: --- Summary: Reduce the number of generated mutable states Key: SPARK-22692 URL: https://issues.apache.org/jira/browse/SPARK-22692 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-22684) Avoid the generation of useless mutable states by datetime functions

2017-12-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-22684: Issue Type: Sub-task (was: Bug) Parent: SPARK-22692 > Avoid the generation of useless

[jira] [Updated] (SPARK-22696) Avoid the generation of useless mutable states by objects functions

2017-12-05 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-22696: Summary: Avoid the generation of useless mutable states by objects functions (was: void the

[jira] [Created] (SPARK-22695) Avoid the generation of useless mutable states by scalaUDF

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22695: --- Summary: Avoid the generation of useless mutable states by scalaUDF Key: SPARK-22695 URL: https://issues.apache.org/jira/browse/SPARK-22695 Project: Spark

[jira] [Created] (SPARK-22696) void the generation of useless mutable states by objects functions

2017-12-05 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22696: --- Summary: void the generation of useless mutable states by objects functions Key: SPARK-22696 URL: https://issues.apache.org/jira/browse/SPARK-22696 Project: Spark

[jira] [Commented] (SPARK-22806) Window Aggregate functions: unexpected result at ordered partition

2017-12-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293716#comment-16293716 ] Marco Gaido commented on SPARK-22806: - This is the right behavior. Also Postgres works like this. if

[jira] [Resolved] (SPARK-22806) Window Aggregate functions: unexpected result at ordered partition

2017-12-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-22806. - Resolution: Invalid > Window Aggregate functions: unexpected result at ordered partition >

[jira] [Commented] (SPARK-22793) Memory leak in Spark Thrift Server

2017-12-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292213#comment-16292213 ] Marco Gaido commented on SPARK-22793: - Have you tried if the problem still exists in current master

[jira] [Commented] (SPARK-22799) Bucketizer should throw exception if single- and multi-column params are both set

2017-12-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292540#comment-16292540 ] Marco Gaido commented on SPARK-22799: - may I work on this? > Bucketizer should throw exception if

[jira] [Commented] (SPARK-22773) Empty arrays are not equal after transformation

2017-12-13 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289491#comment-16289491 ] Marco Gaido commented on SPARK-22773: - It is not a bug, since {res} is not an empty array, but it

[jira] [Commented] (SPARK-22752) FileNotFoundException while reading from Kafka

2017-12-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290659#comment-16290659 ] Marco Gaido commented on SPARK-22752: - thanks [~zsxwing]. You are right. I am closing this as

[jira] [Resolved] (SPARK-22752) FileNotFoundException while reading from Kafka

2017-12-14 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-22752. - Resolution: Duplicate > FileNotFoundException while reading from Kafka >

[jira] [Commented] (SPARK-22841) Select regexp_extract from table with where clause having is null throws indexoutofbounds exception

2017-12-20 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298134#comment-16298134 ] Marco Gaido commented on SPARK-22841: - I am not able to reproduce on current master. Can you try and

[jira] [Commented] (SPARK-22516) CSV Read breaks: When "multiLine" = "true", if "comment" option is set as last line's first character

2017-11-17 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257088#comment-16257088 ] Marco Gaido commented on SPARK-22516: - not sure why but this is caused by the fact that your file

[jira] [Commented] (SPARK-22493) sql null checks for Double.NaN do not work

2017-11-10 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247660#comment-16247660 ] Marco Gaido commented on SPARK-22493: - `NaN` is not `null`. They are different things. If you want to

[jira] [Comment Edited] (SPARK-22493) sql null checks for Double.NaN do not work

2017-11-10 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247660#comment-16247660 ] Marco Gaido edited comment on SPARK-22493 at 11/10/17 3:39 PM: --- {{NaN}} is

[jira] [Created] (SPARK-22494) Coalesce and AtLeastNNonNulls can cause 64KB JVM bytecode limit exception

2017-11-10 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-22494: --- Summary: Coalesce and AtLeastNNonNulls can cause 64KB JVM bytecode limit exception Key: SPARK-22494 URL: https://issues.apache.org/jira/browse/SPARK-22494 Project:

[jira] [Commented] (SPARK-22516) CSV Read breaks: When "multiLine" = "true", if "comment" option is set as last line's first character

2017-11-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260904#comment-16260904 ] Marco Gaido commented on SPARK-22516: - [~crkumaresh24] I can't reproduce the issue with the new file

[jira] [Commented] (SPARK-22576) Spark SQL locate returns incorrect value when the start position is negative

2017-11-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261293#comment-16261293 ] Marco Gaido commented on SPARK-22576: - why do you expect locate to work like this and not as it is

[jira] [Commented] (SPARK-22576) Spark SQL locate returns incorrect value when the start position is negative

2017-11-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261347#comment-16261347 ] Marco Gaido commented on SPARK-22576: - I see, but this is SAP Sysbase. Why do you think Spark should

[jira] [Commented] (SPARK-22575) Making Spark Thrift Server clean up its cache

2017-11-21 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261557#comment-16261557 ] Marco Gaido commented on SPARK-22575: - does it happen because you are caching some tables and never

[jira] [Commented] (SPARK-22501) 64KB JVM bytecode limit problem with in

2017-11-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248983#comment-16248983 ] Marco Gaido commented on SPARK-22501: - [~kiszk] are you working on this or can I take it? > 64KB JVM

[jira] [Commented] (SPARK-22532) Spark SQL function 'drop_duplicates' throws error when passing in a column that is an element of a struct

2017-11-17 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257020#comment-16257020 ] Marco Gaido commented on SPARK-22532: - the reason is that `header.eventId.lo` is not a column name,

  1   2   3   4   5   >