[jira] [Updated] (SPARK-25230) Upper behaves incorrect for string contains "ß"

2018-08-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25230: Attachment: Oracle.png > Upper behaves incorrect for string contains "ß" >

[jira] [Updated] (SPARK-25230) Upper behaves incorrect for string contains "ß"

2018-08-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25230: Description: How to reproduce: {code:sql} spark-sql> SELECT upper('Haßler'); HASSLER {code}

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view

2018-08-26 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592890#comment-16592890 ] Yuming Wang commented on SPARK-25135: - Another serious case: {code:scala} withTempDir { dir =>

[jira] [Created] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-08-29 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25269: --- Summary: SQL interface support specify StorageLevel when cache table Key: SPARK-25269 URL: https://issues.apache.org/jira/browse/SPARK-25269 Project: Spark

[jira] [Commented] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-08-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595967#comment-16595967 ] Yuming Wang commented on SPARK-25269: - I'm working on. > SQL interface support specify StorageLevel

[jira] [Resolved] (SPARK-25259) Left/Right join support push down during-join predicates

2018-08-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25259. - Resolution: Duplicate > Left/Right join support push down during-join predicates >

[jira] [Commented] (SPARK-25259) Left/Right join support push down during-join predicates

2018-08-28 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594563#comment-16594563 ] Yuming Wang commented on SPARK-25259: - I'm working on. > Left/Right join support push down

[jira] [Created] (SPARK-25259) Left/Right join support push down during-join predicates

2018-08-28 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25259: --- Summary: Left/Right join support push down during-join predicates Key: SPARK-25259 URL: https://issues.apache.org/jira/browse/SPARK-25259 Project: Spark Issue

[jira] [Commented] (SPARK-25258) Upgrade kryo package to version 4.0.2+

2018-08-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594557#comment-16594557 ] Yuming Wang commented on SPARK-25258: - I have submitted PR: 

[jira] [Issue Comment Deleted] (SPARK-25259) Left/Right join support push down during-join predicates

2018-08-28 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25259: Comment: was deleted (was: I'm working on.) > Left/Right join support push down during-join

[jira] [Commented] (SPARK-25206) Wrong data may be returned when enable pushdown

2018-08-23 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589952#comment-16589952 ] Yuming Wang commented on SPARK-25206: - OK, I'll backport it. > Wrong data may be returned when

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546596#comment-16546596 ] Yuming Wang commented on SPARK-24828: - Please upload the a2_m2.parquet.zip. > Incompatible parquet

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547264#comment-16547264 ] Yuming Wang commented on SPARK-24828: - I'm working on > Incompatible parquet formats -

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547274#comment-16547274 ] Yuming Wang commented on SPARK-24828: - You can't read different data with a schema. > Incompatible

[jira] [Comment Edited] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547264#comment-16547264 ] Yuming Wang edited comment on SPARK-24828 at 7/18/18 1:27 AM: -- The

[jira] [Commented] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547423#comment-16547423 ] Yuming Wang commented on SPARK-24828: - You should read these files one by one:

[jira] [Updated] (SPARK-24828) Incompatible parquet formats - java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary

2018-07-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24828: Attachment: image-2018-07-18-13-57-21-148.png > Incompatible parquet formats -

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support {{repartitionByRange}} to improvement data pushdown. I have

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Attachment: DISTRIBUTE_BY_SORT_BY.png RANGE_DISTRIBUTE_BY_SORT_BY.png > SQL

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Summary: SQL interface support repartitionByRange (was: Improvement data pushdown by

[jira] [Commented] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544813#comment-16544813 ] Yuming Wang commented on SPARK-24816: - I'm working on. > SQL interface support repartitionByRange >

[jira] [Created] (SPARK-24816) Improvement data pushdown by repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-24816: --- Summary: Improvement data pushdown by repartitionByRange Key: SPARK-24816 URL: https://issues.apache.org/jira/browse/SPARK-24816 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support {{repartitionByRange}} to improvement data pushdown .I have

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support Improvement data pushdown by .I have test this feather with a

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support {{repartitionByRange}} to improvement data pushdown. I have

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support {{repartitionByRange}} to improvement data pushdown. I have

[jira] [Issue Comment Deleted] (SPARK-24816) SQL interface support repartitionByRange

2018-07-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Comment: was deleted (was: I'm working on.) > SQL interface support repartitionByRange >

[jira] [Updated] (SPARK-24816) SQL interface support repartitionByRange

2018-07-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24816: Description: SQL interface support {{repartitionByRange}} to improvement data pushdown. I have

[jira] [Commented] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

2018-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555028#comment-16555028 ] Yuming Wang commented on SPARK-24663: -

[jira] [Issue Comment Deleted] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

2018-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24663: Comment: was deleted (was:

[jira] [Commented] (SPARK-23622) Flaky Test: HiveClientSuites

2018-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555041#comment-16555041 ] Yuming Wang commented on SPARK-23622: -

[jira] [Comment Edited] (SPARK-23622) Flaky Test: HiveClientSuites

2018-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555041#comment-16555041 ] Yuming Wang edited comment on SPARK-23622 at 7/25/18 2:44 AM: --

[jira] [Commented] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

2018-07-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554990#comment-16554990 ] Yuming Wang commented on SPARK-24911: - Can we show non printable field delimiter when {{SHOW CREATE

[jira] [Resolved] (SPARK-11784) Support Timestamp filter pushdown in Parquet datasource

2018-07-19 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-11784. - Resolution: Duplicate > Support Timestamp filter pushdown in Parquet datasource >

[jira] [Commented] (SPARK-24766) CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats in parquet

2018-07-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543734#comment-16543734 ] Yuming Wang commented on SPARK-24766: - cc [~smilegator] [~cloud_fan] > CreateHiveTableAsSelect and

[jira] [Created] (SPARK-24916) Fix type coercion for IN expression with subquery

2018-07-25 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-24916: --- Summary: Fix type coercion for IN expression with subquery Key: SPARK-24916 URL: https://issues.apache.org/jira/browse/SPARK-24916 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24843) Spark2 job (in cluster mode) is unable to execute steps in HBase (error# java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/CompatibilityFactory)

2018-07-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16547674#comment-16547674 ] Yuming Wang commented on SPARK-24843: - It's not a Spark bug. Please make sure you classpath contains

[jira] [Issue Comment Deleted] (SPARK-25269) SQL interface support specify StorageLevel when cache table

2018-08-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25269: Comment: was deleted (was: I'm working on.) > SQL interface support specify StorageLevel when

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view on parquet

2018-08-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597742#comment-16597742 ] Yuming Wang commented on SPARK-25135: - [~dongjoon] orc has this issue. reproduce code: {code:scala}

[jira] [Created] (SPARK-25339) Refactor FilterPushdownBenchmark to use main method

2018-09-04 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25339: --- Summary: Refactor FilterPushdownBenchmark to use main method Key: SPARK-25339 URL: https://issues.apache.org/jira/browse/SPARK-25339 Project: Spark Issue

[jira] [Comment Edited] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605131#comment-16605131 ] Yuming Wang edited comment on SPARK-25330 at 9/6/18 1:09 AM: - I try to build

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605131#comment-16605131 ] Yuming Wang commented on SPARK-25330: - I try to build Hadoop 2.7.7

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605129#comment-16605129 ] Yuming Wang commented on SPARK-25330: - No. The issue occurred in this commit: 

[jira] [Comment Edited] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606638#comment-16606638 ] Yuming Wang edited comment on SPARK-25330 at 9/7/18 3:04 AM: - [~srowen] It

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606659#comment-16606659 ] Yuming Wang commented on SPARK-25330: - It affects Spark enable Hive support with a proxy user. >

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606638#comment-16606638 ] Yuming Wang commented on SPARK-25330: - [~srowen] It affects Hive. {{spark-sql}}, {{spark-shell}} and

[jira] [Commented] (SPARK-25368) Incorrect predicate pushdown returns wrong result

2018-09-08 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608079#comment-16608079 ] Yuming Wang commented on SPARK-25368: - I think the root cause is

[jira] [Created] (SPARK-25385) Upgrade fasterxml.jackson.databind.version to 2.7.8

2018-09-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25385: --- Summary: Upgrade fasterxml.jackson.databind.version to 2.7.8 Key: SPARK-25385 URL: https://issues.apache.org/jira/browse/SPARK-25385 Project: Spark Issue

[jira] [Updated] (SPARK-25385) Upgrade jackson version to 2.7.8

2018-09-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25385: Summary: Upgrade jackson version to 2.7.8 (was: Upgrade fasterxml.jackson.databind.version to

[jira] [Commented] (SPARK-25330) Permission issue after upgrade hadoop version to 2.7.7

2018-09-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605906#comment-16605906 ] Yuming Wang commented on SPARK-25330: - [~brahmareddy] Sorry. I didn't have script because we need a

[jira] [Resolved] (SPARK-25135) insert datasource table may all null when select from view on parquet

2018-09-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25135. - Resolution: Duplicate Fixed by SPARK-25313. > insert datasource table may all null when select

[jira] [Created] (SPARK-25294) Add integration test for Kerberos

2018-08-31 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25294: --- Summary: Add integration test for Kerberos Key: SPARK-25294 URL: https://issues.apache.org/jira/browse/SPARK-25294 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-25294) Add integration test for Kerberos

2018-08-31 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598290#comment-16598290 ] Yuming Wang commented on SPARK-25294: - cc [~srowen] What do you think? > Add integration test for

[jira] [Commented] (SPARK-23789) Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider

2018-08-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598259#comment-16598259 ] Yuming Wang commented on SPARK-23789: - Fixed by https://github.com/apache/spark/pull/20784 >

[jira] [Reopened] (SPARK-23789) Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider

2018-08-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-23789: - > Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider >

[jira] [Resolved] (SPARK-23789) Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider

2018-08-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-23789. - Resolution: Duplicate > Shouldn't set hive.metastore.uris before invoking

[jira] [Commented] (SPARK-25368) Incorrect predicate pushdown returns wrong result

2018-09-08 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608013#comment-16608013 ] Yuming Wang commented on SPARK-25368: - Can you try master, You provided test case can passed since 

[jira] [Commented] (SPARK-19145) Timestamp to String casting is slowing the query significantly

2018-09-04 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603937#comment-16603937 ] Yuming Wang commented on SPARK-19145: - How about we target this ticket and

[jira] [Commented] (SPARK-25339) Refactor FilterPushdownBenchmark to use main method

2018-09-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603984#comment-16603984 ] Yuming Wang commented on SPARK-25339: - cc [~dongjoon], I have created ticket. You can working on

[jira] [Commented] (SPARK-24474) Cores are left idle when there are a lot of tasks to run

2018-07-04 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532599#comment-16532599 ] Yuming Wang commented on SPARK-24474: - Can you try to set {{spark.locality.wait=0}}? > Cores are

[jira] [Created] (SPARK-24766) CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-24766: --- Summary: CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats Key: SPARK-24766 URL: https://issues.apache.org/jira/browse/SPARK-24766

[jira] [Updated] (SPARK-24766) CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24766: Description: How to reproduce: {code:java} INSERT OVERWRITE LOCAL DIRECTORY

[jira] [Updated] (SPARK-24766) CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24766: Description: How to reproduce: {code:java} INSERT OVERWRITE LOCAL DIRECTORY

[jira] [Updated] (SPARK-24766) CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats in parquet

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24766: Summary: CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats in

[jira] [Updated] (SPARK-24766) CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24766: Summary: CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats (was:

[jira] [Commented] (SPARK-24766) CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539420#comment-16539420 ] Yuming Wang commented on SPARK-24766: - It works after upgrade built-in Hive to 2.3.2 and upgrade

[jira] [Updated] (SPARK-24766) CreateHiveTableAsSelectCommand and InsertIntoHiveDirCommand won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24766: Description: How to reproduce: {code:java} INSERT OVERWRITE LOCAL DIRECTORY

[jira] [Comment Edited] (SPARK-24766) CreateHiveTableAsSelect and InsertIntoHiveDir won't generate decimal column stats

2018-07-10 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539420#comment-16539420 ] Yuming Wang edited comment on SPARK-24766 at 7/11/18 1:24 AM: -- It works

[jira] [Commented] (SPARK-22977) DataFrameWriter operations do not show details in SQL tab

2018-01-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331754#comment-16331754 ] Yuming Wang commented on SPARK-22977: - cc [~cloud_fan] > DataFrameWriter operations do not show

[jira] [Commented] (SPARK-25409) Speed up Spark History at start if there are tens of thousands of applications.

2018-09-11 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611409#comment-16611409 ] Yuming Wang commented on SPARK-25409: - Please create pull request from Github:

[jira] [Resolved] (SPARK-25385) Upgrade jackson version to 2.7.8

2018-09-11 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25385. - Resolution: Won't Fix > Upgrade jackson version to 2.7.8 > > >

[jira] [Commented] (SPARK-23659) Spark Job gets stuck during shuffle

2018-03-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396553#comment-16396553 ] Yuming Wang commented on SPARK-23659: - It may be is duplicated by

[jira] [Commented] (SPARK-23659) Spark Job gets stuck during shuffle

2018-03-12 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396332#comment-16396332 ] Yuming Wang commented on SPARK-23659: - Can you please check your {{netty-all}} version? > Spark Job

[jira] [Created] (SPARK-23640) Hadoop config may override spark config

2018-03-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-23640: --- Summary: Hadoop config may override spark config Key: SPARK-23640 URL: https://issues.apache.org/jira/browse/SPARK-23640 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23640) Hadoop config may override spark config

2018-03-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23640: Description: More details: https://github.com/apache/spark/pull/20785#issue-173982312 > Hadoop

[jira] [Commented] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430632#comment-16430632 ] Yuming Wang commented on SPARK-23906: - Duplicate with SPARK-20754's {{TRUNC}}? > Add UDF

[jira] [Issue Comment Deleted] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23906: Comment: was deleted (was: Duplicate with SPARK-20754's {{TRUNC}}?) > Add UDF trunc(numeric) >

[jira] [Commented] (SPARK-23946) 2.3.0 and Latest ScalaDocs are linked to the wrong source code

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431555#comment-16431555 ] Yuming Wang commented on SPARK-23946: - You are right. I'm working on this. > 2.3.0 and Latest

[jira] [Commented] (SPARK-23900) format_number udf should take user specifed format as argument

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430149#comment-16430149 ] Yuming Wang commented on SPARK-23900: - I'm work on this. > format_number udf should take user

[jira] [Updated] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2018-03-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23831: Description: How to reproduce: {noformat} sed

[jira] [Created] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2018-03-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-23831: --- Summary: Add org.apache.derby to IsolatedClientLoader Key: SPARK-23831 URL: https://issues.apache.org/jira/browse/SPARK-23831 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2018-03-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23831: Description: Add org.apache.derby to IsolatedClientLoader,otherwise it may throw an exception:

[jira] [Updated] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2018-03-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23831: Description: Add org.apache.derby to IsolatedClientLoader,otherwise it may throw an exception:

[jira] [Resolved] (SPARK-23789) Shouldn't set hive.metastore.uris before invoking HiveDelegationTokenProvider

2018-03-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-23789. - Resolution: Won't Fix > Shouldn't set hive.metastore.uris before invoking

[jira] [Commented] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

2018-03-28 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418430#comment-16418430 ] Yuming Wang commented on SPARK-23812: - [~wangtao93] Would you like to work on this? > DFS should be

[jira] [Commented] (SPARK-23790) proxy-user failed connecting to a kerberos configured metastore

2018-03-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412960#comment-16412960 ] Yuming Wang commented on SPARK-23790: - Can you try https://github.com/apache/spark/pull/20898? >

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402117#comment-16402117 ] Yuming Wang commented on SPARK-23710: - Hive materialized view may be supported after upgrading to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Created] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-23710: --- Summary: Upgrade Hive to 2.3.2 Key: SPARK-23710 URL: https://issues.apache.org/jira/browse/SPARK-23710 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402110#comment-16402110 ] Yuming Wang commented on SPARK-23710: - We don't need to consider Kryo's problem because {{Hive

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403886#comment-16403886 ] Yuming Wang commented on SPARK-23710: - Without {{-Phive}} works fine, because {{$SPARK_HOME/jars}} 

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Comment Edited] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402117#comment-16402117 ] Yuming Wang edited comment on SPARK-23710 at 3/17/18 12:37 AM: --- Hive

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

[jira] [Updated] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Description: {noformat} 18/03/15 22:34:46 WARN Hive: Failed to register all functions.

[jira] [Created] (SPARK-23749) Avoid Hive.get() to compatible with different Hive metastore

2018-03-20 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-23749: --- Summary: Avoid Hive.get() to compatible with different Hive metastore Key: SPARK-23749 URL: https://issues.apache.org/jira/browse/SPARK-23749 Project: Spark

[jira] [Updated] (SPARK-23710) Upgrade Hive to 2.3.2

2018-03-17 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: h1. Mainly changes * Maven dependency: hive.version from {{1.2.1.spark2}} to

<    1   2   3   4   5   6   7   8   9   10   >