[jira] [Created] (SPARK-21117) Built-in SQL Function Support - WIDTH_BUCKET

2017-06-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21117: --- Summary: Built-in SQL Function Support - WIDTH_BUCKET Key: SPARK-21117 URL: https://issues.apache.org/jira/browse/SPARK-21117 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-21133) HighlyCompressedMapStatus#writeExternal throws NPE

2017-06-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053091#comment-16053091 ] Yuming Wang commented on SPARK-21133: - I'll create a PR later. > HighlyCompressedMap

[jira] [Created] (SPARK-21133) HighlyCompressedMapStatus#writeExternal throws NPE

2017-06-18 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21133: --- Summary: HighlyCompressedMapStatus#writeExternal throws NPE Key: SPARK-21133 URL: https://issues.apache.org/jira/browse/SPARK-21133 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-21133) HighlyCompressedMapStatus#writeExternal throws NPE

2017-06-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21133: Comment: was deleted (was: I'll create a PR later.) > HighlyCompressedMapStatus#writeExternal thro

[jira] [Commented] (SPARK-21101) Error running Hive temporary UDTF on latest Spark 2.2

2017-06-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053145#comment-16053145 ] Yuming Wang commented on SPARK-21101: - [~dyzhou], Can you try to override https://gi

[jira] [Resolved] (SPARK-20931) Built-in SQL Function ABS support string type

2017-06-19 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-20931. - Resolution: Fixed Fix Version/s: 2.3.0 > Built-in SQL Function ABS support string type > -

[jira] [Resolved] (SPARK-20951) Built-in SQL Function FormatNumber should validate input value

2017-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-20951. - Resolution: Won't Fix > Built-in SQL Function FormatNumber should validate input value >

[jira] [Commented] (SPARK-20466) HadoopRDD#addLocalConfiguration throws NPE

2017-06-20 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056985#comment-16056985 ] Yuming Wang commented on SPARK-20466: - [~kellyzly] How to reproduce it? > HadoopRDD#

[jira] [Created] (SPARK-21205) pmod(number, 0) should be null

2017-06-24 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21205: --- Summary: pmod(number, 0) should be null Key: SPARK-21205 URL: https://issues.apache.org/jira/browse/SPARK-21205 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2017-06-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062368#comment-16062368 ] Yuming Wang commented on SPARK-21067: - Are you enable [HDFS Federation|http://hadoop

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

2017-06-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16062370#comment-16062370 ] Yuming Wang commented on SPARK-21063: - May be you should do this before submit your S

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

2017-06-26 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064007#comment-16064007 ] Yuming Wang commented on SPARK-21063: - [~pbykov], I just verified it. It can get the

[jira] [Commented] (SPARK-21246) Unexpected Data Type conversion from LONG to BIGINT

2017-06-28 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16067620#comment-16067620 ] Yuming Wang commented on SPARK-21246: - {{Seq(3)}} should be {{Seq(3L)}}, This works f

[jira] [Created] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21253: --- Summary: Cannot fetch big blocks to disk Key: SPARK-21253 URL: https://issues.apache.org/jira/browse/SPARK-21253 Project: Spark Issue Type: Bug Comp

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Attachment: ui-thread-dump-jqhadoop221-154.gif > Cannot fetch big blocks to disk > ---

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068232#comment-16068232 ] Yuming Wang commented on SPARK-21253: - It may be hang for a {{spark-sql}} application

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with {{spark.r

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with {{spark.r

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069109#comment-16069109 ] Yuming Wang commented on SPARK-21253: - I checked it, all jars are latest 2.2.0-rcX.

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with {{spark.r

[jira] [Created] (SPARK-21269) MetadataFetchFailedException: Missing an output location for shuffle 0

2017-06-30 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21269: --- Summary: MetadataFetchFailedException: Missing an output location for shuffle 0 Key: SPARK-21269 URL: https://issues.apache.org/jira/browse/SPARK-21269 Project: Spark

[jira] [Updated] (SPARK-21269) MetadataFetchFailedException: Missing an output location for shuffle 0

2017-06-30 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21269: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with {{spark.r

[jira] [Created] (SPARK-21574) set hive.exec.max.dynamic.partitions lose effect

2017-07-29 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21574: --- Summary: set hive.exec.max.dynamic.partitions lose effect Key: SPARK-21574 URL: https://issues.apache.org/jira/browse/SPARK-21574 Project: Spark Issue Type: Bu

[jira] [Created] (SPARK-21625) sqrt(negative number) should be null

2017-08-03 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21625: --- Summary: sqrt(negative number) should be null Key: SPARK-21625 URL: https://issues.apache.org/jira/browse/SPARK-21625 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-21625) sqrt(negative number) should be null

2017-08-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21625: Description: Both Hive and MySQL are null: {code:sql} hive> select SQRT(-10.0); OK NULL Time taken

[jira] [Commented] (SPARK-21625) sqrt(negative number) should be null

2017-08-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112454#comment-16112454 ] Yuming Wang commented on SPARK-21625: - [~panbingkun] Yes, This is Hive's logic: {cod

[jira] [Created] (SPARK-21635) ACOS(2) and ASIN(2) should be null

2017-08-03 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21635: --- Summary: ACOS(2) and ASIN(2) should be null Key: SPARK-21635 URL: https://issues.apache.org/jira/browse/SPARK-21635 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-16625) Oracle JDBC table creation fails with ORA-00902: invalid datatype

2016-07-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395307#comment-15395307 ] Yuming Wang commented on SPARK-16625: - {code:java} val jdbcUrl = "jdbc:oracle:thi

[jira] [Created] (SPARK-27054) Remove Calcite dependency

2019-03-04 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27054: --- Summary: Remove Calcite dependency Key: SPARK-27054 URL: https://issues.apache.org/jira/browse/SPARK-27054 Project: Spark Issue Type: Improvement Com

[jira] [Created] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27074: --- Summary: Refactor HiveClientImpl runHive Key: SPARK-27074 URL: https://issues.apache.org/jira/browse/SPARK-27074 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add ReExe

[jira] [Updated] (SPARK-27074) Refactor HiveClientImpl runHive

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27074: Description: Hive 3.1.1's {{CommandProcessor}} have 2 changes: # HIVE-17626(Hive 3.0.0) add ReExe

[jira] [Commented] (SPARK-27077) DataFrameReader and Number of Connection Limitation

2019-03-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786228#comment-16786228 ] Yuming Wang commented on SPARK-27077: - Could you try to set {{numPartitions}} please

[jira] [Created] (SPARK-27078) Read Hive materialized view throw MatchError

2019-03-06 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27078: --- Summary: Read Hive materialized view throw MatchError Key: SPARK-27078 URL: https://issues.apache.org/jira/browse/SPARK-27078 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23749) Workaround built-in Hive api changes (phase 1)

2019-03-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Summary: Workaround built-in Hive api changes (phase 1) (was: Avoid Hive.get() to compatible with

[jira] [Updated] (SPARK-23749) Workaround built-in Hive api changes (phase 1)

2019-03-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Description: (was: {noformat} 18/03/15 22:34:46 WARN Hive: Failed to register all functions. or

[jira] [Updated] (SPARK-23749) Workaround built-in Hive api changes (phase 1)

2019-03-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Affects Version/s: (was: 2.4.0) 3.0.0 > Workaround built-in Hive api ch

[jira] [Commented] (SPARK-23012) Support for predicate pushdown and partition pruning when left joining large Hive tables

2019-03-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787485#comment-16787485 ] Yuming Wang commented on SPARK-23012: - [~Saurabh Santhosh] Could you try Spark 2.4.0

[jira] [Commented] (SPARK-27100) dag-scheduler-event-loop" java.lang.StackOverflowError

2019-03-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787530#comment-16787530 ] Yuming Wang commented on SPARK-27100: - Could you try the latest Spark version please

[jira] [Commented] (SPARK-26004) InMemoryTable support StartsWith predicate push down

2019-03-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788643#comment-16788643 ] Yuming Wang commented on SPARK-26004: - [~maropu] Could you help close this ticket?

[jira] [Created] (SPARK-27118) Upgrade to latest Hive version for Hive Metastore Client 1.1 and 1.0

2019-03-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27118: --- Summary: Upgrade to latest Hive version for Hive Metastore Client 1.1 and 1.0 Key: SPARK-27118 URL: https://issues.apache.org/jira/browse/SPARK-27118 Project: Spark

[jira] [Created] (SPARK-27120) Upgrade scalatest version to 3.0.5

2019-03-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27120: --- Summary: Upgrade scalatest version to 3.0.5 Key: SPARK-27120 URL: https://issues.apache.org/jira/browse/SPARK-27120 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-12 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27130: --- Summary: Automatically select profile when executing sbt-checkstyle Key: SPARK-27130 URL: https://issues.apache.org/jira/browse/SPARK-27130 Project: Spark Issu

[jira] [Created] (SPARK-27158) dev/mima and dev/scalastyle support dynamic profiles

2019-03-14 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27158: --- Summary: dev/mima and dev/scalastyle support dynamic profiles Key: SPARK-27158 URL: https://issues.apache.org/jira/browse/SPARK-27158 Project: Spark Issue Type

[jira] [Updated] (SPARK-23749) Replace built-in Hive API (isSub/toKryo) and remove OrcProto.Type usage

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23749: Summary: Replace built-in Hive API (isSub/toKryo) and remove OrcProto.Type usage (was: Workaround

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Summary: Upgrade the built-in Hive to 2.3.4 for hadoop-3.1 (was: Upgrade built-in Hive to 2.3.4)

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: Upgrade built-in Hive to 2.3.4 for Hadoop-3.1(Please note that this upgrade only for

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: Upgrade built-in Hive to 2.3.4 for Hadoop-3.1(Please note that this upgrade only for

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Description: Spark fail to run on Hadoop 3.x, because Hive's ShimLoader considers Hadoop 3.x to b

[jira] [Updated] (SPARK-27054) Remove Calcite dependency

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27054: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Remove Calcite dependency >

[jira] [Updated] (SPARK-27130) Automatically select profile when executing sbt-checkstyle

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27130: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Automatically select profil

[jira] [Updated] (SPARK-23710) Upgrade the built-in Hive to 2.3.4 for hadoop-3.1

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23710: Issue Type: Umbrella (was: Improvement) > Upgrade the built-in Hive to 2.3.4 for hadoop-3.1 > ---

[jira] [Updated] (SPARK-27158) dev/mima and dev/scalastyle support dynamic profiles

2019-03-14 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27158: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > dev/mima and dev/scalastyle

[jira] [Commented] (SPARK-27173) For hive parquet table,codes(lz4,brotli,zstd) are not available

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793472#comment-16793472 ] Yuming Wang commented on SPARK-27173: - Yes. We are upgrading the built-in Hive now.

[jira] [Created] (SPARK-27175) Upgrade hadoop-3 to 3.2.0

2019-03-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27175: --- Summary: Upgrade hadoop-3 to 3.2.0 Key: SPARK-27175 URL: https://issues.apache.org/jira/browse/SPARK-27175 Project: Spark Issue Type: Improvement Com

[jira] [Created] (SPARK-27176) Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

2019-03-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27176: --- Summary: Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4 Key: SPARK-27176 URL: https://issues.apache.org/jira/browse/SPARK-27176 Project: Spark I

[jira] [Created] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27177: --- Summary: Update jenkins locale to en_US.UTF-8 Key: SPARK-27177 URL: https://issues.apache.org/jira/browse/SPARK-27177 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27177: Description: Two test cases will failed on our jenkins since HADOOP-12045(Hadoop-2.8.0). I'd like

[jira] [Commented] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793698#comment-16793698 ] Yuming Wang commented on SPARK-27177: - cc [~shaneknapp] [~srowen] > Update jenkins

[jira] [Commented] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794035#comment-16794035 ] Yuming Wang commented on SPARK-27177: - I’d prefer to PRB + sbt builds. > Update jen

[jira] [Created] (SPARK-27179) Exclude javax.ws.rs:jsr311-api from hadoop-client

2019-03-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27179: --- Summary: Exclude javax.ws.rs:jsr311-api from hadoop-client Key: SPARK-27179 URL: https://issues.apache.org/jira/browse/SPARK-27179 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-27179) Exclude javax.ws.rs:jsr311-api from hadoop-client

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27179: Description: Since YARN-7113(Hadoop-3.1.0), hadoop-client add javax.ws.rs:jsr311-api to its depend

[jira] [Updated] (SPARK-27179) Exclude javax.ws.rs:jsr311-api from hadoop-client

2019-03-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27179: Environment: (was: Since YARN-7113(Hadoop-3.1.0), hadoop-client add javax.ws.rs:jsr311-api to

[jira] [Created] (SPARK-27180) Fix testing issues with yarn module in Hadoop-3

2019-03-16 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27180: --- Summary: Fix testing issues with yarn module in Hadoop-3 Key: SPARK-27180 URL: https://issues.apache.org/jira/browse/SPARK-27180 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-21918) HiveClient shouldn't share Hive object between different thread

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794382#comment-16794382 ] Yuming Wang commented on SPARK-21918: - DDL should be supported after the metastore i

[jira] [Commented] (SPARK-21395) Spark SQL hive-thriftserver doesn't register operation log before execute sql statement

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794383#comment-16794383 ] Yuming Wang commented on SPARK-21395: - Fixed by https://issues.apache.org/jira/brows

[jira] [Resolved] (SPARK-19412) HiveContext hive queries differ from CLI hive queries

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-19412. - Resolution: Invalid > HiveContext hive queries differ from CLI hive queries > --

[jira] [Updated] (SPARK-27175) Upgrade hadoop-3 to 3.2.0

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27175: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Upgrade hadoop-3 to 3.2.0 >

[jira] [Commented] (SPARK-23749) Replace built-in Hive API (isSub/toKryo) and remove OrcProto.Type usage

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794396#comment-16794396 ] Yuming Wang commented on SPARK-23749: - [~dongjoon] Could we update the status? > Re

[jira] [Updated] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27177: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Update jenkins locale to en

[jira] [Updated] (SPARK-27179) Exclude javax.ws.rs:jsr311-api from hadoop-client

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27179: Issue Type: Sub-task (was: Bug) Parent: SPARK-23710 > Exclude javax.ws.rs:jsr311-api from

[jira] [Updated] (SPARK-27180) Fix testing issues with yarn module in Hadoop-3

2019-03-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27180: Issue Type: Sub-task (was: Improvement) Parent: SPARK-23710 > Fix testing issues with yar

[jira] [Created] (SPARK-27182) Move the conflict source code of the sql/core module to sql/core/v1.2.1

2019-03-17 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27182: --- Summary: Move the conflict source code of the sql/core module to sql/core/v1.2.1 Key: SPARK-27182 URL: https://issues.apache.org/jira/browse/SPARK-27182 Project: Spark

[jira] [Commented] (SPARK-27186) Optimize SortShuffleWriter writing process

2019-03-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16794881#comment-16794881 ] Yuming Wang commented on SPARK-27186: - Please avoid to set {{Fix Version/s}} which i

[jira] [Updated] (SPARK-27186) Optimize SortShuffleWriter writing process

2019-03-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27186: Fix Version/s: (was: 3.0.0) > Optimize SortShuffleWriter writing process > ---

[jira] [Commented] (SPARK-27191) union of dataframes depends on order of the columns in 2.4.0

2019-03-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795018#comment-16795018 ] Yuming Wang commented on SPARK-27191: - cc [~viirya] > union of dataframes depends o

[jira] [Created] (SPARK-27226) Reduce the code duplicate

2019-03-21 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27226: --- Summary: Reduce the code duplicate Key: SPARK-27226 URL: https://issues.apache.org/jira/browse/SPARK-27226 Project: Spark Issue Type: Sub-task Compon

[jira] [Updated] (SPARK-27226) Reduce the code duplicate when upgrading built-in Hive

2019-03-21 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27226: Summary: Reduce the code duplicate when upgrading built-in Hive (was: Reduce the code duplicate)

[jira] [Commented] (SPARK-27177) Update jenkins locale to en_US.UTF-8

2019-03-22 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798989#comment-16798989 ] Yuming Wang commented on SPARK-27177: - It works. Thank you [~shaneknapp] > Update j

[jira] [Commented] (SPARK-27264) spark sql released all executor but the job is not done

2019-03-24 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800351#comment-16800351 ] Yuming Wang commented on SPARK-27264: - Data skew? > spark sql released all executor

[jira] [Commented] (SPARK-27267) spark 2.4 use 1.1.7.x snappy-java, but its behavior is different from 1.1.2.x

2019-03-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16801316#comment-16801316 ] Yuming Wang commented on SPARK-27267: - [~max2049] (y) > spark 2.4 use 1.1.7.x sna

[jira] [Commented] (SPARK-27267) spark 2.4 use 1.1.7.x snappy-java, but its behavior is different from 1.1.2.x

2019-03-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16801396#comment-16801396 ] Yuming Wang commented on SPARK-27267: - Thank you [~taroleo]. @[~max2049] Could you 

[jira] [Commented] (SPARK-18262) JSON.org license is now CatX

2019-03-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802719#comment-16802719 ] Yuming Wang commented on SPARK-18262: - [~srowen] I'm not sure do we need exclude {{o

[jira] [Commented] (SPARK-18262) JSON.org license is now CatX

2019-03-27 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802752#comment-16802752 ] Yuming Wang commented on SPARK-18262: - Yes. It only happens when running the Hive te

[jira] [Created] (SPARK-27349) Dealing with TimeVars removed in Hive 2.x

2019-04-02 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27349: --- Summary: Dealing with TimeVars removed in Hive 2.x Key: SPARK-27349 URL: https://issues.apache.org/jira/browse/SPARK-27349 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-27349) Dealing with TimeVars removed in Hive 2.x

2019-04-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27349: Description: {{hive.stats.jdbc.timeout}} and {{hive.stats.retries.wait}}  were removed by HIVE-1216

[jira] [Updated] (SPARK-27349) Dealing with TimeVars removed in Hive 2.x

2019-04-02 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27349: Description: {{hive.stats.jdbc.timeout}} and {{hive.stats.retries.wait}}  were removed by HIVE-1216

[jira] [Created] (SPARK-27354) Add a new empty hive-thriftserver module for Hive 2.3.4

2019-04-03 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27354: --- Summary: Add a new empty hive-thriftserver module for Hive 2.3.4 Key: SPARK-27354 URL: https://issues.apache.org/jira/browse/SPARK-27354 Project: Spark Issue

[jira] [Created] (SPARK-27383) Avoid using hard-coded jar names in Hive tests

2019-04-03 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27383: --- Summary: Avoid using hard-coded jar names in Hive tests Key: SPARK-27383 URL: https://issues.apache.org/jira/browse/SPARK-27383 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-27383) Avoid using hard-coded jar names in Hive tests

2019-04-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27383: Description: Avoid using hard-coded jar names({{hive-contrib-0.13.1.jar}}  and {{hive-hcatalog-core

[jira] [Commented] (SPARK-27176) Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

2019-04-05 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810652#comment-16810652 ] Yuming Wang commented on SPARK-27176: - For Hive 2.3.4, we also need {{hive-llap-comm

[jira] [Comment Edited] (SPARK-27176) Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4

2019-04-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810652#comment-16810652 ] Yuming Wang edited comment on SPARK-27176 at 4/6/19 2:37 PM: -

[jira] [Created] (SPARK-27402) Support HiveExternalCatalog backward compatibility test

2019-04-07 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27402: --- Summary: Support HiveExternalCatalog backward compatibility test Key: SPARK-27402 URL: https://issues.apache.org/jira/browse/SPARK-27402 Project: Spark Issue T

[jira] [Updated] (SPARK-27402) Support HiveExternalCatalog backward compatibility test

2019-04-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27402: Description: When we upgrade the built-in Hive to 2.3.4, the default spark.sql.hive.metastore.vers

[jira] [Updated] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

2019-04-07 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23831: Description: Add org.apache.derby to IsolatedClientLoader,otherwise it may throw an exception: {n

[jira] [Created] (SPARK-27441) Add read/write tests to Hive serde tables

2019-04-11 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27441: --- Summary: Add read/write tests to Hive serde tables Key: SPARK-27441 URL: https://issues.apache.org/jira/browse/SPARK-27441 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-27441) Add read/write tests to Hive serde tables

2019-04-11 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27441: Description: The versions between Hive, Parquet and ORC after the built-in Hive upgrade to 2.3.4:

[jira] [Created] (SPARK-27475) dev/deps/spark-deps-hadoop-3.2 is incorrect

2019-04-16 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27475: --- Summary: dev/deps/spark-deps-hadoop-3.2 is incorrect Key: SPARK-27475 URL: https://issues.apache.org/jira/browse/SPARK-27475 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27475) dev/deps/spark-deps-hadoop-3.2 is incorrect

2019-04-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27475: Description: parquet-hadoop-bundle-1.6.0.jar should be  parquet-hadoop-bundle-1.8.1.jar. (was: par

[jira] [Commented] (SPARK-27475) dev/deps/spark-deps-hadoop-3.2 is incorrect

2019-04-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819124#comment-16819124 ] Yuming Wang commented on SPARK-27475: - [~srowen] One way to fix it is to change [th

  1   2   3   4   5   6   7   8   9   10   >