[jira] [Assigned] (HIVE-24408) Upgrade Parquet to 1.11.1

2020-11-20 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24408: --- > Upgrade Parquet to 1.11.1 > - > > Key: HIVE-24408 >

[jira] [Updated] (HIVE-24379) Backport HIVE-19662 to branch-2.3

2020-11-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-24379: Description: In order to backport HIVE-24324 to branch-2.3 to remove deprecated Avro API so that

[jira] [Resolved] (HIVE-24324) Remove deprecated API usage from Avro

2020-11-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24324. - Fix Version/s: 4.0.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove deprecated API usage

[jira] [Assigned] (HIVE-24379) Backport HIVE-19662 to branch-2.3

2020-11-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24379: --- > Backport HIVE-19662 to branch-2.3 > - > > Key:

[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava

2020-11-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228795#comment-17228795 ] Chao Sun commented on HIVE-22126: - Curious how does this work for interop with Calcite? we were trying to

[jira] [Resolved] (HIVE-24316) Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1

2020-11-04 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-24316. - Hadoop Flags: Reviewed Resolution: Fixed > Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1 >

[jira] [Commented] (HIVE-24324) Remove deprecated API usage from Avro

2020-11-02 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17225057#comment-17225057 ] Chao Sun commented on HIVE-24324: - [~aihuaxu] can you review this? Thanks > Remove deprecated API usage

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.10.0

2020-10-30 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223853#comment-17223853 ] Chao Sun commented on HIVE-21737: - [~iemejia] Yes we'd need a new release (perhaps 2.3.8) with these

[jira] [Assigned] (HIVE-24331) Add Jenkinsfile for branch-3.1

2020-10-29 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24331: --- > Add Jenkinsfile for branch-3.1 > -- > > Key: HIVE-24331 >

[jira] [Assigned] (HIVE-24324) Remove deprecated API usage from Avro

2020-10-28 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24324: --- > Remove deprecated API usage from Avro > - > >

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.10.0

2020-10-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220351#comment-17220351 ] Chao Sun commented on HIVE-21737: - [~fokko] I'm not proposing to restore the API. Instead, I'm proposing

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.10.0

2020-10-23 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219858#comment-17219858 ] Chao Sun commented on HIVE-21737: - [~iemejia] instead of upgrading Avro in Hive, I think alternatively we

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.10.0

2020-10-13 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213206#comment-17213206 ] Chao Sun commented on HIVE-21737: - Sounds good to me [~iemejia]. Thanks! > Upgrade Avro to version

[jira] [Updated] (HIVE-23980) Shade guava from existing Hive versions

2020-08-18 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-23980: Fix Version/s: 2.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved

[jira] [Assigned] (HIVE-24035) Add Jenkinsfile for branch-2.3

2020-08-12 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-24035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-24035: --- > Add Jenkinsfile for branch-2.3 > -- > > Key: HIVE-24035 >

[jira] [Assigned] (HIVE-23998) Upgrave Guava to 27 for Hive 2.3

2020-08-06 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-23998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-23998: --- Assignee: L. C. Hsieh > Upgrave Guava to 27 for Hive 2.3 > > >

[jira] [Assigned] (HIVE-23980) Shade guava from existing Hive versions

2020-08-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-23980: --- Assignee: L. C. Hsieh > Shade guava from existing Hive versions >

[jira] [Commented] (HIVE-23980) Shade guava from existing Hive versions

2020-08-03 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HIVE-23980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170176#comment-17170176 ] Chao Sun commented on HIVE-23980: - [~viirya] have you considered using {{hive-exec-\-core.jar}}? see

[jira] [Assigned] (HIVE-17718) Hive on Spark Debugging Improvements

2018-12-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-17718: --- Assignee: (was: Chao Sun) > Hive on Spark Debugging Improvements >

[jira] [Assigned] (HIVE-17718) Hive on Spark Debugging Improvements

2018-12-04 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-17718: --- Assignee: Chao Sun (was: Sahil Takiar) > Hive on Spark Debugging Improvements >

[jira] [Comment Edited] (HIVE-15131) Change Parquet reader to read metadata on the task side

2018-07-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549653#comment-16549653 ] Chao Sun edited comment on HIVE-15131 at 7/19/18 6:23 PM: -- [~adeshrao]: seems

[jira] [Commented] (HIVE-15131) Change Parquet reader to read metadata on the task side

2018-07-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549653#comment-16549653 ] Chao Sun commented on HIVE-15131: - [~adeshrao]: seems the test failure

[jira] [Commented] (HIVE-15131) Change Parquet reader to read metadata on the task side

2018-07-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16546918#comment-16546918 ] Chao Sun commented on HIVE-15131: - Looks good [~adeshrao]. Could you also remove the "TODO" comment above

[jira] [Assigned] (HIVE-19087) Hive error when moving between different viewfs mount points

2018-03-30 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-19087: --- > Hive error when moving between different viewfs mount points >

[jira] [Commented] (HIVE-14303) CommonJoinOperator.checkAndGenObject should return directly to avoid NPE if ExecReducer.close is called twice.

2018-03-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409820#comment-16409820 ] Chao Sun commented on HIVE-14303: - [~tartarus]: since this patch is already committed, could you open a

[jira] [Commented] (HIVE-18673) ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly

2018-02-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360338#comment-16360338 ] Chao Sun commented on HIVE-18673: - +1 > ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly >

[jira] [Commented] (HIVE-17835) HS2 Logs print unnecessary stack trace when HoS query is cancelled

2018-02-08 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357325#comment-16357325 ] Chao Sun commented on HIVE-17835: - Patch LGTM. +1 > HS2 Logs print unnecessary stack trace when HoS query

[jira] [Commented] (HIVE-8436) Modify SparkWork to split works with multiple child works [Spark Branch]

2018-01-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-8436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335184#comment-16335184 ] Chao Sun commented on HIVE-8436: Hi [~kellyzly], without the copying function, the RDD cache will cache

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) The code

[jira] [Commented] (HIVE-17257) Hive should merge empty files

2018-01-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330983#comment-16330983 ] Chao Sun commented on HIVE-17257: - In the latest test run, most test failures are not new except the

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: (was: HIVE-17257.3.patch) > Hive should merge empty files > - >

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.3.patch > Hive should merge empty files > - > >

[jira] [Commented] (HIVE-17257) Hive should merge empty files

2018-01-17 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329074#comment-16329074 ] Chao Sun commented on HIVE-17257: - {quote}+1 for the patch. However, I'm not sure if those test failures

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: (was: HIVE-17257.3.patch) > Hive should merge empty files > - >

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-16 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.3.patch > Hive should merge empty files > - > >

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-15 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.3.patch > Hive should merge empty files > - > >

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2018-01-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.2.patch Re-attach patch v0 to trigger tests. > Hive should merge empty files >

[jira] [Commented] (HIVE-14531) Support deep nested struct for INSERT OVER DIRECTORY

2018-01-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315422#comment-16315422 ] Chao Sun commented on HIVE-14531: - [~priyagupta]: I think this may work: {code} INSERT OVERWRITE DIRECTORY

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed

[jira] [Commented] (HIVE-18310) Test 'vector_reduce_groupby_duplicate_cols.q' is misspelled in testconfiguration.properties

2017-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297214#comment-16297214 ] Chao Sun commented on HIVE-18310: - +1 on pending test. > Test 'vector_reduce_groupby_duplicate_cols.q' is

[jira] [Commented] (HIVE-18304) datediff() UDF returns a wrong result when dealing with a (date, string) input

2017-12-19 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297138#comment-16297138 ] Chao Sun commented on HIVE-18304: - Thanks [~hengyu.dai] for the patch! Could you explain why using

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.3.patch Thanks [~xuefuz] and [~asherman] for the comments! They are very helpful.

[jira] [Commented] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295876#comment-16295876 ] Chao Sun commented on HIVE-18283: - I don't think the test failures are related. [~xuefuz], [~asherman]:

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.2.patch Patch v2 to fix check style issues. > Better error message and error code

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.1.patch > Better error message and error code for HoS exceptions >

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: (was: HIVE-18283.1.patch) > Better error message and error code for HoS exceptions >

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: (was: HIVE-18283.1.patch) > Better error message and error code for HoS exceptions >

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.1.patch > Better error message and error code for HoS exceptions >

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Status: Patch Available (was: Open) > Better error message and error code for HoS exceptions >

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.1.patch Attaching patch v1 with a test for {{SparkSessionImpl#getHiveException()}}.

[jira] [Commented] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-15 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292958#comment-16292958 ] Chao Sun commented on HIVE-18283: - [~asherman] Yes, this definitely deserves a test case, although it

[jira] [Updated] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-15 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-18283: Attachment: HIVE-18283.0.patch Attaching patch v0. cc [~xuefuz]. > Better error message and error code for

[jira] [Assigned] (HIVE-18283) Better error message and error code for HoS exceptions

2017-12-14 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-18283: --- > Better error message and error code for HoS exceptions >

[jira] [Commented] (HIVE-17261) Hive use deprecated ParquetInputSplit constructor which blocked parquet dictionary filter

2017-09-15 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168216#comment-16168216 ] Chao Sun commented on HIVE-17261: - [~Ferd]: does

[jira] [Assigned] (HIVE-15156) Support Nested Column Field Pruning for Parquet Vectorized Reader

2017-09-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-15156: --- Assignee: Chao Sun > Support Nested Column Field Pruning for Parquet Vectorized Reader >

[jira] [Updated] (HIVE-16853) Minor org.apache.hadoop.hive.ql.exec.HashTableSinkOperator Improvement

2017-08-13 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-16853: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed

[jira] [Commented] (HIVE-16853) Minor org.apache.hadoop.hive.ql.exec.HashTableSinkOperator Improvement

2017-08-09 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120282#comment-16120282 ] Chao Sun commented on HIVE-16853: - +1 > Minor org.apache.hadoop.hive.ql.exec.HashTableSinkOperator

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2017-08-08 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.1.patch Submitting patch v1 for testing (it is not ready yet). > Hive should merge

[jira] [Updated] (HIVE-16758) Better Select Number of Replications

2017-08-08 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-16758: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed

[jira] [Updated] (HIVE-16758) Better Select Number of Replications

2017-08-08 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-16758: Component/s: Spark > Better Select Number of Replications > > >

[jira] [Commented] (HIVE-16758) Better Select Number of Replications

2017-08-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117568#comment-16117568 ] Chao Sun commented on HIVE-16758: - +1 on patch v3. > Better Select Number of Replications >

[jira] [Commented] (HIVE-17257) Hive should merge empty files

2017-08-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117565#comment-16117565 ] Chao Sun commented on HIVE-17257: - [~kellyzly]: the empty files maybe generated if the result set is empty

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed

[jira] [Commented] (HIVE-16758) Better Select Number of Replications

2017-08-07 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116819#comment-16116819 ] Chao Sun commented on HIVE-16758: - [~belugabehr] Now I'm looking at the Hadoop code, it is using 10 as the

[jira] [Commented] (HIVE-16758) Better Select Number of Replications

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115974#comment-16115974 ] Chao Sun commented on HIVE-16758: - [~belugabehr] Thanks. If it fails in a small cluster with default value

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Status: Patch Available (was: Open) > Hive should merge empty files > - > >

[jira] [Updated] (HIVE-17257) Hive should merge empty files

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17257: Attachment: HIVE-17257.0.patch Submitting patch v0 for testing. > Hive should merge empty files >

[jira] [Assigned] (HIVE-17257) Hive should merge empty files

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-17257: --- > Hive should merge empty files > - > > Key: HIVE-17257 >

[jira] [Commented] (HIVE-16758) Better Select Number of Replications

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115902#comment-16115902 ] Chao Sun commented on HIVE-16758: - {quote} We should be allowing users to use Hive-on-Spark without

[jira] [Commented] (HIVE-16758) Better Select Number of Replications

2017-08-06 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115878#comment-16115878 ] Chao Sun commented on HIVE-16758: - Thanks [~belugabehr] for the patch and my apology for replying so late!

[jira] [Commented] (HIVE-17250) Avoid manually deploy changes to Jenkins server in testutils/ptest2/conf

2017-08-03 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113920#comment-16113920 ] Chao Sun commented on HIVE-17250: - [~stakiar] FYI. > Avoid manually deploy changes to Jenkins server in

[jira] [Updated] (HIVE-17250) Avoid manually deploy changes to Jenkins server in testutils/ptest2/conf

2017-08-03 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17250: Issue Type: Improvement (was: Bug) > Avoid manually deploy changes to Jenkins server in

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-02 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.5.patch Patch v4 used INSERT OVERWRITE DIRECTORY, which didn't work for mini

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-02 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.4.patch > HoS: file merging doesn't work for union all >

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.3.patch > HoS: file merging doesn't work for union all >

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: (was: HIVE-17213.3.patch) > HoS: file merging doesn't work for union all >

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.3.patch Test failures unrelated for patch v2. Attaching patch v3 with qtest.

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-08-01 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.2.patch Attaching patch v2 to address the case when linkedFileSinkDesc is not set.

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-07-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.1.patch Patch v0 incorrectly deleted {{SparkFileSinkProcessor}}, which is used to

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-07-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Attachment: HIVE-17213.0.patch > HoS: file merging doesn't work for union all >

[jira] [Updated] (HIVE-17213) HoS: file merging doesn't work for union all

2017-07-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17213: Status: Patch Available (was: Open) > HoS: file merging doesn't work for union all >

[jira] [Assigned] (HIVE-17213) HoS: file merging doesn't work for union all

2017-07-31 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HIVE-17213: --- > HoS: file merging doesn't work for union all > > >

[jira] [Commented] (HIVE-17117) Metalisteners are not notified when threadlocal metaconf is cleanup

2017-07-24 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099064#comment-16099064 ] Chao Sun commented on HIVE-17117: - Tests failures are not related. Committed this to master. Thanks

[jira] [Updated] (HIVE-17117) Metalisteners are not notified when threadlocal metaconf is cleanup

2017-07-24 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-17117: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) >

[jira] [Commented] (HIVE-16336) Rename hive.spark.use.file.size.for.mapjoin to hive.spark.use.ts.stats.for.mapjoin

2017-07-24 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097989#comment-16097989 ] Chao Sun commented on HIVE-16336: - Changed. Thanks [~leftylev]. > Rename

[jira] [Updated] (HIVE-16336) Rename hive.spark.use.file.size.for.mapjoin to hive.spark.use.ts.stats.for.mapjoin

2017-07-24 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-16336: Fix Version/s: (was: 2.3.0) 3.0.0 > Rename hive.spark.use.file.size.for.mapjoin to

[jira] [Commented] (HIVE-17114) HoS: Possible skew in shuffling when data is not really skewed

2017-07-21 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096570#comment-16096570 ] Chao Sun commented on HIVE-17114: - Looks good. +1. > HoS: Possible skew in shuffling when data is not

[jira] [Commented] (HIVE-17117) Metalisteners are not notified when threadlocal metaconf is cleanup

2017-07-20 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095073#comment-16095073 ] Chao Sun commented on HIVE-17117: - +1 > Metalisteners are not notified when threadlocal metaconf is

[jira] [Commented] (HIVE-17111) TestSparkCliDriver does not use LocalHiveSparkClient

2017-07-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092597#comment-16092597 ] Chao Sun commented on HIVE-17111: - I don't see a strong reason of migrating the tests to the SparkOnYarn

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092277#comment-16092277 ] Chao Sun commented on HIVE-17018: - Agreed. I think in the mean time at least we should put more doc on how

[jira] [Commented] (HIVE-17117) Metalisteners are not notified when threadlocal metaconf is cleanup

2017-07-18 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092270#comment-16092270 ] Chao Sun commented on HIVE-17117: - Patch looks good to me. [~ngangam], [~mohitsabharwal], [~thejas], could

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-12 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083610#comment-16083610 ] Chao Sun commented on HIVE-17018: - [~kellyzly] Yes. I think we don't need to change the existing behavior.

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083411#comment-16083411 ] Chao Sun commented on HIVE-17018: - What I'm thinking is that the new config (say A) will be a value

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-11 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081743#comment-16081743 ] Chao Sun commented on HIVE-17018: - [~kellyzly]: yes, this is to prevent a Spark task from using unbounded

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-10 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081648#comment-16081648 ] Chao Sun commented on HIVE-17018: - Thanks for the examples [~kellyzly]. Are you trying to explain that HoS

[jira] [Commented] (HIVE-17018) Small table is converted to map join even the total size of small tables exceeds the threshold(hive.auto.convert.join.noconditionaltask.size)

2017-07-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075871#comment-16075871 ] Chao Sun commented on HIVE-17018: - [~kellyzly] I'm not sure if I understand what you described. Can you

[jira] [Comment Edited] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

2017-07-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075852#comment-16075852 ] Chao Sun edited comment on HIVE-17010 at 7/6/17 3:27 AM: - Ah I see. Sometimes the

[jira] [Commented] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

2017-07-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075852#comment-16075852 ] Chao Sun commented on HIVE-17010: - Ah I see. Sometimes the stats estimation could generate negative

[jira] [Commented] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

2017-07-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075826#comment-16075826 ] Chao Sun commented on HIVE-17010: - With long you need to have ~9000 PB of {{numberOfBytes}} for the

[jira] [Commented] (HIVE-17010) Fix the overflow problem of Long type in SetSparkReducerParallelism

2017-07-05 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075268#comment-16075268 ] Chao Sun commented on HIVE-17010: - bq. We use Long type and it happens overflow when the data is too big.

<    1   2   3   4   5   6   7   8   >