[jira] [Comment Edited] (HIVE-28277) HIVE does not support update operations for ICEBERG of type location_based_table.

2024-05-23 Thread yongzhi.shao (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-28277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848817#comment-17848817
 ] 

yongzhi.shao edited comment on HIVE-28277 at 5/24/24 2:50 AM:
--

I've updated the code and the problem has gone away. Thank you, sir.

This problem was fixed in HIVE-28069.


was (Author: lisoda):
I've updated the code and the problem has gone away. Thank you, sir.

> HIVE does not support update operations for ICEBERG of type 
> location_based_table.
> -
>
> Key: HIVE-28277
> URL: https://issues.apache.org/jira/browse/HIVE-28277
> Project: Hive
>  Issue Type: Bug
>  Components: Iceberg integration
>Affects Versions: 4.0.0
> Environment: ICEBERG:1.5.2
> HIVE 4.0.0
>Reporter: yongzhi.shao
>Priority: Major
> Fix For: 4.0.0
>
>
> Currently, when I update the location_based_table using hive, hive 
> incorrectly empties all data directories and metadata directories.
> After the update statement is executed, the iceberg table is corrupted.
>  
> {code:java}
> --spark 3.4.1 + iceberg 1.5.2:
> CREATE TABLE IF NOT EXISTS datacenter.default.test_data_04 (
> id string,name string
> )
> using iceberg
> PARTITIONED BY (name)
> TBLPROPERTIES 
> ('read.orc.vectorization.enabled'='true','write.format.default'='orc','write.orc.bloom.filter.columns'='id','write.orc.compression-codec'='zstd','write.metadata.previous-versions-max'='3','write.metadata.delete-after-commit.enabled'='true');
> insert into datacenter.default.test_data_04(id,name) 
> values('1','a'),('2','b');
> --hive4:
> CREATE EXTERNAL TABLE default.test_data_04
> STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' 
> LOCATION 'hdfs:///iceberg-catalog/warehouse/default/test_data_04'
> TBLPROPERTIES 
> ('iceberg.catalog'='location_based_table','engine.hive.enabled'='true');
> select id,name from default.test_data_04; --2 row
> update test_data_04 set name = 'adasd' where id = '1';
> ERROR:
> 2024-05-23T10:26:32,028 ERROR [HiveServer2-Background-Pool: Thread-297] 
> hive.HiveIcebergStorageHandler: Error while trying to commit job: 
> job_17061635207991_169536, job_17061635207990_169536, 
> job_17061635207992_169536, starting rollback changes for table: 
> default.test_data_04
> org.apache.iceberg.exceptions.NoSuchTableException: Table does not exist at 
> location: /iceberg-catalog/warehouse/default/test_data_04
> BEFORE UPDATE:
> ICEBERG TABLE DIR:
> [root@ ~]# hdfs dfs -ls /iceberg-catalog/warehouse/default/test_data_04
> Found 2 items
> drwxr-xr-x   - hive hdfs          0 2024-05-23 09:26 
> /iceberg-catalog/warehouse/default/test_data_04/data
> drwxr-xr-x   - hive hdfs          0 2024-05-23 09:26 
> /iceberg-catalog/warehouse/default/test_data_04/metadata
> AFTER UPDATE:
> ICEBERG TABLE DIR:
> [root@XXX ~]# hdfs dfs -ls /iceberg-catalog/warehouse/default/test_data_04
> Found 3 items
> drwxr-xr-x   - hive hdfs          0 2024-05-23 10:26 
> /iceberg-catalog/warehouse/default/test_data_04/-tmp.HIVE_UNION_SUBDIR_1
> drwxr-xr-x   - hive hdfs          0 2024-05-23 10:26 
> /iceberg-catalog/warehouse/default/test_data_04/-tmp.HIVE_UNION_SUBDIR_2
> drwxr-xr-x   - hive hdfs          0 2024-05-23 10:26 
> /iceberg-catalog/warehouse/default/test_data_04/-tmp.HIVE_UNION_SUBDIR_3
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-27498) Support custom delimiter in SkippingTextInputFormat

2024-05-23 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-27498:
--
Labels: pull-request-available  (was: )

> Support custom delimiter in SkippingTextInputFormat
> ---
>
> Key: HIVE-27498
> URL: https://issues.apache.org/jira/browse/HIVE-27498
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Mayank Kunwar
>Priority: Major
>  Labels: pull-request-available
>
> Simple select is returning results as expected when there are configs
> {noformat}
> 'skip.header.line.count'='1',                    
> 'textinputformat.record.delimiter'='|'{noformat}
> but if we execute select count(*) or any query that launches a tez job is 
> considering the whole text as single line
> *Test case*
> data.csv
> {noformat}
> CodeName|A |B 
> |C  {noformat}
> DDL
> {noformat}
> create external table test(code string,name string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
>  WITH SERDEPROPERTIES (
>'field.delim'='\t')
>  STORED AS INPUTFORMAT
>'org.apache.hadoop.mapred.TextInputFormat'
>  OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
>location '${system:test.tmp.dir}/test'
>  TBLPROPERTIES (
>'skip.header.line.count'='1',
>'textinputformat.record.delimiter'='|');{noformat}
> Query result
> select code,name from test;
> {noformat}
> A 
> B 
> 
> C {noformat}
> *Problem:* But query _+select count(*) from test+_  is returning 1 instead of 
> 3
> It used to work in older hive versions.
> The difference in behaviour started to happen after the introduction of 
> feature https://issues.apache.org/jira/browse/HIVE-21924
> The feature aims at splitting the text files while reading even though the 
> table has configuration to skip headers. There by increasing the number of 
> mappers to process the query there by improving throughput of the query.
> The actual problem lies in how new feature is reading a file. It does not 
> consider 'textinputformat.record.delimiter' property and tries to read the 
> file looking for new line characters. Since the input file does not have a 
> new line for every record, it is reading the whole file as single line and 
> count is returned as 1
> Ref: 
> [https://github.com/apache/hive/blob/24a82a65f96b65eeebe4e23b2fec425037a70216/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L548]
>  
>  *Workaround*
> If we can remove headers in the data and skip header config in table 
> properties or compress the files, then we will not get into this issue
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28279) Output the database name for SHOW EXTENDED TABLES statement

2024-05-23 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-28279:
--
Labels: pull-request-available  (was: )

> Output the database name for SHOW EXTENDED TABLES statement
> ---
>
> Key: HIVE-28279
> URL: https://issues.apache.org/jira/browse/HIVE-28279
> Project: Hive
>  Issue Type: Task
>  Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
>  Labels: pull-request-available
>
> HIVE-21301 introduced {{SHOW EXTENDED TABLES}} statement which output table 
> name and table type while listing tables in a database.
> In this patch, we aim to add a new output filed for database name with 
> following reasons:
> 1. database name in {{SHOW EXTENDED TABLES}} statement is optional, output 
> the database is informal in this case.
> 2. when statistic table names and database names by this statement for list 
> of databases, the output result including database name is much more helpful.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-28278:
--
Labels: pull-request-available  (was: )

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Bug
>  Components: Iceberg integration
>Affects Versions: 4.0.0
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>  Labels: pull-request-available
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-28279) Output the database name for SHOW EXTENDED TABLES statement

2024-05-23 Thread Wechar (Jira)
Wechar created HIVE-28279:
-

 Summary: Output the database name for SHOW EXTENDED TABLES 
statement
 Key: HIVE-28279
 URL: https://issues.apache.org/jira/browse/HIVE-28279
 Project: Hive
  Issue Type: Task
  Components: Hive
Reporter: Wechar
Assignee: Wechar


HIVE-21301 introduced {{SHOW EXTENDED TABLES}} statement which output table 
name and table type while listing tables in a database.

In this patch, we aim to add a new output filed for database name with 
following reasons:
1. database name in {{SHOW EXTENDED TABLES}} statement is optional, output the 
database is informal in this case.
2. when statistic table names and database names by this statement for list of 
databases, the output result including database name is much more helpful.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-28278:
--
Issue Type: Bug  (was: Task)

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-28278:
--
Affects Version/s: 4.0.0

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-28278:
--
Component/s: Iceberg integration

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Bug
>  Components: Iceberg integration
>Affects Versions: 4.0.0
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-28278:
--
Status: Patch Available  (was: Open)

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Task
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko reassigned HIVE-28278:
-

Assignee: Denys Kuzmenko

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Task
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-28278) CDPD-70188

2024-05-23 Thread Denys Kuzmenko (Jira)
Denys Kuzmenko created HIVE-28278:
-

 Summary: CDPD-70188
 Key: HIVE-28278
 URL: https://issues.apache.org/jira/browse/HIVE-28278
 Project: Hive
  Issue Type: Task
Reporter: Denys Kuzmenko


BugFix, can happen when the stats file was already created but stats object has 
not yet been written, and someone tried to read it.

Why are the changes needed?
{code}
ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
minimal length of the footer tail 12
java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
minimal length of the footer tail 12
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (HIVE-28278) Iceberg: Stats: IllegalStateException Invalid file: file length 0

2024-05-23 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-28278:
--
Summary: Iceberg: Stats: IllegalStateException Invalid file: file length 0  
(was: CDPD-70188)

> Iceberg: Stats: IllegalStateException Invalid file: file length 0
> -
>
> Key: HIVE-28278
> URL: https://issues.apache.org/jira/browse/HIVE-28278
> Project: Hive
>  Issue Type: Task
>Reporter: Denys Kuzmenko
>Priority: Major
>
> BugFix, can happen when the stats file was already created but stats object 
> has not yet been written, and someone tried to read it.
> Why are the changes needed?
> {code}
> ERROR : FAILED: IllegalStateException Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> java.lang.IllegalStateException: Invalid file: file length 0 is less tha 
> minimal length of the footer tail 12
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (HIVE-27498) Support custom delimiter in SkippingTextInputFormat

2024-05-23 Thread Mayank Kunwar (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mayank Kunwar reopened HIVE-27498:
--

The issue is hitting again, so reopening the ticket

> Support custom delimiter in SkippingTextInputFormat
> ---
>
> Key: HIVE-27498
> URL: https://issues.apache.org/jira/browse/HIVE-27498
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Priority: Major
>
> Simple select is returning results as expected when there are configs
> {noformat}
> 'skip.header.line.count'='1',                    
> 'textinputformat.record.delimiter'='|'{noformat}
> but if we execute select count(*) or any query that launches a tez job is 
> considering the whole text as single line
> *Test case*
> data.csv
> {noformat}
> CodeName|A |B 
> |C  {noformat}
> DDL
> {noformat}
> create external table test(code string,name string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
>  WITH SERDEPROPERTIES (
>'field.delim'='\t')
>  STORED AS INPUTFORMAT
>'org.apache.hadoop.mapred.TextInputFormat'
>  OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
>location '${system:test.tmp.dir}/test'
>  TBLPROPERTIES (
>'skip.header.line.count'='1',
>'textinputformat.record.delimiter'='|');{noformat}
> Query result
> select code,name from test;
> {noformat}
> A 
> B 
> 
> C {noformat}
> *Problem:* But query _+select count(*) from test+_  is returning 1 instead of 
> 3
> It used to work in older hive versions.
> The difference in behaviour started to happen after the introduction of 
> feature https://issues.apache.org/jira/browse/HIVE-21924
> The feature aims at splitting the text files while reading even though the 
> table has configuration to skip headers. There by increasing the number of 
> mappers to process the query there by improving throughput of the query.
> The actual problem lies in how new feature is reading a file. It does not 
> consider 'textinputformat.record.delimiter' property and tries to read the 
> file looking for new line characters. Since the input file does not have a 
> new line for every record, it is reading the whole file as single line and 
> count is returned as 1
> Ref: 
> [https://github.com/apache/hive/blob/24a82a65f96b65eeebe4e23b2fec425037a70216/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L548]
>  
>  *Workaround*
> If we can remove headers in the data and skip header config in table 
> properties or compress the files, then we will not get into this issue
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (HIVE-27498) Support custom delimiter in SkippingTextInputFormat

2024-05-23 Thread Mayank Kunwar (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-27498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mayank Kunwar reassigned HIVE-27498:


Assignee: Mayank Kunwar

> Support custom delimiter in SkippingTextInputFormat
> ---
>
> Key: HIVE-27498
> URL: https://issues.apache.org/jira/browse/HIVE-27498
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Taraka Rama Rao Lethavadla
>Assignee: Mayank Kunwar
>Priority: Major
>
> Simple select is returning results as expected when there are configs
> {noformat}
> 'skip.header.line.count'='1',                    
> 'textinputformat.record.delimiter'='|'{noformat}
> but if we execute select count(*) or any query that launches a tez job is 
> considering the whole text as single line
> *Test case*
> data.csv
> {noformat}
> CodeName|A |B 
> |C  {noformat}
> DDL
> {noformat}
> create external table test(code string,name string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
>  WITH SERDEPROPERTIES (
>'field.delim'='\t')
>  STORED AS INPUTFORMAT
>'org.apache.hadoop.mapred.TextInputFormat'
>  OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
>location '${system:test.tmp.dir}/test'
>  TBLPROPERTIES (
>'skip.header.line.count'='1',
>'textinputformat.record.delimiter'='|');{noformat}
> Query result
> select code,name from test;
> {noformat}
> A 
> B 
> 
> C {noformat}
> *Problem:* But query _+select count(*) from test+_  is returning 1 instead of 
> 3
> It used to work in older hive versions.
> The difference in behaviour started to happen after the introduction of 
> feature https://issues.apache.org/jira/browse/HIVE-21924
> The feature aims at splitting the text files while reading even though the 
> table has configuration to skip headers. There by increasing the number of 
> mappers to process the query there by improving throughput of the query.
> The actual problem lies in how new feature is reading a file. It does not 
> consider 'textinputformat.record.delimiter' property and tries to read the 
> file looking for new line characters. Since the input file does not have a 
> new line for every record, it is reading the whole file as single line and 
> count is returned as 1
> Ref: 
> [https://github.com/apache/hive/blob/24a82a65f96b65eeebe4e23b2fec425037a70216/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L548]
>  
>  *Workaround*
> If we can remove headers in the data and skip header config in table 
> properties or compress the files, then we will not get into this issue
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (HIVE-28273) Test data generation failure in HIVE-28249 related tests

2024-05-23 Thread Stamatis Zampetakis (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-28273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stamatis Zampetakis resolved HIVE-28273.

Fix Version/s: 4.1.0
   Resolution: Fixed

Fixed in 
https://github.com/apache/hive/commit/019017d0909a17d6e85d519f5c3f4f52828fd509

Thanks for the PR [~Csaba]!

> Test data generation failure in HIVE-28249 related tests
> 
>
> Key: HIVE-28273
> URL: https://issues.apache.org/jira/browse/HIVE-28273
> Project: Hive
>  Issue Type: Bug
>Reporter: Csaba Juhász
>Assignee: Csaba Juhász
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.1.0
>
> Attachments: image-2024-05-22-19-11-35-890.png
>
>
> generateJulianLeapYearTimestamps and generateJulianLeapYearTimestamps28thFeb 
> are throwing NegativeArraySizeException once the base value equals or is over 
> 999
> This is caused by the below code, supplying a negative value (when digits 
> return a value larger than 4) to zeros, which in turn is used to create a new 
> char array.
> {code:java}
> StringBuilder sb = new StringBuilder(29);
> int year = ((i % ) + 1) * 100;
> sb.append(zeros(4 - digits(year)));
> {code}
> When the tests are run using maven, the error in the generation function is 
> caught but never rethrown or reported and  the build is reported successful. 
> For example running
> _TestParquetTimestampsHive2Compatibility#testWriteHive2ReadHive4UsingLegacyConversionWithJulianLeapYearsFor28thFeb_
>  has the result:
> {code:java}
> [INFO] ---
> [INFO]  T E S T S
> [INFO] ---
> [INFO] Running 
> org.apache.hadoop.hive.ql.io.parquet.serde.TestParquetTimestampsHive2Compatibility
> [INFO] Tests run: 396, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
> 0.723 s - in 
> org.apache.hadoop.hive.ql.io.parquet.serde.TestParquetTimestampsHive2Compatibility
> [INFO] 
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 396, Failures: 0, Errors: 0, Skipped: 0
> ...
> [INFO] BUILD SUCCESS
> {code}
> When the test is run through an IDE (eg VSCode), the failure is reported 
> properly.
>  !image-2024-05-22-19-11-35-890.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)