[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables

2023-04-25 Thread tanghui (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tanghui updated HIVE-17361:
---
Description: 
引入 ACID 后不支持加载数据。需要填补 ACID 表和常规配置单元表之间的差距。

当前文档位于[DML 
操作|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations]和[将文件加载到表中|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]:

 
 * 加载数据对数据执行非常有限的验证,特别是它使用可能不在 0_0 中的输入文件名,这可能会破坏某些读取逻辑。(当然会酸)。
 * 它不检查文件的架构。这对于 Acid 来说可能不是问题,它需要自描述的 ORC,因此 Schema Evolution 
可以无缝地处理这个问题。(假设架构没有太大不同)。
 * 它会检查 _InputFormat_S 是否兼容。
 * 分桶(并因此排序)表不支持加载数据(但仅当 hive.strict.checks.bucketing=true(默认))。将保留对 Acid 的限制。
 * 加载数据支持 OVERWRITE 子句
 * 文件权限/所有权会发生什么:重命名与复制差异


实施将遵循与中相同的想法HIVE-14988并为 OVERWRITE 子句使用 base_N/ 目录。


minor compaction 如何处理原始文件的 delta/base?
由于 delta_8_8/_meta_data 是在文件移动之前创建的,因此 delta_8_8 在填充之前变得可见。这是一个问题吗?
不是因为 txn 8 没有提交。
h3. 实施说明/限制(补丁 25)
 * 不支持分桶/排序表
 * 输入文件名必须采用 0_0/0_0_copy_1 形式 - 强制执行。(HIVE-18125)
 * 加载数据创建一个包含新文件的 delta_x_x/
 * Load Data w/Overwrite 创建一个包含新文件的 base_x/
 * “_metadata_acid”文件放置在目标目录中以指示它需要在读取时进行特殊处理
 * 输入文件必须是“普通”ORC 文件,即不包含 acid 元数据列,如果这些文件是从另一个 Acid 
表复制的,就会出现这种情况。在后一种情况下,数据中嵌入的 ROW_ID 
在目标表中可能没有意义(例如,如果它在不同的集群中)。此类文件也可能混合了已提交和已中止的数据。
 ** 稍后可以通过向 _metadata_acid 文件添加信息以在读取时忽略现有的 ROW_ID 来放松这一点。
 * ROW_ID 在读取时动态附加,并通过压缩永久保存。这与处理在转换为 Acid 之前写入表的文件的方式相同。
 * 支持矢量化

  was:
LOAD DATA was not supported since ACID was introduced. Need to fill this gap 
between ACID table and regular hive table.

Current Documentation is under [DML 
Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations]
 and [Loading files into 
tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]:

\\
* Load Data performs very limited validations of the data, in particular it 
uses the input file name which may not be in 0_0 which can break some read 
logic.  (Certainly will for Acid).
* It does not check the schema of the file.  This may be a non issue for Acid 
which requires ORC which is self describing so Schema Evolution may handle this 
seamlessly.  (Assuming Schema is not too different).
* It does check that _InputFormat_S are compatible. 
* Bucketed (and thus sorted) tables don't support Load Data (but only if 
hive.strict.checks.bucketing=true (default)).  Will keep this restriction for 
Acid.
* Load Data supports OVERWRITE clause
* What happens to file permissions/ownership: rename vs copy differences

\\
The implementation will follow the same idea as in HIVE-14988 and use a base_N/ 
dir for OVERWRITE clause.

\\
How is minor compaction going to handle delta/base with original files?
Since delta_8_8/_meta_data is created before files are moved, delta_8_8 becomes 
visible before it's populated.  Is that an issue?
It's not since txn 8 is not committed.

h3. Implementation Notes/Limitations (patch 25)
* bucketed/sorted tables are not supported
* input files names must be of the form 0_0/0_0_copy_1 - enforced. 
(HIVE-18125)
* Load Data creates a delta_x_x/ that contains new files
* Load Data w/Overwrite creates a base_x/ that contains new files
* A '_metadata_acid' file is placed in the target directory to indicate it 
requires special handling on read
* The input files must be 'plain' ORC files, i.e. not contain acid metadata 
columns as would be the case if these files were copied from another Acid 
table.  In the latter case, the ROW_IDs embedded in the data may not make sense 
in the target table (if it's in a different cluster, for example).  Such files 
may also have a mix of committed and aborted data.
** this could be relaxed later by adding info to the _metadata_acid file to 
ignore existing ROW_IDs on read.
* ROW_IDs are attached dynamically at read time and made permanent by 
compaction.  This is done the same way has handling of files that were written 
to a table before it was converted to Acid.
* Vectorization is supported




> Support LOAD DATA for transactional tables
> --
>
> Key: HIVE-17361
> URL: https://issues.apache.org/jira/browse/HIVE-17361
> Project: Hive
>  Issue Type: New Feature
>  Components: Transactions
>Reporter: Wei Zheng
>Assignee: Eugene Koifman
>Priority: Critical
>  Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, 
> HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, 
> HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.14.patch, 
> HIVE-17361.16.patch, HIVE-17361.17.patch, HIVE-17361.19.patch, 
> HIVE-17361.2.patch, HIVE-17361.20.patch, HIVE-17361.21.patch, 
> HIVE-17361.23.patch, HIVE-17361.24.patch, HIVE-17361.25.patch, 
> HIVE-17361.3.patch, HIVE-17361.4.patch
>
>
> 引入 ACID 后不支持加载数据。需要填补 

[jira] [Commented] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-12-07 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644572#comment-17644572
 ] 

tanghui commented on HIVE-26158:


[~LiPenglin] Yes, if your table is an external table not a 
TRANSLATED_TO_EXTERNAL table , the previous rules will be maintained when 
rename table, that is, the storage path of the data will not be modified。

> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0-alpha-2
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |*LOCATION*|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> ---
>  
> We need to modify the partition location of the table in SDS to ensure that 
> the query results are normal



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-05-11 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17535839#comment-17535839
 ] 

tanghui commented on HIVE-26158:


[~kgyrtkirk]  thank you so much for the patch,I haven't found any other tickets 
yet

> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0, 4.0.0-alpha-2
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |*LOCATION*|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> ---
>  
> We need to modify the partition location of the table in SDS to ensure that 
> the query results are normal



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-27 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529168#comment-17529168
 ] 

tanghui edited comment on HIVE-26158 at 4/28/22 3:31 AM:
-

[~kgyrtkirk] yes , I want pick this up ,thanks


was (Author: sanguines):
yes , I want pick this up ,thanks

> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Assignee: Zoltan Haindrich
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |*LOCATION*|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> ---
>  
> We need to modify the partition location of the table in SDS to ensure that 
> the query results are normal



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-27 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529168#comment-17529168
 ] 

tanghui commented on HIVE-26158:


yes , I want pick this up ,thanks

> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Assignee: Zoltan Haindrich
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |*LOCATION*|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> ---
>  
> We need to modify the partition location of the table in SDS to ensure that 
> the query results are normal



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (HIVE-24920) TRANSLATED_TO_EXTERNAL tables may write to the same location

2022-04-20 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524983#comment-17524983
 ] 

tanghui edited comment on HIVE-24920 at 4/21/22 12:41 AM:
--

After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |

-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|*LOCATION*|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal:

https://issues.apache.org/jira/browse/HIVE-26158


was (Author: sanguines):
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |

-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|*LOCATION*|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

> TRANSLATED_TO_EXTERNAL tables may write to the same location
> 
>
> Key: HIVE-24920
> URL: https://issues.apache.org/jira/browse/HIVE-24920
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0, 4.0.0-alpha-1
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {code}
> create table t (a integer);
> insert into t values(1);
> alter table t rename to t2;
> create table t (a integer); -- I expected an exception from this command 
> (location already exists) but because its an external table no exception
> insert into t values(2);
> select * from t;  -- shows 1 and 2
> drop table t2;-- wipes out data location
> select * from t;  -- empty resultset
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (HIVE-24920) TRANSLATED_TO_EXTERNAL tables may write to the same location

2022-04-20 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524983#comment-17524983
 ] 

tanghui edited comment on HIVE-24920 at 4/20/22 1:38 PM:
-

After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |

-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|*LOCATION*|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal


was (Author: sanguines):
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

 

> TRANSLATED_TO_EXTERNAL tables may write to the same location
> 
>
> Key: HIVE-24920
> URL: https://issues.apache.org/jira/browse/HIVE-24920
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0, 4.0.0-alpha-1
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {code}
> create table t (a integer);
> insert into t values(1);
> alter table t rename to t2;
> create table t (a integer); -- I expected an exception from this command 
> (location already exists) but because its an external table no exception
> insert into t values(2);
> select * from t;  -- shows 1 and 2
> drop table t2;-- wipes out data location
> select * from t;  -- empty resultset
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-20 Thread tanghui (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tanghui updated HIVE-26158:
---
Description: 
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |

-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|*LOCATION*|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

  was:
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |


-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal


> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |*LOCATION*|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> 

[jira] [Updated] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-20 Thread tanghui (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tanghui updated HIVE-26158:
---
Description: 
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this result is null.
select * from part_test11 where dat="20220101";
||part_test.c1||part_test.c2||part_test.dat||
| | | |


-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

  was:
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal


> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this result is null.
> select * from part_test11 where dat="20220101";
> ||part_test.c1||part_test.c2||part_test.dat||
> | | | |
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |LOCATION|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> 

[jira] [Updated] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-20 Thread tanghui (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tanghui updated HIVE-26158:
---
Description: 
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.

 

in beeline:



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

  was:
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal


> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
>  
> in beeline:
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this resulting in no data in the query partition.
> select * from part_test11 where dat="20220101";
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |LOCATION|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> 

[jira] [Updated] (HIVE-26158) TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table

2022-04-20 Thread tanghui (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tanghui updated HIVE-26158:
---
Summary: TRANSLATED_TO_EXTERNAL partition tables cannot query partition 
data after rename table  (was: TRANSLATED_TO_EXTERNAL partition tables cannot 
query partition data after rename)

> TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after 
> rename table
> --
>
> Key: HIVE-26158
> URL: https://issues.apache.org/jira/browse/HIVE-26158
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 4.0.0, 4.0.0-alpha-1, 4.0.0-alpha-2
>Reporter: tanghui
>Priority: Major
>
> After the patch is updated, the partition table location and hdfs data 
> directory are displayed normally, but the partition location of the table in 
> the SDS in the Hive metabase is still displayed as the location of the old 
> table, resulting in no data in the query partition.
> 
> set hive.create.as.external.legacy=true;
> CREATE TABLE part_test(
> c1 string
> ,c2 string
> )PARTITIONED BY (dat string)
> insert into part_test values ("11","th","20220101")
> insert into part_test values ("22","th","20220102")
> alter table part_test rename to part_test11;
> --this resulting in no data in the query partition.
> select * from part_test11 where dat="20220101";
> -
> SDS in the Hive metabase:
> select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
> TBLS.TBL_ID=SDS.CD_ID;
> ---
> |LOCATION|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
> |hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|
> ---
>  
> We need to modify the partition location of the table in SDS to ensure that 
> the query results are normal



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (HIVE-24920) TRANSLATED_TO_EXTERNAL tables may write to the same location

2022-04-20 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524983#comment-17524983
 ] 

tanghui edited comment on HIVE-24920 at 4/20/22 1:18 PM:
-

After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.



set hive.create.as.external.legacy=true;

CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

 


was (Author: sanguines):
After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.


CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

 

> TRANSLATED_TO_EXTERNAL tables may write to the same location
> 
>
> Key: HIVE-24920
> URL: https://issues.apache.org/jira/browse/HIVE-24920
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0, 4.0.0-alpha-1
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {code}
> create table t (a integer);
> insert into t values(1);
> alter table t rename to t2;
> create table t (a integer); -- I expected an exception from this command 
> (location already exists) but because its an external table no exception
> insert into t values(2);
> select * from t;  -- shows 1 and 2
> drop table t2;-- wipes out data location
> select * from t;  -- empty resultset
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (HIVE-24920) TRANSLATED_TO_EXTERNAL tables may write to the same location

2022-04-20 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17524983#comment-17524983
 ] 

tanghui commented on HIVE-24920:


After the patch is updated, the partition table location and hdfs data 
directory are displayed normally, but the partition location of the table in 
the SDS in the Hive metabase is still displayed as the location of the old 
table, resulting in no data in the query partition.


CREATE TABLE part_test(
c1 string
,c2 string
)PARTITIONED BY (dat string)

insert into part_test values ("11","th","20220101")
insert into part_test values ("22","th","20220102")

alter table part_test rename to part_test11;

--this resulting in no data in the query partition.
select * from part_test11 where dat="20220101";
-

SDS in the Hive metabase:
select SDS.LOCATION from TBLS,SDS where TBLS.TBL_NAME="part_test11" AND 
TBLS.TBL_ID=SDS.CD_ID;

---
|LOCATION|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test11|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220101|
|hdfs://nameservice1/warehouse/tablespace/external/hive/part_test/dat=20220102|

---

 

We need to modify the partition location of the table in SDS to ensure that the 
query results are normal

 

> TRANSLATED_TO_EXTERNAL tables may write to the same location
> 
>
> Key: HIVE-24920
> URL: https://issues.apache.org/jira/browse/HIVE-24920
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>  Labels: metastore_translator, pull-request-available
> Fix For: 4.0.0, 4.0.0-alpha-1
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {code}
> create table t (a integer);
> insert into t values(1);
> alter table t rename to t2;
> create table t (a integer); -- I expected an exception from this command 
> (location already exists) but because its an external table no exception
> insert into t values(2);
> select * from t;  -- shows 1 and 2
> drop table t2;-- wipes out data location
> select * from t;  -- empty resultset
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (HIVE-16398) [Hive on Tez] Jar Localization issues

2020-08-17 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179318#comment-17179318
 ] 

tanghui edited comment on HIVE-16398 at 8/18/20, 3:06 AM:
--

May be hive-env.sh or hive-site.xml missing HIVE_AUX_JARS_PATH, you can set 
HIVE_AUX_JARS_PATH , add your jar to the path like /usr/lib/hivejar on 
HiveServer2 and Hive Metastore Server host。chown hive:hive /usr/lib/hivejar 


was (Author: sanguines):
May be hive-env.sh or hive-site.xml missing HIVE_AUX_JARS_PATH, you can set 
HIVE_AUX_JARS_PATH , add your jar to the path like /usr/lib/hivejar on 
HiveServer2 and HiveMetastore host。chown hive:hive /usr/lib/hivejar 

> [Hive on Tez] Jar Localization issues
> -
>
> Key: HIVE-16398
> URL: https://issues.apache.org/jira/browse/HIVE-16398
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 2.1.0
>Reporter: Ravi Teja Chilukuri
>Priority: Minor
>
> The jar localization fails with the following error in these cases
> 1) If a jar is already present in the classpath and is added again via 
> aux.jars or using add jar command by the user.
> 2) If the jar is missing- Ideally it should ignore or either throw a 
> error/warning stating jar not found.
> 3) When a  "*" is present in some of the classpath entries, it will try to 
> replicate the same * path on hdfs and fails.  
> Error:
> {noformat}
> Total jobs = 1
> Launching Job 1 out of 1
> Exception in thread "Thread-17" java.lang.RuntimeException: 
> java.io.IOException: Previous writer likely failed to write 
> hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
>  Failing because I am unlikely to write too.
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:617)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:526)
>   at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)
> Caused by: java.io.IOException: Previous writer likely failed to write 
> hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
>  Failing because I am unlikely to write too.
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1020)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:900)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:843)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:451)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:247)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:222)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:189)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:612)
>   ... 2 more
> FAILED: Execution Error, return code -1 from 
> org.apache.hadoop.hive.ql.exec.tez.TezTask
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-16398) [Hive on Tez] Jar Localization issues

2020-08-17 Thread tanghui (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179318#comment-17179318
 ] 

tanghui commented on HIVE-16398:


May be hive-env.sh or hive-site.xml missing HIVE_AUX_JARS_PATH, you can set 
HIVE_AUX_JARS_PATH , add your jar to the path like /usr/lib/hivejar on 
HiveServer2 and HiveMetastore host。chown hive:hive /usr/lib/hivejar 

> [Hive on Tez] Jar Localization issues
> -
>
> Key: HIVE-16398
> URL: https://issues.apache.org/jira/browse/HIVE-16398
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Affects Versions: 2.1.0
>Reporter: Ravi Teja Chilukuri
>Priority: Minor
>
> The jar localization fails with the following error in these cases
> 1) If a jar is already present in the classpath and is added again via 
> aux.jars or using add jar command by the user.
> 2) If the jar is missing- Ideally it should ignore or either throw a 
> error/warning stating jar not found.
> 3) When a  "*" is present in some of the classpath entries, it will try to 
> replicate the same * path on hdfs and fails.  
> Error:
> {noformat}
> Total jobs = 1
> Launching Job 1 out of 1
> Exception in thread "Thread-17" java.lang.RuntimeException: 
> java.io.IOException: Previous writer likely failed to write 
> hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
>  Failing because I am unlikely to write too.
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:617)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:526)
>   at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:78)
> Caused by: java.io.IOException: Previous writer likely failed to write 
> hdfs://clstr1/tmp/hive/adhoc-hive-user/_tez_session_dir/edd821a7-d4f5-4980-b693-6b9c17ac4257/hadoop-4mc-1.1.0.jar.
>  Failing because I am unlikely to write too.
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1020)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:900)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:843)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:451)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:247)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:222)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:189)
>   at 
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:612)
>   ... 2 more
> FAILED: Execution Error, return code -1 from 
> org.apache.hadoop.hive.ql.exec.tez.TezTask
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)