[
https://issues.apache.org/jira/browse/SPARK-30261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenliang updated SPARK-30261:
------------------------------
Description:
For SparkSQL,When we do some alter operations on hive table, the owner of hive
table would be changed to someone who invoked the operation, it's unresonable.
And in fact, the owner should not changed for the real prodcution environment,
otherwise the authority check is out of order.
The problem can be reproduced as described in the below:
1.First I create a table with username='xie' and then \{{desc formatted table
}},the owner is 'xiepengjie'
{code:java}
spark-sql> desc formatted bigdata_test.tt1;
col_name data_type comment c int NULL
# Detailed Table Information
Database bigdata_test Table tt1
Owner xie
Created Time Wed Sep 11 11:30:49 CST 2019
Last Access Thu Jan 01 08:00:00 CST 1970
Created By Spark 2.2 or prior
Type MANAGED
Provider hive
Table Properties [PART_LIMIT=10000, transient_lastDdlTime=1568172649, LEVEL=1,
TTL=60]
Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog Time taken: 0.371 seconds, Fetched 18 row(s)
{code}
2.Then I use another username='johnchen' and execute {{alter table
bigdata_test.tt1 set location
'hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1'}}, check the owner
of hive table is 'johnchen', it's unresonable
{code:java}
spark-sql> desc formatted bigdata_test.tt1;
col_name data_type comment c int NULL
# Detailed Table Information
Database bigdata_test
Table tt1
Owner johnchen
Created Time Wed Sep 11 11:30:49 CST 2019
Last Access Thu Jan 01 08:00:00 CST 1970
Created By Spark 2.2 or prior
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1568871017, PART_LIMIT=10000, LEVEL=1,
TTL=60]
Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.041 seconds, Fetched 18 row(s){code}
was:
For SparkSQL,When we do some alter operations on hive table, the owner of hive
table would be changed to someone who invoked the operation, it's unresonable.
And in fact, the owner should not changed for the real prodcution environment,
otherwise the authority check is out of order.
The problem can be reproduced as described in the below:
1.First I create a table with username='xie' and then \{{desc formatted table
}},the owner is 'xiepengjie'
{code:java}
spark-sql> desc formatted bigdata_test.tt1;
col_name data_type comment c int NULL
# Detailed Table Information
Database bigdata_test Table tt1
Owner xie
Created Time Wed Sep 11 11:30:49 CST 2019
Last Access Thu Jan 01 08:00:00 CST 1970
Created By Spark 2.2 or prior
Type MANAGED
Provider hive
Table Properties [PART_LIMIT=10000, transient_lastDdlTime=1568172649, LEVEL=1,
TTL=60]
Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog Time taken: 0.371 seconds, Fetched 18 row(s)
{code}
{code:java}
spark-sql> desc formatted bigdata_test.tt1;
col_name data_type comment c int NULL
# Detailed Table Information
Database bigdata_test
Table tt1
Owner johnchen
Created Time Wed Sep 11 11:30:49 CST 2019
Last Access Thu Jan 01 08:00:00 CST 1970
Created By Spark 2.2 or prior
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1568871017, PART_LIMIT=10000, LEVEL=1,
TTL=60]
Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.041 seconds, Fetched 18 row(s){code}
{{}}
> Should not change owner of hive table for some commands like 'alter'
> operation
> --------------------------------------------------------------------------------
>
> Key: SPARK-30261
> URL: https://issues.apache.org/jira/browse/SPARK-30261
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0, 2.3.0, 2.4.3
> Reporter: chenliang
> Priority: Critical
> Fix For: 2.2.0, 2.3.0, 2.4.3
>
>
> For SparkSQL,When we do some alter operations on hive table, the owner of
> hive table would be changed to someone who invoked the operation, it's
> unresonable. And in fact, the owner should not changed for the real
> prodcution environment, otherwise the authority check is out of order.
> The problem can be reproduced as described in the below:
> 1.First I create a table with username='xie' and then \{{desc formatted table
> }},the owner is 'xiepengjie'
> {code:java}
> spark-sql> desc formatted bigdata_test.tt1;
> col_name data_type comment c int NULL
> # Detailed Table Information
> Database bigdata_test Table tt1
> Owner xie
> Created Time Wed Sep 11 11:30:49 CST 2019
> Last Access Thu Jan 01 08:00:00 CST 1970
> Created By Spark 2.2 or prior
> Type MANAGED
> Provider hive
> Table Properties [PART_LIMIT=10000, transient_lastDdlTime=1568172649,
> LEVEL=1, TTL=60]
> Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
> Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
> InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
> OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
> Storage Properties [serialization.format=1]
> Partition Provider Catalog Time taken: 0.371 seconds, Fetched 18 row(s)
> {code}
> 2.Then I use another username='johnchen' and execute {{alter table
> bigdata_test.tt1 set location
> 'hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1'}}, check the owner
> of hive table is 'johnchen', it's unresonable
> {code:java}
> spark-sql> desc formatted bigdata_test.tt1;
> col_name data_type comment c int NULL
> # Detailed Table Information
> Database bigdata_test
> Table tt1
> Owner johnchen
> Created Time Wed Sep 11 11:30:49 CST 2019
> Last Access Thu Jan 01 08:00:00 CST 1970
> Created By Spark 2.2 or prior
> Type MANAGED
> Provider hive
> Table Properties [transient_lastDdlTime=1568871017, PART_LIMIT=10000,
> LEVEL=1, TTL=60]
> Location hdfs://NS1/user/hive_admin/warehouse/bigdata_test.db/tt1
> Serde Library org.apache.hadoop.hive.ql.io.orc.OrcSerde
> InputFormat org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
> OutputFormat org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
> Storage Properties [serialization.format=1]
> Partition Provider Catalog
> Time taken: 0.041 seconds, Fetched 18 row(s){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
