[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenjianjian updated HIVE-11427:
Description:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions using CREATE TABLE AS SELECT from a table in the default
database, the following happens:
{code:java}
use default;
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
{code}
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
[https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278]
{code:java}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
// and then something incorrect happens in getExtTmpPathRelTo()
{code}
In any event - the bug is that the location chosen for the temporary storage is
not in the same place as the target table. It should be same as the target
table (/user/hive/warehouse/grisha.db in the above example) because this is
where presumably the user running the query would have write permissions to.
was:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions using CREATE TABLE AS SELECT from a table in the default
database, the following happens:
{code}
use default;
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
{code}
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
// and then something incorrect happens in getExtTmpPathRelTo()
{code}
In any event - the bug is that the location chosen for the temporary storage is
not in the same place as the target table. It should be same as the target
table (/user/hive/warehouse/grisha.db in the above example) because this is
where presumably the user running the query would have write permissions to.
> Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
>
>
> Key: HIVE-11427
> URL: https://issues.apache.org/jira/browse/HIVE-11427
> Project: Hive
> Issue Type: Bug
>Reporter: Gregory Trubetskoy
>Assignee: Yongzhi Chen
>Priority: Major
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-11427.1.patch, HIVE-11427.2.patch
>
>
> If a user _does not_ have HDFS write permissions to the _default_ database,
> and attempts to create a table in a _private_ database to which the user
> _does_ have permissions using CREATE TABLE AS SELECT from a table in the
> default database, the following happens:
> {code:java}
> use default;
> create table grisha.blahblah as select * from some_table;
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
> 'TOK_TMP_FILE’
> {code}
> I've edited this issue because my initial explanation was completely bogus. A
> more likely explanation is in
> [https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278]
> {code:java}
> -fname = ctx.getExternalTmpPath(
> +fname = ctx.getExtTmpPathRelTo(
> // and then something incorrect happens in getExtTmpPathRelTo()
> {code}
> In any event - the bug is that the location chosen for the temporary storage
> is not in the same place as the target table. It should be same as the target
> table (/user/hive/warehouse/grisha.db in the above example) because this is
> where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-11427:
Resolution: Fixed
Fix Version/s: 2.1.0
1.3.0
Status: Resolved (was: Patch Available)
Committed to master and branch-1
> Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
>
>
> Key: HIVE-11427
> URL: https://issues.apache.org/jira/browse/HIVE-11427
> Project: Hive
> Issue Type: Bug
>Reporter: Grisha Trubetskoy
>Assignee: Yongzhi Chen
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-11427.1.patch, HIVE-11427.2.patch
>
>
> If a user _does not_ have HDFS write permissions to the _default_ database,
> and attempts to create a table in a _private_ database to which the user
> _does_ have permissions using CREATE TABLE AS SELECT from a table in the
> default database, the following happens:
> {code}
> use default;
> create table grisha.blahblah as select * from some_table;
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
> 'TOK_TMP_FILE’
> {code}
> I've edited this issue because my initial explanation was completely bogus. A
> more likely explanation is in
> https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
> {code}
> -fname = ctx.getExternalTmpPath(
> +fname = ctx.getExtTmpPathRelTo(
> // and then something incorrect happens in getExtTmpPathRelTo()
> {code}
> In any event - the bug is that the location chosen for the temporary storage
> is not in the same place as the target table. It should be same as the target
> table (/user/hive/warehouse/grisha.db in the above example) because this is
> where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-11427:
Attachment: HIVE-11427.2.patch
[~spena], I attached the second patch with an encryption test.
> Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
>
>
> Key: HIVE-11427
> URL: https://issues.apache.org/jira/browse/HIVE-11427
> Project: Hive
> Issue Type: Bug
>Reporter: Grisha Trubetskoy
>Assignee: Yongzhi Chen
> Attachments: HIVE-11427.1.patch, HIVE-11427.2.patch
>
>
> If a user _does not_ have HDFS write permissions to the _default_ database,
> and attempts to create a table in a _private_ database to which the user
> _does_ have permissions using CREATE TABLE AS SELECT from a table in the
> default database, the following happens:
> {code}
> use default;
> create table grisha.blahblah as select * from some_table;
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
> 'TOK_TMP_FILE’
> {code}
> I've edited this issue because my initial explanation was completely bogus. A
> more likely explanation is in
> https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
> {code}
> -fname = ctx.getExternalTmpPath(
> +fname = ctx.getExtTmpPathRelTo(
> // and then something incorrect happens in getExtTmpPathRelTo()
> {code}
> In any event - the bug is that the location chosen for the temporary storage
> is not in the same place as the target table. It should be same as the target
> table (/user/hive/warehouse/grisha.db in the above example) because this is
> where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-11427:
Status: Patch Available (was: Open)
Need code review.
> Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
>
>
> Key: HIVE-11427
> URL: https://issues.apache.org/jira/browse/HIVE-11427
> Project: Hive
> Issue Type: Bug
>Reporter: Grisha Trubetskoy
>Assignee: Yongzhi Chen
> Attachments: HIVE-11427.1.patch
>
>
> If a user _does not_ have HDFS write permissions to the _default_ database,
> and attempts to create a table in a _private_ database to which the user
> _does_ have permissions using CREATE TABLE AS SELECT from a table in the
> default database, the following happens:
> {code}
> use default;
> create table grisha.blahblah as select * from some_table;
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
> 'TOK_TMP_FILE’
> {code}
> I've edited this issue because my initial explanation was completely bogus. A
> more likely explanation is in
> https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
> {code}
> -fname = ctx.getExternalTmpPath(
> +fname = ctx.getExtTmpPathRelTo(
> // and then something incorrect happens in getExtTmpPathRelTo()
> {code}
> In any event - the bug is that the location chosen for the temporary storage
> is not in the same place as the target table. It should be same as the target
> table (/user/hive/warehouse/grisha.db in the above example) because this is
> where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-11427:
Attachment: HIVE-11427.1.patch
> Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
>
>
> Key: HIVE-11427
> URL: https://issues.apache.org/jira/browse/HIVE-11427
> Project: Hive
> Issue Type: Bug
>Reporter: Grisha Trubetskoy
>Assignee: Yongzhi Chen
> Attachments: HIVE-11427.1.patch
>
>
> If a user _does not_ have HDFS write permissions to the _default_ database,
> and attempts to create a table in a _private_ database to which the user
> _does_ have permissions using CREATE TABLE AS SELECT from a table in the
> default database, the following happens:
> {code}
> use default;
> create table grisha.blahblah as select * from some_table;
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
> 'TOK_TMP_FILE’
> {code}
> I've edited this issue because my initial explanation was completely bogus. A
> more likely explanation is in
> https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
> {code}
> -fname = ctx.getExternalTmpPath(
> +fname = ctx.getExtTmpPathRelTo(
> // and then something incorrect happens in getExtTmpPathRelTo()
> {code}
> In any event - the bug is that the location chosen for the temporary storage
> is not in the same place as the target table. It should be same as the target
> table (/user/hive/warehouse/grisha.db in the above example) because this is
> where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grisha Trubetskoy updated HIVE-11427:
-
Description:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions, the following happens:
{code}
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
{code}
In any event - the bug is that the location chosen for the temporary storage
has to be in the same place as the target table because that is where
presumably the user running the query would have write permissions to.
was:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions, the following happens:
{code}
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
{code}
The reason for this seems to be
https://github.com/apache/hive/commit/05a2aff71c2682e01331cd333189ce7802233a75#diff-f2040374293a91cbcc6594ee571b20e4L1425,
specifically this line:
https://github.com/apache/hive/blob/branch-0.14/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L1787,
which changed like this in the aforementioned commit:
{code}
-location =
wh.getDatabasePath(db.getDatabase(newTable.getDbName()));
+location = wh.getDatabasePath(db.getDatabase(names[0]));
{code}
So before the database of the new table was used, and now the database of the
table from the select is used as I understand it.
NB: This was all inferred from just reading the code, I have not verified it.
Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
Key: HIVE-11427
URL: https://issues.apache.org/jira/browse/HIVE-11427
Project: Hive
Issue Type: Bug
Reporter: Grisha Trubetskoy
If a user _does not_ have HDFS write permissions to the _default_ database,
and attempts to create a table in a _private_ database to which the user
_does_ have permissions, the following happens:
{code}
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
{code}
In any event - the bug is that the location chosen for the temporary storage
has to be in the same place as the target table because that is where
presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-11427) Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
[
https://issues.apache.org/jira/browse/HIVE-11427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grisha Trubetskoy updated HIVE-11427:
-
Description:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions using CREATE TABLE AS SELECT from a table in the default
database, the following happens:
{code}
use default;
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
{code}
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
// and then something incorrect happens in getExtTmpPathRelTo()
{code}
In any event - the bug is that the location chosen for the temporary storage is
not in the same place as the target table. It should be same as the target
table (/user/hive/warehouse/grisha.db in the above example) because this is
where presumably the user running the query would have write permissions to.
was:
If a user _does not_ have HDFS write permissions to the _default_ database, and
attempts to create a table in a _private_ database to which the user _does_
have permissions, the following happens:
{code}
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
{code}
In any event - the bug is that the location chosen for the temporary storage
has to be in the same place as the target table because that is where
presumably the user running the query would have write permissions to.
Location of temporary table for CREATE TABLE SELECT broken by HIVE-7079
Key: HIVE-11427
URL: https://issues.apache.org/jira/browse/HIVE-11427
Project: Hive
Issue Type: Bug
Reporter: Grisha Trubetskoy
If a user _does not_ have HDFS write permissions to the _default_ database,
and attempts to create a table in a _private_ database to which the user
_does_ have permissions using CREATE TABLE AS SELECT from a table in the
default database, the following happens:
{code}
use default;
create table grisha.blahblah as select * from some_table;
FAILED: SemanticException 0:0 Error creating temporary folder on:
hdfs://nn.example.com/user/hive/warehouse. Error encountered near token
'TOK_TMP_FILE’
{code}
I've edited this issue because my initial explanation was completely bogus. A
more likely explanation is in
https://github.com/apache/hive/commit/1614314ef7bd0c3b8527ee32a434ababf7711278
{code}
-fname = ctx.getExternalTmpPath(
+fname = ctx.getExtTmpPathRelTo(
// and then something incorrect happens in getExtTmpPathRelTo()
{code}
In any event - the bug is that the location chosen for the temporary storage
is not in the same place as the target table. It should be same as the target
table (/user/hive/warehouse/grisha.db in the above example) because this is
where presumably the user running the query would have write permissions to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
