[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=837297=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837297 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 05/Jan/23 18:33 Start Date: 05/Jan/23 18:33 Worklog Time Spent: 10m Work Description: ramesh0201 merged PR #3854: URL: https://github.com/apache/hive/pull/3854 Issue Time Tracking --- Worklog Id: (was: 837297) Time Spent: 2h 20m (was: 2h 10m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=837123=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837123 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 05/Jan/23 04:38 Start Date: 05/Jan/23 04:38 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1371773551 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3854) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3854=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3854=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 837123) Time Spent: 2h 10m (was: 2h) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=837096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837096 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 05/Jan/23 00:49 Start Date: 05/Jan/23 00:49 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1371601883 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3854) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3854=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3854=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 837096) Time Spent: 2h (was: 1h 50m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=837093=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837093 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 05/Jan/23 00:17 Start Date: 05/Jan/23 00:17 Worklog Time Spent: 10m Work Description: saihemanth-cloudera commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1371572142 Pending green jenkins tests. Issue Time Tracking --- Worklog Id: (was: 837093) Time Spent: 1h 50m (was: 1h 40m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=837085=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-837085
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 04/Jan/23 23:43
Start Date: 04/Jan/23 23:43
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1061974132
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
@saihemanth-cloudera Can you review the latest?
Issue Time Tracking
---
Worklog Id: (was: 837085)
Time Spent: 1h 40m (was: 1.5h)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default. Use below to reproduce.
> create external table test_ext(empno int, name string) partitioned by(dept
> string) stored as orc;
> desc formatted test_ext;
> set hive.create.as.external.legacy=true;
> create table test_external like test_ext;
> desc formatted test_external;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=836500=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-836500
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 03/Jan/23 05:27
Start Date: 03/Jan/23 05:27
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1060286475
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
@ramesh0201 - There are a couple of other classes that distinguish managed
and external locations but those are irrelevant to the current functionality.
So I would suggest adding a new method to this class. This new method would use
an HMS client that is built based on configs as you mentioned in the above .q
file, and use this client to create a table so that the created table is
translated to an external table based on the configs, then we can do an assert
on the table locations.
Issue Time Tracking
---
Worklog Id: (was: 836500)
Time Spent: 1.5h (was: 1h 20m)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default. Use below to reproduce.
> create external table test_ext(empno int, name string) partitioned by(dept
> string) stored as orc;
> desc formatted test_ext;
> set hive.create.as.external.legacy=true;
> create table test_external like test_ext;
> desc formatted test_external;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=835377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-835377
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 22/Dec/22 18:10
Start Date: 22/Dec/22 18:10
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1055728771
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
@saihemanth-cloudera Historically this test has been flaky for multiple
times and been ignored sometime before. I am reluctant to bring this back. Do
you have a different class/test that you used for previous work in this area?
Issue Time Tracking
---
Worklog Id: (was: 835377)
Time Spent: 1h 20m (was: 1h 10m)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default. Use below to reproduce.
> create external table test_ext(empno int, name string) partitioned by(dept
> string) stored as orc;
> desc formatted test_ext;
> set hive.create.as.external.legacy=true;
> create table test_external like test_ext;
> desc formatted test_external;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=835158=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-835158
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 21/Dec/22 23:22
Start Date: 21/Dec/22 23:22
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1054930790
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
Yeah looks like in the TestDatabaseTableDefault class, the
metastore.metadata.transformer config is coming as empty and as a result, the
transformer is not coming into effect which leaves managed and external paths
as same.
Can you use this class
https://github.com/apache/hive/blob/master/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestWarehouseExternalDir.java#L175
to test your patch? This class was able to distinguish between managed and
external directories.
Issue Time Tracking
---
Worklog Id: (was: 835158)
Time Spent: 1h 10m (was: 1h)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default. Use below to reproduce.
> create external table test_ext(empno int, name string) partitioned by(dept
> string) stored as orc;
> desc formatted test_ext;
> set hive.create.as.external.legacy=true;
> create table test_external like test_ext;
> desc formatted test_external;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=835138=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-835138 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 21/Dec/22 20:04 Start Date: 21/Dec/22 20:04 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1362029767 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3854) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3854=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3854=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 835138) Time Spent: 1h (was: 50m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=834947=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-834947 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 21/Dec/22 00:14 Start Date: 21/Dec/22 00:14 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1360542168 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3854) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3854=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3854=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 834947) Time Spent: 50m (was: 40m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=834943=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-834943
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 20/Dec/22 23:22
Start Date: 20/Dec/22 23:22
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1053838306
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
Looked at unit tests at TestDatabaseTableDefault that identify the table
location, but in managed and external tables both seem to have the same path.
Do we have a way to identify or differentiate the paths?
Issue Time Tracking
---
Worklog Id: (was: 834943)
Time Spent: 40m (was: 0.5h)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default. Use below to reproduce.
> create external table test_ext(empno int, name string) partitioned by(dept
> string) stored as orc;
> desc formatted test_ext;
> set hive.create.as.external.legacy=true;
> create table test_external like test_ext;
> desc formatted test_external;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[
https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=833927=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-833927
]
ASF GitHub Bot logged work on HIVE-26837:
-
Author: ASF GitHub Bot
Created on: 15/Dec/22 18:03
Start Date: 15/Dec/22 18:03
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on code in PR #3854:
URL: https://github.com/apache/hive/pull/3854#discussion_r1049981315
##
ql/src/test/queries/clientpositive/ctlt_translate_external.q:
##
@@ -0,0 +1,20 @@
+set hive.fetch.task.conversion=none;
+set hive.compute.query.using.stats=false;
+set hive.support.concurrency=true;
+set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
+set hive.strict.managed.tables=true;
+set hive.create.as.acid=true;
+set hive.create.as.insert.only=true;
+set hive.default.fileformat.managed=ORC;
+
+create table test_mm(empno int, name string) partitioned by(dept string)
stored as orc tblproperties('transactional'='true',
'transactional_properties'='default');
Review Comment:
You don't need to mention ('transactional'='true',
'transactional_properties'='default') properties here. Since
hive.create.as.acid is set to true, create table by default creates a
transactional table.
##
ql/src/test/results/clientpositive/llap/ctlt_translate_external.q.out:
##
@@ -0,0 +1,108 @@
+PREHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_mm
+POSTHOOK: query: create table test_mm(empno int, name string) partitioned
by(dept string) stored as orc tblproperties('transactional'='true',
'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_mm
+PREHOOK: query: create table test_external like test_mm
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_external
+POSTHOOK: query: create table test_external like test_mm
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_external
+PREHOOK: query: desc formatted test_external
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@test_external
+POSTHOOK: query: desc formatted test_external
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@test_external
+# col_name data_type comment
+empno int
+name string
+
+# Partition Information
+# col_name data_type comment
+dept string
+
+# Detailed Table Information
+Database: default
+ A masked pattern was here
+Retention: 0
+ A masked pattern was here
+Table Type:MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ bucketing_version 2
+ numFiles0
+ numPartitions 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+ transactional true
+ transactional_propertiesdefault
+ A masked pattern was here
Review Comment:
Since the output location is masked out here, we really don't know if the
table is pointing to managed or external location. I think it would be nice to
have Junit test and then assert on table location. Then we can make sure no
regressions happen in the future.
Issue Time Tracking
---
Worklog Id: (was: 833927)
Time Spent: 0.5h (was: 20m)
> CTLT with hive.create.as.external.legacy as true creates managed table
> instead of external table
>
>
> Key: HIVE-26837
> URL: https://issues.apache.org/jira/browse/HIVE-26837
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When CTLT is used with the config hive.create.as.external.legacy=true, it
> still creates managed table by default.
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=833233=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-833233 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 14/Dec/22 01:39 Start Date: 14/Dec/22 01:39 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3854: URL: https://github.com/apache/hive/pull/3854#issuecomment-1350225138 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3854) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3854=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3854=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3854=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3854=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 833233) Time Spent: 20m (was: 10m) > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26837) CTLT with hive.create.as.external.legacy as true creates managed table instead of external table
[ https://issues.apache.org/jira/browse/HIVE-26837?focusedWorklogId=833227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-833227 ] ASF GitHub Bot logged work on HIVE-26837: - Author: ASF GitHub Bot Created on: 14/Dec/22 00:47 Start Date: 14/Dec/22 00:47 Worklog Time Spent: 10m Work Description: ramesh0201 opened a new pull request, #3854: URL: https://github.com/apache/hive/pull/3854 …anaged table instead of external table ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 833227) Remaining Estimate: 0h Time Spent: 10m > CTLT with hive.create.as.external.legacy as true creates managed table > instead of external table > > > Key: HIVE-26837 > URL: https://issues.apache.org/jira/browse/HIVE-26837 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When CTLT is used with the config hive.create.as.external.legacy=true, it > still creates managed table by default. Use below to reproduce. > create external table test_ext(empno int, name string) partitioned by(dept > string) stored as orc; > desc formatted test_ext; > set hive.create.as.external.legacy=true; > create table test_external like test_ext; > desc formatted test_external; -- This message was sent by Atlassian Jira (v8.20.10#820010)
