[
https://issues.apache.org/jira/browse/HIVE-25963?focusedWorklogId=735855&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-735855
]
ASF GitHub Bot logged work on HIVE-25963:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Mar/22 10:49
Start Date: 03/Mar/22 10:49
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #3040:
URL: https://github.com/apache/hive/pull/3040#discussion_r818520185
##########
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -1247,31 +1247,11 @@ public void createTable(Table tbl) throws
AlreadyExistsException,
public void createTable(Table tbl, EnvironmentContext envContext) throws
AlreadyExistsException,
InvalidObjectException, MetaException, NoSuchObjectException, TException
{
- if (!tbl.isSetCatName()) {
- tbl.setCatName(getDefaultCatalog(conf));
Review comment:
catalog name needs to be set if it is not set. Can you please add these
lines back?
##########
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##########
@@ -1247,31 +1247,11 @@ public void createTable(Table tbl) throws
AlreadyExistsException,
public void createTable(Table tbl, EnvironmentContext envContext) throws
AlreadyExistsException,
InvalidObjectException, MetaException, NoSuchObjectException, TException
{
- if (!tbl.isSetCatName()) {
- tbl.setCatName(getDefaultCatalog(conf));
- }
- HiveMetaHook hook = getHook(tbl);
- if (hook != null) {
- hook.preCreateTable(tbl);
- }
- boolean success = false;
- try {
- // Subclasses can override this step (for example, for temporary tables)
- create_table_with_environment_context(tbl, envContext);
- if (hook != null) {
- hook.commitCreateTable(tbl);
- }
- success = true;
- }
- finally {
- if (!success && (hook != null)) {
- try {
- hook.rollbackCreateTable(tbl);
- } catch (Exception e){
- LOG.error("Create rollback failed with", e);
- }
- }
+ CreateTableRequest request = new CreateTableRequest(tbl);
+ if (envContext != null) {
+ request.setEnvContext(envContext);
}
+ createTable(request);
Review comment:
Can you please add L4448-4451 to the CreateTableRequest object here
before calling the CreateTable() api? We would need to send the processor
capabilities (if available) to HMS.
##########
File path:
ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out
##########
@@ -6411,53 +6364,16 @@ STAGE PLANS:
expressions: UDFToInteger(key) (type: int)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 2000 Basic stats:
COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: enforce_constraint(_col0 is not null) (type:
boolean)
Review comment:
I think this is the right behavior. Constraints on temp tables will not
be persisted in metastore and hence the explain plan shouldn't show stats e.t.c.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 735855)
Time Spent: 1h 10m (was: 1h)
> Temporary table creation with not null constraint gets converted to external
> table
> -----------------------------------------------------------------------------------
>
> Key: HIVE-25963
> URL: https://issues.apache.org/jira/browse/HIVE-25963
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
> Reporter: Sourabh Goyal
> Assignee: Sourabh Goyal
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When creating a temporary table with not null, constraint it gets covered to
> external table. For example:
> create temporary table t2 (a int not null);
> table t2' metadata looks like:
> {code:java}
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | col_name | data_type
> | comment |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | a | int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | default
> | NULL |
> | OwnerType: | USER
> | NULL |
> | Owner: | sourabh
> | NULL |
> | CreateTime: | Tue Feb 15 15:20:13 PST 2022
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Retention: | 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/tmp/hive/sourabh/80d374a8-cd7a-4fcf-ae72-51b04ff9c3d8/_tmp_space.db/4574446d-c144-48f9-b4b6-2e9ee0ce5be4
> | NULL |
> | Table Type: | EXTERNAL_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | COLUMN_STATS_ACCURATE
> | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\"}} |
> | | EXTERNAL
> | TRUE |
> | | TRANSLATED_TO_EXTERNAL
> | TRUE |
> | | bucketing_version
> | 2 |
> | | external.table.purge
> | TRUE |
> | | numFiles
> | 0 |
> | | numRows
> | 0 |
> | | rawDataSize
> | 0 |
> | | totalSize
> | 0 |
> | | transient_lastDdlTime
> | 1644967213 |
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library: |
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
> | NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | -1
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> | Storage Desc Params: | NULL
> | NULL |
> | | serialization.format
> | 1 |
> | | NULL
> | NULL |
> | # Constraints | NULL
> | NULL |
> | | NULL
> | NULL |
> | # Not Null Constraints | NULL
> | NULL |
> | Table: | default.t2
> | NULL |
> | Constraint Name: | nn_157620106_1644967213860_0
> | NULL |
> | Column Name: | a
> | NULL |
> | | NULL
> | NULL |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> {code}
> However if a temporary table is created *without,* not null constraint, it
> works as expected. For example:
> create temporary table t5 (a int);
>
> {code:java}
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | col_name | data_type
> | comment |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | a | int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | default
> | NULL |
> | OwnerType: | USER
> | NULL |
> | Owner: | sourabh
> | NULL |
> | CreateTime: | Tue Feb 15 15:34:44 PST 2022
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Retention: | 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/tmp/hive/sourabh/22fcb5c1-b9a6-4d51-8efc-86d7c167a242/_tmp_space.db/0515167f-ca1a-4c53-a1de-e4385fc96079
> | NULL |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | COLUMN_STATS_ACCURATE
> | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\"}} |
> | | bucketing_version
> | 2 |
> | | numFiles
> | 0 |
> | | numRows
> | 0 |
> | | rawDataSize
> | 0 |
> | | totalSize
> | 0 |
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library: |
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
> | NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | -1
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> | Storage Desc Params: | NULL
> | NULL |
> | | serialization.format
> | 1 |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
>
> {code}
> Temporary tables are managed tables as HS2 keeps them in memory. Their
> metadata is not persisted in HMS.
> But the above temp table t2 got converted into external table metadata for
> which gets persisted into HMS which is not the desired behavior.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
