[
https://issues.apache.org/jira/browse/HIVE-21897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868104#comment-16868104
]
Miklos Gergely edited comment on HIVE-21897 at 6/19/19 11:40 PM:
-----------------------------------------------------------------
[~mithun] after executing those commands:
SHOW EXTENDED foobar;
{code:java}
+-----------------------------+----------------------------------------------------+----------+
| col_name | data_type
| comment |
+-----------------------------+----------------------------------------------------+----------+
| foo | string
| |
| bar | string
| |
| dt | string
| |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name | data_type
| comment |
| dt | string
| |
| | NULL
| NULL |
| Detailed Table Information | Table(tableName:foobar, dbName:default,
owner:hive, createTime:1560986681, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:foo, type:string, comment:null),
FieldSchema(name:bar, type:string, comment:null), FieldSchema(name:dt,
type:string, comment:null)],
location:hdfs://hive-on-tezt-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/foobar,
inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde,
parameters:{serialization.format=1}), bucketCols:[], sortCols:[],
parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}), storedAsSubDirectories:false),
partitionKeys:[FieldSchema(name:dt, type:string, comment:null)],
parameters:{last_modified_time=1560986885, totalSize=0, numRows=0,
rawDataSize=0, transactional_properties=insert_only,
COLUMN_STATS_ACCURATE={\"BASIC_STATS\":\"true\"}, numFiles=0, numPartitions=2,
transient_lastDdlTime=1560986885, bucketing_version=2, last_modified_by=hive,
transactional=true}, viewOriginalText:null, viewExpandedText:null,
tableType:MANAGED_TABLE, rewriteEnabled:false, catName:hive, ownerType:USER,
writeId:0) | |
{code}
SHOW CREATE TABLE foobar;
{code:java}
+----------------------------------------------------+
| createtab_stmt |
+----------------------------------------------------+
| CREATE TABLE `foobar`( |
| `foo` string, |
| `bar` string) |
| PARTITIONED BY ( |
| `dt` string) |
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
| LOCATION |
|
'hdfs://hive-on-tezt-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/foobar'
|
| TBLPROPERTIES ( |
| 'bucketing_version'='2', |
| 'last_modified_by'='hive', |
| 'last_modified_time'='1560986885', |
| 'transactional'='true', |
| 'transactional_properties'='insert_only', |
| 'transient_lastDdlTime'='1560986885') |
+----------------------------------------------------+
{code}
So as it seems the table has only one SerDe, not per partition. Do we want to
allow a different SerDe per partition? Because if we do, it needs planning, and
code changes. Or for now we may stick to the one SerDe / table.
was (Author: mgergely):
[~mithun] after executing those commands:
SHOW EXTENDED foobar;
{code:java}
+-----------------------------+----------------------------------------------------+----------+
| col_name | data_type
| comment |
+-----------------------------+----------------------------------------------------+----------+
| foo | string
| |
| bar | string
| |
| dt | string
| |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name | data_type
| comment |
| dt | string
| |
| | NULL
| NULL |
| Detailed Table Information | Table(tableName:foobar, dbName:default,
owner:hive, createTime:1560986681, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:foo, type:string, comment:null),
FieldSchema(name:bar, type:string, comment:null), FieldSchema(name:dt,
type:string, comment:null)],
location:hdfs://hive-on-tezt-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/foobar,
inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde,
parameters:{serialization.format=1}), bucketCols:[], sortCols:[],
parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}), storedAsSubDirectories:false),
partitionKeys:[FieldSchema(name:dt, type:string, comment:null)],
parameters:{last_modified_time=1560986885, totalSize=0, numRows=0,
rawDataSize=0, transactional_properties=insert_only,
COLUMN_STATS_ACCURATE={\"BASIC_STATS\":\"true\"}, numFiles=0, numPartitions=2,
transient_lastDdlTime=1560986885, bucketing_version=2, last_modified_by=hive,
transactional=true}, viewOriginalText:null, viewExpandedText:null,
tableType:MANAGED_TABLE, rewriteEnabled:false, catName:hive, ownerType:USER,
writeId:0) | |
{code}
SHOW CREATE TABLE foobar;
{code:java}
+----------------------------------------------------+
| createtab_stmt |
+----------------------------------------------------+
| CREATE TABLE `foobar`( |
| `foo` string, |
| `bar` string) |
| PARTITIONED BY ( |
| `dt` string) |
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
| LOCATION |
|
'hdfs://hive-on-tezt-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/foobar'
|
| TBLPROPERTIES ( |
| 'bucketing_version'='2', |
| 'last_modified_by'='hive', |
| 'last_modified_time'='1560986885', |
| 'transactional'='true', |
| 'transactional_properties'='insert_only', |
| 'transient_lastDdlTime'='1560986885') |
+----------------------------------------------------+
{code}
So as it seems the table has only one SerDe, not per partition. Do we want to
allow a different SerDe per partition? Because if we do, it needs planning, and
code changes. Or for now we may stick to the one SerDe / table.
> Setting serde / serde properties for partitions
> -----------------------------------------------
>
> Key: HIVE-21897
> URL: https://issues.apache.org/jira/browse/HIVE-21897
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 3.1.1
> Reporter: Miklos Gergely
> Assignee: Ashutosh Chauhan
> Priority: Major
> Fix For: 4.0.0
>
>
> According to
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AddSerDeProperties]
> the SerDe and the SerDe properties can be set for a partition too, so
>
> {code:java}
> ALTERT TABLE table PARTITION (partition_col='partition_value') SET SERDE
> 'serde.class.name';{code}
> Is a valid statement. In fact it is not rejected, but it is not doing
> anything at all. The execution is successful, everything remains the same.
> The same is true for setting the serde properties:
> {code:java}
> ALTER TABLE table PARTITION (partition_col='partition_value') SET
> SERDEPROPERTIES ('property_name'='property_value');{code}
> is also a valid statement, and not doing anything.
> I suggest to modify the parser, and reject these statements. SerDe is for a
> table, and not for a partition.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
