wei created HIVE-18143:
--------------------------
Summary: LazySimpleSerDe.LazyMap doesn't handle the condition of
when map key is another column value
Key: HIVE-18143
URL: https://issues.apache.org/jira/browse/HIVE-18143
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.14.0
Reporter: wei
LazySimpleSerDe.LazyMap doesn't handle the condition of when map key is another
column value.
eg:
CREATE TABLE `demo`(
`create_timestamp` bigint,
`ctime` string,
`event_data` map<string,string>
)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
add data
INSERT INTO TABLE demo
select
1511192567909,
'1511192567909',
map('1511192567909', '05020918127a82816437ba9edac17fca42601');
Do query:
SELECT
create_timestamp,
ctime,
event_data,
event_data[create_timestamp],
event_data[ctime]
FROM demo;
RES：
| create_timestamp | ctime | event_data
| _c3 | _c4 |
+-------------------+----------------+------------------------------------------------------------+----------------------------------------+-------+--+
| 1511192567909 | 1511192567909 |
{"1511192567909":"05020918127a82816437ba9edac17fca42601"} |
05020918127a82816437ba9edac17fca42601 | NULL |
event_data[ctime] get value NULL
_______________________________________
In LazyMap.getMapValueElement method.
{code:java}
// Some comments here
public Object getMapValueElement(Object key) {
if (!parsed) {
parse();
}
// search for the key
for (int i = 0; i < mapSize; i++) {
LazyPrimitive<?, ?> lazyKeyI = uncheckedGetKey(i);
if (lazyKeyI == null) {
continue;
}
// getWritableObject() will convert LazyPrimitive to actual primitive
// writable objects.
Object keyI = lazyKeyI.getWritableObject();
if (keyI == null) {
continue;
}
if (keyI.equals(key)) {
// Got a match, return the value
return uncheckedGetValue(i);
}
}
return null;
}
{code}
When the key is a cloumn and don't need convert, the key will be a
LazyPrimitive object.
keyI = lazyKeyI.getWritableObject() is actual primitive writable objects.
keyI.equals(key) will be false.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
