[
https://issues.apache.org/jira/browse/HIVE-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954108#comment-14954108
]
Mohammad Kamrul Islam commented on HIVE-12080:
----------------------------------------------
I'm considering two possible options to support this:
Option 1:
Replace get() method of WritableLongObjectInspector
class with the following code. (Same for IntOI and DoubleOI)
{code}
public long get(Object o) {
if(o instanceof LongWritable) {
return ((LongWritable) o).get();
} else if(o instanceof IntWritable) {
return (long)((IntWritable) o).get();
} else if(o instanceof ShortWritable) {
return (long) ((ShortWritable) o).get();
} else {
throw new RuntimeException("Type mismatched :" + o.getClass() + "
Expecting either " +
"LongWritable/IntWritable/ShortWriteable");
}
//return ((LongWritable) o).get();
}
{code}
I tested a basic use case and it is working. It is adding one extra "if" which
I assume may not impact the performance.
Option 2:
Modify ETypeConverter:getConverter() method that was adopted in PigLoader
(https://issues.apache.org/jira/browse/PARQUET-2).
this will need newer version of Parquet. (1.6+)
Planning to add a patch based on Option #1
[~spena] : could you please make comment on this?
> Support auto type widening for Parquet table
> --------------------------------------------
>
> Key: HIVE-12080
> URL: https://issues.apache.org/jira/browse/HIVE-12080
> Project: Hive
> Issue Type: New Feature
> Components: File Formats
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
>
> Currently Hive+Parquet doesn't support it. It should include at least basic
> type promotions short->int->bigint, float->double etc, that are already
> supported for other file formats.
> There were similar effort (Hive-6784) but was not committed. This JIRA is to
> address the same in different way with little (no) performance impact.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
