[
https://issues.apache.org/jira/browse/KYLIN-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701796#comment-16701796
]
Wu Bin commented on KYLIN-1111:
-------------------------------
May I have a try?
Based on my understanding, there is already a method to fit some gaps between
Hive's columnTypes & Kylin's :
HiveMetadataExplorer#extractColumnFromMeta:
{code:java}
private ColumnDesc[] extractColumnFromMeta(HiveTableMeta hiveTableMeta) {
int columnNumber = hiveTableMeta.allColumns.size();
List<ColumnDesc> columns = new ArrayList<ColumnDesc>(columnNumber);
for (int i = 0; i < columnNumber; i++) {
HiveTableMeta.HiveTableColumnMeta field =
hiveTableMeta.allColumns.get(i);
ColumnDesc cdesc = new ColumnDesc();
cdesc.setName(field.name.toUpperCase(Locale.ROOT));
// use "double" in kylin for "float"
if ("float".equalsIgnoreCase(field.dataType)) {
cdesc.setDatatype("double");
} else {
cdesc.setDatatype(field.dataType);
}
cdesc.setId(String.valueOf(i + 1));
cdesc.setComment(field.comment);
columns.add(cdesc);
}
return columns.toArray(new ColumnDesc[columnNumber]);
}
{code}
Can we filter the unsupported columns during the loop?
> Ignore unsupported hive column types when sync hive table
> ---------------------------------------------------------
>
> Key: KYLIN-1111
> URL: https://issues.apache.org/jira/browse/KYLIN-1111
> Project: Kylin
> Issue Type: Improvement
> Reporter: liyang
> Priority: Major
> Labels: newbie
>
> Currently unsupported hive column type like map, array will cause failing to
> load hive table. Instead, we could just ignore the unsupported column, and
> still load the table with remaining columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
