[
https://issues.apache.org/jira/browse/KUDU-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated KUDU-1059:
------------------------------
Priority: Major (was: Critical)
> Make Kudu's wire format be compatible with Impala's tuple/row layout
> --------------------------------------------------------------------
>
> Key: KUDU-1059
> URL: https://issues.apache.org/jira/browse/KUDU-1059
> Project: Kudu
> Issue Type: Improvement
> Components: client, tserver
> Affects Versions: Feature Complete
> Reporter: David Alves
> Assignee: David Alves
>
> Kudu's wire format is actually very close to impala's and we should probably
> take it the rest of the way before we release and start to impact "released"
> clients.
> The potential performance upside for the kudu-impala integration is pretty
> big, we can copy whole rows instead of doing tuple by tuple transformations
> and eventually we can make impala just adopt the data as it arrives from kudu
> and do no copying or transformations at all.
> Here is the list of things that need addressing:
> - The bitmaps are in opposite sides of the row (Kudu's are at the end and
> Impala's are at the beginning).
> - Kudu's bitmaps are proportional to the whole column set and contain garbage
> for non-nullable columns, Impala's bitmaps only refer to the nullable columns
> (and thus do not contain garbage).
> - Impala's row layout does padding (8 byte alignment). We should mimic that,
> though it should be optional since it seems like it can be costly space wise.
> - Impala's timestamps have a different size and format from kudu's. We should
> create rowwiserow blocks with space for impala to do the transformation in
> place, versus having to memcopy the whole thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
