David Alves reassigned KUDU-1059:

    Assignee:     (was: David Alves)

> Make Kudu's wire format be compatible with Impala's tuple/row layout
> --------------------------------------------------------------------
>                 Key: KUDU-1059
>                 URL: https://issues.apache.org/jira/browse/KUDU-1059
>             Project: Kudu
>          Issue Type: Improvement
>          Components: client, tserver
>    Affects Versions: Feature Complete
>            Reporter: David Alves
> Kudu's wire format is actually very close to impala's and we should probably 
> take it the rest of the way before we release and start to impact "released" 
> clients.
> The potential performance upside for the kudu-impala integration is pretty 
> big, we can copy whole rows instead of doing tuple by tuple transformations 
> and eventually we can make impala just adopt the data as it arrives from kudu 
> and do no copying or transformations at all.
> Here is the list of things that need addressing:
> - The bitmaps are in opposite sides of the row (Kudu's are at the end and 
> Impala's are at the beginning).
> - Kudu's bitmaps are proportional to the whole column set and contain garbage 
> for non-nullable columns, Impala's bitmaps only refer to the nullable columns 
> (and thus do not contain garbage).
> - Impala's row layout does padding (8 byte alignment). We should mimic that, 
> though it should be optional since it seems like it can be costly space wise.
> - Impala's timestamps have a different size and format from kudu's. We should 
> create rowwiserow blocks with space for impala to do the transformation in 
> place, versus having to memcopy the whole thing.

This message was sent by Atlassian JIRA

Reply via email to