Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23493 )
Change subject: IMPALA-14472: Add create/read support for ARRAY column of Kudu ...................................................................... Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/23493/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/23493/4//COMMIT_MSG@12 PS4, Line 12: > Is there a documentation about Kudu array feature and mem layout? It would As for docs, there isn't anything published yet, except for the documentation on the array data block internals (FS-level details): http://gerrit.cloudera.org:8080/22058 Since that's only about internal FS-level details, I guess there isn't anything to reference in this commit message as of today. As for the memory layout, Kudu 1D arrays are encoded internally as flatbuffers. This is plain C-style layout for integer types, little-endian byte ordering; for binary/string types, Kudu's KuduArrayCellView wrapper provides access to C-style layout of Slice elements, and they point to consecutive memory ranges in the internal buffer. For direct memory access, Kudu client API provides zero-copy direct access to the internal array cell data via KuduArrayCellView facade. An example of using this: https://github.com/apache/kudu/blob/53263659ea3f05ca19c0f43f03691a75ed55e82a/examples/cpp/example.cc#L240-L261 http://gerrit.cloudera.org:8080/#/c/23493/4/be/src/exec/kudu/kudu-util-ir.cc File be/src/exec/kudu/kudu-util-ir.cc: http://gerrit.cloudera.org:8080/#/c/23493/4/be/src/exec/kudu/kudu-util-ir.cc@165 PS4, Line 165: // KuduScanner::PAD_UNIXTIME_MICROS_TO_16_BYTES does not apply to array elements. > Do you know if this is is by design or just a feature gap in Kudu? As of now, data layout within a array cell isn't affected by any of the existing row format flags (PAD_UNIXTIME_MICROS_TO_16_BYTES, COLUMNAR_LAYOUT). Since these seem to be rather inter-cell behavior (not intra-cell) in the Kudu code, I didn't realize it is something that requires implementation within an array cell. I guess it might be implemented, but I don't expect it be done in the nearest future, at least. -- To view, visit http://gerrit.cloudera.org:8080/23493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9282aac821bd30668189f84b2ed8fff7047e7310 Gerrit-Change-Number: 23493 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Comment-Date: Wed, 08 Oct 2025 20:41:51 +0000 Gerrit-HasComments: Yes
