[
https://issues.apache.org/jira/browse/ASTERIXDB-3029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698125#comment-17698125
]
ASF subversion and git services commented on ASTERIXDB-3029:
------------------------------------------------------------
Commit 4b5a6a7f071d1540d913d40ff1c24d9d72cabfd8 in asterixdb's branch
refs/heads/master from Wail Alkowaileet
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=4b5a6a7f07 ]
[ASTERIXDB-3029][RT] Introduce LazyVisitablePointable
- user mode changes: no
- storage format changes: no
- interface changes: no
Details:
- The lazy visitable pointable allows for traversing a record and
processing it in a single pass compared to the IVisitablePointable,
which requires two passes - one for calling set() and another when
calling accept().
- Does not require writing the nested values into temporary buffers.
- However, it is suitable when traversing the records in DFS.
- Traversing and processing records efficiently is crucial for
ASTERIXDB-3002 to infer the schema and covert rows into columns.
Change-Id: Iaf2c827d80a2a7e77f5f6eb5438cea417387e132
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17412
Tested-by: Jenkins <[email protected]>
Integration-Tests: Jenkins <[email protected]>
Reviewed-by: Wail Alkowaileet <[email protected]>
Reviewed-by: Ali Alsuliman <[email protected]>
> Allow single-pass nested value traversal
> ----------------------------------------
>
> Key: ASTERIXDB-3029
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3029
> Project: Apache AsterixDB
> Issue Type: Improvement
> Components: RT - Runtime
> Affects Versions: 0.9.9
> Reporter: Wail Y. Alkowaileet
> Assignee: Wail Y. Alkowaileet
> Priority: Major
> Fix For: 0.9.9
>
>
> Currently, AsterixDB uses IVisitablePointable to allow functions (e.g.,
> IPrintVisitor) to traverse and process (print for IPrintVisitor) nested
> values. The problem is that IVisitablePointable eagerly materializes all
> nested values into temporary buffers before a nested value could be
> processed, which is expensive.
> For the columnar format, AsterixDB needs to infer the schema and columnize
> the rows during data ingestion, which requires traversing nested values.
> Using IVisitablePointable in this case could hinder the ingestion rate
> drastically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
