[
https://issues.apache.org/jira/browse/ASTERIXDB-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718364#comment-17718364
]
ASF subversion and git services commented on ASTERIXDB-3169:
------------------------------------------------------------
Commit 964097b10d4650a2aaaee59af0decf74e52f9e4e in asterixdb's branch
refs/heads/master from Wail Alkowaileet
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=964097b10d ]
[ASTERIXDB-3169] Fix column assembler for repeated complex values
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
See ASTERIXDB-3169 for details about the issue
Change-Id: Ib2cede772f3db534b6c0c93c10d51ec076747e19
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17496
Integration-Tests: Jenkins <[email protected]>
Tested-by: Jenkins <[email protected]>
Reviewed-by: Wail Alkowaileet <[email protected]>
Reviewed-by: Murtadha Hubail <[email protected]>
> Column Assembler fails on MISSING repeated values
> -------------------------------------------------
>
> Key: ASTERIXDB-3169
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3169
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: RT - Runtime
> Affects Versions: 0.9.9
> Reporter: Wail Y. Alkowaileet
> Assignee: Wail Y. Alkowaileet
> Priority: Critical
> Fix For: 0.9.9
>
>
> While testing the assembly cost, I found an issue with certain types of
> columns (namely array items with union types). Upon querying, an exception
> was thrown:
> {code:java}
> Caused by: java.lang.IllegalAccessError: no more values, column index: 924
> at
> org.apache.asterix.column.assembler.RepeatedPrimitiveValueAssembler.next(RepeatedPrimitiveValueAssembler.java:44)
> ~[classes/:?]
> at
> org.apache.asterix.column.operation.query.ColumnAssembler.nextValue(ColumnAssembler.java:80)
> ~[classes/:?]
> at
> org.apache.asterix.column.tuple.QueryColumnTupleReference.getAssembledValue(QueryColumnTupleReference.java:133)
> ~[classes/:?]
> at
> org.apache.asterix.column.operation.query.QueryColumnTupleProjector.getAssembledValue(QueryColumnTupleProjector.java:107)
> ~[classes/:?]
> at
> org.apache.asterix.column.operation.query.QueryColumnTupleProjector.project(QueryColumnTupleProjector.java:84)
> ~[classes/:?]
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeTupleToOutput(IndexSearchOperatorNodePushable.java:390)
> ~[classes/:?]
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:270)
> ~[classes/:?]
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.searchAllPartitions(IndexSearchOperatorNodePushable.java:459)
> ~[classes/:?]
> at
> org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:312)
> ~[classes/:?]
> at
> org.apache.hyracks.api.dataflow.EnforceFrameWriter.nextFrame(EnforceFrameWriter.java:76)
> ~[classes/:?]
> at
> org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:94)
> ~[classes/:?]
> at
> org.apache.hyracks.algebricks.runtime.operators.std.EmptyTupleSourceRuntimeFactory$1.open(EmptyTupleSourceRuntimeFactory.java:55)
> ~[classes/:?]
> at
> org.apache.hyracks.api.dataflow.EnforceFrameWriter.open(EnforceFrameWriter.java:59)
> ~[classes/:?]
> at
> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$SourcePushRuntime.initialize(AlgebricksMetaOperatorDescriptor.java:105)
> ~[classes/:?]
> at
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$0(SuperActivityOperatorNodePushable.java:236)
> ~[classes/:?]
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
> ~[?:?]
> at java.util.concurrent.FutureTask.run(FutureTask.java) ~[?:?] {code}
> Schema of the affected column(s)
> {code:java}
> root
> |-- geo: object <level: 1>
> | |-- coordinates: array <level: 2>
> | | |-- item: union <level: 3>
> | | | |-- bigint: bigint <level: 3, index: 924>
> | | | |-- double: double <level: 3, index: 888> {code}
>
> *After Investigation:*
> The problem appears in a certain circumstance where the first _*N*_ pages are
> written with the a schema as follows:
> {code:java}root
> |-- geo: object <level: 1>
> | |-- coordinates: array <level: 2>
> | | |-- item: double <level: 3, index: 888> {code}
> Then, for all subsequent pages (i.e., {*}{_}N+M{_}){*}, the schema changes to:
> {code:java}root
> |-- geo: object <level: 1>
> | |-- coordinates: array <level: 2>
> | | |-- item: union <level: 3>
> | | | |-- bigint: bigint <level: 3, index: 924>
> | | | |-- double: double <level: 3, index: 888> {code}
> So the first *N* pages are missing column 924. However, the component's
> ({_}*with # of* *pages > N*{_}) schema contains the information of column
> 924. The assembler was blindly setting the delegate for the array
> *coordinates* (from the schema) to column 924 – a newer column for the array
> *coordinates.* Hence, column 924 does not contain all information necessary
> to assemble {*}coordinates{*}.
>
> This is not a union issue per-se, but could also appear with objects with new
> fields with array types.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
