Zoltán Borók-Nagy created IMPALA-9484:
-----------------------------------------
Summary: Milestone 1: properly scan files that has full ACID schema
Key: IMPALA-9484
URL: https://issues.apache.org/jira/browse/IMPALA-9484
Project: IMPALA
Issue Type: Sub-task
Reporter: Zoltán Borók-Nagy
Full ACID row format looks like this:
{
"operation": 0,
"originalTransaction": 1,
"bucket": 536870912,
"rowId": 0,
"currentTransaction": 1,
"row": \{"i": 1}
}
User columns are nested under "row". The frontend should create proper tuples
and slot descriptors for the scan nodes to read the files correctly.
We should be able to query the ACID columns, at least for debugging/testing.
Hive uses the special “row__id” identifier for that.
Impala should raise an error if there are delete deltas. Directory filtering
should filter out minor compacted directories since the records from those need
validation.
Non-goals in this sub-task:
* row validation against validWriteIdList
* reading "original files" (files in non-ACID format)
* reading delete deltas
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
