Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2254#discussion_r185433421
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/dev/DataMapWriter.java ---
@@ -66,22 +85,22 @@ public DataMapWriter(AbsoluteTableIdentifier
identifier, Segment segment,
*
* @param blockletId sequence number of blocklet in the block
*/
- public abstract void onBlockletStart(int blockletId);
+ public abstract void onBlockletStart(int blockletId) throws IOException;
/**
* End of blocklet notification
*
* @param blockletId sequence number of blocklet in the block
*/
- public abstract void onBlockletEnd(int blockletId);
+ public abstract void onBlockletEnd(int blockletId) throws IOException;
/**
- * Add the column pages row to the datamap, order of pages is same as
`indexColumns` in
+ * Add row data to the datamap, order of field is same as `indexColumns`
in
* DataMapMeta returned in DataMapFactory.
- * Implementation should copy the content of `pages` as needed, because
`pages` memory
- * may be freed after this method returns, if using unsafe column page.
+ * Implementation should copy the content of `row` as needed, because
its memory
+ * may be freed after this method returns, in case of unsafe memory
*/
- public abstract void onPageAdded(int blockletId, int pageId,
ColumnPage[] pages)
+ public abstract void addRow(int blockletId, int pageId, int rowId,
CarbonRow row)
--- End diff --
Because currently all datamap implementation (lucene, bloom filter, minmax)
are adding index row by row, even if the framework gives ColumnPage data to the
datamap implementation. I think it is not coincidence, so I changed the
interface to give row data.
I will change to `onRowAdded`
---
