Jackie-Jiang commented on code in PR #11453:
URL: https://github.com/apache/pinot/pull/11453#discussion_r1309355859
##########
pinot-core/src/main/java/org/apache/pinot/core/common/datablock/DataBlockBuilder.java:
##########
@@ -107,144 +104,83 @@ public void setNullRowIds(@Nullable RoaringBitmap
nullRowIds)
public static RowDataBlock buildFromRows(List<Object[]> rows, DataSchema
dataSchema)
throws IOException {
- DataBlockBuilder rowBuilder = new DataBlockBuilder(dataSchema,
DataBlock.Type.ROW, rows.size());
+ int numRows = rows.size();
+ DataBlockBuilder rowBuilder = new DataBlockBuilder(dataSchema,
DataBlock.Type.ROW, numRows);
// TODO: consolidate these null utils into data table utils.
// Selection / Agg / Distinct all have similar code.
- int numColumns = rowBuilder._numColumns;
+ ColumnDataType[] storedTypes = dataSchema.getStoredColumnDataTypes();
+ int numColumns = storedTypes.length;
RoaringBitmap[] nullBitmaps = new RoaringBitmap[numColumns];
- DataSchema.ColumnDataType[] columnDataTypes =
dataSchema.getColumnDataTypes();
- DataSchema.ColumnDataType[] storedColumnDataTypes =
dataSchema.getStoredColumnDataTypes();
Object[] nullPlaceholders = new Object[numColumns];
for (int colId = 0; colId < numColumns; colId++) {
nullBitmaps[colId] = new RoaringBitmap();
- nullPlaceholders[colId] =
columnDataTypes[colId].convert(storedColumnDataTypes[colId].getNullPlaceholder());
+ nullPlaceholders[colId] = storedTypes[colId].getNullPlaceholder();
}
- rowBuilder._numRows = rows.size();
ByteBuffer byteBuffer = ByteBuffer.allocate(rowBuilder._rowSizeInBytes);
- for (int rowId = 0; rowId < rows.size(); rowId++) {
+ for (int rowId = 0; rowId < numRows; rowId++) {
byteBuffer.clear();
Object[] row = rows.get(rowId);
- for (int colId = 0; colId < rowBuilder._numColumns; colId++) {
+ for (int colId = 0; colId < numColumns; colId++) {
Object value = row[colId];
if (value == null) {
nullBitmaps[colId].add(rowId);
value = nullPlaceholders[colId];
}
- switch (rowBuilder._columnDataTypes[colId]) {
+ switch (storedTypes[colId]) {
// Single-value column
case INT:
- byteBuffer.putInt(((Number) value).intValue());
+ byteBuffer.putInt((int) value);
Review Comment:
The main intention is not performance, but ensuring the wiring is correct
and we can catch wrong wiring easily. Without this PR, a lot of wiring is
wrong, but we are not able to catch them.
Added some comment explaining this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
