siddharthteotia commented on a change in pull request #4790: Support ORDER BY
for DISTINCT queries
URL: https://github.com/apache/incubator-pinot/pull/4790#discussion_r347642025
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/DistinctTable.java
##########
@@ -197,28 +132,70 @@ private void serializeColumns(final Object[] columns,
final DataSchema.ColumnDat
}
}
- public int size() {
- return _table.size();
- }
-
- public Iterator<Key> getIterator() {
- return _table.iterator();
- }
-
- public void setColumnNames(String[] columnNames) {
- _columnNames = columnNames;
- }
+ /**
+ * DESERIALIZE: Broker side
+ * @param byteBuffer data to deserialize
+ * @throws IOException
+ */
+ public DistinctTable(ByteBuffer byteBuffer) throws IOException {
+ // This is called by the BrokerReduceService when it de-serializes the
+ // DISTINCT result from the DataTable. As of now we don't have all the
+ // information to pass to super class so just pass null, empty lists
+ // and the broker will set the correct information before merging the
+ // data tables.
+ super(null, new ArrayList<>(), new ArrayList<>(), 0);
+ DataTable dataTable = DataTableFactory.getDataTable(byteBuffer);
+ _dataSchema = dataTable.getDataSchema();
- public void setColumnTypes(FieldSpec.DataType[] columnTypes) {
- _columnTypes = columnTypes;
- }
+ int numRows = dataTable.getNumberOfRows();
+ int numColumns = _dataSchema.size();
- public String[] getColumnNames() {
- return _columnNames;
+ // extract rows from the datatable
+ for (int rowIndex = 0; rowIndex < numRows; rowIndex++) {
+ Object[] columnValues = new Object[numColumns];
+ for (int colIndex = 0; colIndex < numColumns; colIndex++) {
+ DataSchema.ColumnDataType columnDataType =
_dataSchema.getColumnDataType(colIndex);
+ switch (columnDataType) {
+ case INT:
+ columnValues[colIndex] = dataTable.getInt(rowIndex, colIndex);
+ break;
+ case LONG:
+ columnValues[colIndex] = dataTable.getLong(rowIndex, colIndex);
+ break;
+ case FLOAT:
+ columnValues[colIndex] = dataTable.getFloat(rowIndex, colIndex);
+ break;
+ case DOUBLE:
+ columnValues[colIndex] = dataTable.getDouble(rowIndex, colIndex);
+ break;
+ case STRING:
+ columnValues[colIndex] = dataTable.getString(rowIndex, colIndex);
+ break;
+ case BYTES:
+ columnValues[colIndex] = dataTable.getString(rowIndex, colIndex);
+ default:
+ throw new IllegalStateException(
+ "Unexpected column data type " + columnDataType + " while
deserializing data table for DISTINCT query");
+ }
+ }
+ Key key = new Key(columnValues);
+ Record record = new Record(key, null);
+ _lookupMap.put(record.getKey(), record);
Review comment:
Not needed based on the recent changes after extending BaseTable
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
