mcvsubbu commented on a change in pull request #5320:
URL: https://github.com/apache/incubator-pinot/pull/5320#discussion_r418202359
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/indexsegment/mutable/MutableSegmentImpl.java
##########
@@ -860,6 +876,17 @@ private int getOrCreateDocId(Map<String, Object>
dictIdMap) {
}
}
+ // Date time columns should be dictionary encoded.
+ for (FieldSpec fieldSpec : _physicalDateTimeFieldSpecs) {
+ String dateTimeColumn = fieldSpec.getName();
+ if (noDictionaryColumns.contains(dateTimeColumn)) {
+ _logger.warn("Metrics aggregation cannot be turned ON in presence of
no-dictionary datetime columns, eg: {}",
Review comment:
nit:
```suggestion
_logger.warn("Metrics aggregation cannot be turned on in presence of
no-dictionary datetime columns, eg: {}",
```
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/minion/SegmentPurger.java
##########
@@ -85,7 +85,10 @@ public File purgeSegment()
}
Schema schema = purgeRecordReader.getSchema();
- SegmentGeneratorConfig config = new SegmentGeneratorConfig(schema);
+ // FIXME: figure out how to get table config here.
Review comment:
create an issue for this and add the issue number here?
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/realtime/converter/RealtimeSegmentConverter.java
##########
@@ -35,114 +35,116 @@
import
org.apache.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl;
import org.apache.pinot.spi.config.table.ColumnPartitionConfig;
import org.apache.pinot.spi.config.table.SegmentPartitionConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
import org.apache.pinot.spi.data.FieldSpec;
import org.apache.pinot.spi.data.Schema;
import org.apache.pinot.spi.data.TimeFieldSpec;
import org.apache.pinot.spi.data.TimeGranularitySpec;
public class RealtimeSegmentConverter {
- private MutableSegmentImpl realtimeSegmentImpl;
- private String outputPath;
- private Schema dataSchema;
- private String tableName;
- private String timeColumnName;
- private String segmentName;
- private String sortedColumn;
- private List<String> invertedIndexColumns;
- private List<String> textIndexColumns;
- private List<String> noDictionaryColumns;
- private List<String> varLengthDictionaryColumns;
+ private MutableSegmentImpl _realtimeSegmentImpl;
+ private final String _outputPath;
+ private final Schema _dataSchema;
+ private final String _tableName;
+ private final TableConfig _tableConfig;
+ private final String _segmentName;
+ private final String _sortedColumn;
+ private final List<String> _invertedIndexColumns;
+ private final List<String> _textIndexColumns;
+ private final List<String> _noDictionaryColumns;
+ private final List<String> _varLengthDictionaryColumns;
private final boolean _nullHandlingEnabled;
public RealtimeSegmentConverter(MutableSegmentImpl realtimeSegment, String
outputPath, Schema schema,
- String tableName, String timeColumnName, String segmentName, String
sortedColumn,
+ String tableName, TableConfig tableConfig, String segmentName, String
sortedColumn,
Review comment:
remove the the other unneeded arguments as well (tableName,
sortedColumn, inv index columns, text index columnms, etc.). Another PR is also
fine
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
