jackjlli commented on a change in pull request #4462: Time-aware resizing
URL: https://github.com/apache/incubator-pinot/pull/4462#discussion_r307066940
##########
File path:
pinot-hadoop/src/main/java/org/apache/pinot/hadoop/job/mappers/SegmentPreprocessingMapper.java
##########
@@ -61,6 +87,20 @@ public void setup(final Context context) {
@Override
public void map(AvroKey<GenericRecord> record, NullWritable value, final
Context context)
throws IOException, InterruptedException {
+
+ if (_isAppend) {
+ // Normalize time column value and check against sample value
+ String timeColumnValue = (String) record.datum().get(_timeColumn);
+ String normalizedTimeColumnValue =
_normalizedDateSegmentNameGenerator.getNormalizedDate(timeColumnValue);
+
+ if (!normalizedTimeColumnValue.equals(_sampleNormalizedTimeColumnValue))
{
+ // TODO: Create a custom exception and gracefully catch this exception
outside, changing what the path to input
+ // into segment creation should be
+ throw new IllegalArgumentException("Your segment spans multiple time
units. Preprocess is not currently allowed for"
Review comment:
missing a space after `for`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
