KKcorps opened a new pull request, #9505:
URL: https://github.com/apache/pinot/pull/9505
Currently, we simply fail the batch ingestion process if segments if
start/end time is outside of time-range or values are not parsed correctly.
This is not needed now since we have a new API from PR #9413
So users should be able to ingest segments and fix their time interval later
on.
This behaviour is disabled by default and can only be switched with
`continueOnError: true` flag for a table.
This is being done in the following way -
* If there is an error in parsing time values during Segment creation for
start/end time, we simply don't store the START/END time in the metadata.
* The ZkMetadata simply returns -1 as start/end time in this case.
* All the places which call `SegmentZKMetadata.getStartTimeMs` or
`SegmentZKMetadata.getEndTimeMs` handle the negative timestamps. Either an
exception is thrown that is already handled (e.g.
TimeBasedTierSegmentSelector.selectSegment` OR we ignore the segments (e.g.
`MergeRollupTask`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
