Manis99803 opened a new issue #8068:
URL: https://github.com/apache/pinot/issues/8068
Issue:
Currently, with a Fixed type in the Pinot SegmentNameGenerator, we are able
to generate a segment with an user-defined name. However, there is some issue
with respect to this when we want to generate more than one segment.
For Example:
Let's say we have 5 different CSVs and we are trying to push these CSVs to
the Pinot Offline table. The ingestion job has the following configuration:
```
segmentNameGeneratorSpec:
configs:
segment.name: SEGEMENT
type: fixed
```
So the expectation was that, if we try to push 5 CSVs to the Pinot table we
would have 5 segments with the following names:
SEGMENT_0
SEGMENT_1
SEGMENT_2
SEGMENT_3
SEGMENT_4
However, we see only one segment getting created with the name "SEGMENT".
And the data this segment would have will be the data present in the last CSV
in our case that becomes (5th one).
On checking the [code
link](https://github.com/apache/pinot/blob/3f93cfb93a386ee3e8bd108ffe34814b734575e9/pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/name/FixedSegmentNameGenerator.java#L36),
it looks like in case of Fixed naming scheme sequenceId is not getting
appended to the segment name due to which every time same segment name is
getting generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
