fx19880617 commented on issue #6539:
URL:
https://github.com/apache/incubator-pinot/issues/6539#issuecomment-772961312
> Thanks @fx19880617 for the suggestions. I tried Ideal state and can see
all the 4 partitions are getting consumed as shown below which is good. However
there is an issue when you shrink Kafka partition i.e delete and recreate a
topic with less number of partition than before. When that happens Pinot server
keeps on ERROR-ing. Even though everything works the error is cluttering the
console also not sure whether this have any other performance impact.
>
> ```
> "mylog__0__314__20210203T1856Z" : {
>
"Server_my-pinot-server-2.my-pinot-server-headless.logging.svc.cluster.local_8098"
: "CONSUMING"
> },
> "mylog__1__7__20210204T0043Z" : {
>
"Server_my-pinot-server-0.my-pinot-server-headless.logging.svc.cluster.local_8098"
: "CONSUMING"
> },
> "mylog__2__7__20210204T0043Z" : {
>
"Server_my-pinot-server-1.my-pinot-server-headless.logging.svc.cluster.local_8098"
: "CONSUMING"
> },
> "mylog__3__0__20210203T1927Z" : {
>
"Server_my-pinot-server-2.my-pinot-server-headless.logging.svc.cluster.local_8098"
: "CONSUMING"
> }
> ```
By design, Pinot is not handling the shrinking case. I think you need to
manually delete the consuming segment `mylog__3__0__20210203T1927Z`. Also
delete topic and recreate may mess up the consumer offset, @npawar may have
more saying about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
