jamesyfshao commented on pull request #5394: URL: https://github.com/apache/incubator-pinot/pull/5394#issuecomment-637721377
> Hey James, the last we discussed, I thought the only callback needed was after the realtime row was indexed. > > Can you add to your design doc the following information > > 1. At what points do you need callbacks from Pinot > 2. What does the callback do? > > For example: > Callbacks Needed: > > 1. onRowIndexed(): This callback needs to happen each time a row ingested from a realtime stream-partition is successfully indexed. upon receiving this callback, the pinot-upsert pushes an event into Kafka (see xxx of design doc). Arguments to this callback should be: > a. Offset of the row > b. Table name > c. Segment name > d. : whatever... > 2. onSegmentLoad(): This callback needs to happen each time an offline segment is loaded successfully. ... etc. > > I know this is probably spread across your design doc, but to have it concisely in one place will help us review. > > thanks a lot, and sorry for the delay in review. I had a hard deadline of May 31 on an internal project that I have been working on. thanks for reviewing, I have added a session [here](https://docs.google.com/document/d/1SFFir7ByxCff-aVYxQeTHpNhPXeP5q7P4g_6O2iNGgU/edit#heading=h.5g392vjhtywt). Also all the current callback are contained in this [package](https://github.com/jamesyfshao/pinot/tree/ecbef927507edd4239682aab3b0ef0f65a351a78/pinot-core/src/main/java/org/apache/pinot/core/data/manager/callback). If you have time to apply the patch to our local pinot repo it will be more straight forward and the method itself is also documnted ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
