mcvsubbu commented on issue #151: Feature request: realtime table bouncing URL: https://github.com/apache/incubator-pinot/issues/151#issuecomment-525504160 There has been no movement on this so far. Contributions welcome. A few ways to address this, but we need to take care of the corner case where the schema is updated exactly at the time the segments are nearing or at completion. Some partitions may have completed and started a new segment (but no data yet, so we cannot commit them -- we don't allow 0-row segments). One way I can think of is to provide an API that atomically changes all CONSUMING partitions to OFFLINE, and another controller API to just run the realtime segment validation manager, perhaps on the specific table (these things get harder with controller separation). Another way is to send a helix message (RELOAD) to all realtime segments and handle all corner cases in the server (devil is in the details, I have not given it much thought, I admit). The completed segments should also be updated with the default value for the new columns added. So far in production, my observation is the schema evolutions are rare, and applications are ok waiting for a few hours (or even days) in order for the new schema to take effect (before they start querying the new fields added). I will be interested in knowing about use cases (other than testing, because I assume you can always restart in test environment) where the need is instantaneous. I would also like to know what the limits are (1 minute? 1 hour?)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
