#general
@pankaj: Hey guys, would there be a support for something like a tile in pinot apart from segmenting. I was looking at amazon timestream and it seems this will be a useful feature to have. In general is there a comparison of pinot against timestream.
@g.kishore: I dont see any difference between pinot segment and tile,
@g.kishore: we have partitioning on time as well as any other dimension
@pankaj: Can I do it on both? I guess my impression from the article I posted is that it allows that
@g.kishore: yes
@g.kishore: this was supported in pinot since a long time
@pankaj: say I have lots of data coming; today I have to control kafka partitions and I am limited by the number of realtime servers I have put on how fast I can process and serve the data and this is limited now
@g.kishore: thats a completely different problem right.. max number of consumers for kafka cannot go beyond the number of partitions
@g.kishore: violating that will result in out of order processing
@g.kishore: what you are looking for is supported in kinesis
@pankaj: yeah I am trying to figure out if tile is a giving a higher abstraction or not over segment
@g.kishore: and Pinot's Kinesis connector will probably be the right thing for your case
@g.kishore: because kinesis increases and decreases the number of partitions based on load
@g.kishore: and Pinot adapts accordingly
@pankaj: so pinot handles the assignments across realtime servers for these partitions? and splitting it into number of segments?
@pankaj: is there some doc for this?
@pankaj: or you guys are still working on this one?
@g.kishore: yes.. this is already there
@g.kishore:
@pankaj: cool! will look this up. Thanks.
@g.kishore: this is another one -
@pankaj: Thanks for the links @g.kishore; @npawar awesome job on the segment lifecycle videos!
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org
