Seems fair. File a JIRA and we can look into it
- Bobby On Wednesday, January 18, 2017, 10:21:36 AM CST, Jungtaek Lim <[email protected]> wrote:Personally I met many situations in production which force me start reprocessing from specific offset (actually data source was not Kafka but log-like structured MySQL table). Reprocessing is a common case which users occasionally meet the needs, and if possible it would be better for storm-kafka and storm-kafka-client to support this with various arguments (offset, date/time with newer Kafka version). - Jungtaek Lim (HeartSaVioR) 2017년 1월 19일 (목) 오전 1:18, Bobby Evans <[email protected]>님이 작성: > In newer versions of Kafka they support seeking to a specific date/time. > That would probably be a more useful one to look into supporting. > > > - Bobby > > On Wednesday, January 18, 2017, 10:00:50 AM CST, Hugo Da Cruz Louro < > [email protected]> wrote:Hi Sachin, > > The 2nd case can likely handled with the committed offset, which is > covered by UNCOMMITTED_EARLIEST or UNCOMMITTED_LATEST. > > The 1st case it may make sense but even if you give the start offset, > since Kafka polls a certain number of bytes, and not specifically a number > of records, it may not be trivial to guarantee that the same exact dataset > is polled each time. > > However, If we as a community agree that it is useful to support your > proposed feature, I have no particular argument to do so. > > Best, > Hugo > PS. We usually have the practice to initiate discussion threads with email > subject prefixed with [DISCUSS] > > > > > On Jan 18, 2017, at 6:33 AM, Sachin Pasalkar < > [email protected]> wrote: > > > > Hi, > > > > I was looking at code of current KafkaTridentSpoutEmitter & KafkaSpout > class. Can we add functionality based on user provided offset to start from > particular offset? This would be useful incase user wants to reprocess > particular data set. Another example user has changed the group id & aware > where old offset committed & he wants to start processing from same > position. > > > > Does this make sense? OR its explicit that it will not be supported? > > > > Thanks, > > Sachin >
