Caideyipi commented on PR #11489: URL: https://github.com/apache/iotdb/pull/11489#issuecomment-1803687502
> > I finally discovered that when an event is collected in a connector, it's sequence is eliminated....Maybe the commitId must be assigned before any subtask picking this event, and the commitId may better be something like "static Map<connectorAttributes, commitId>" or "static Map<pendingQueue, commitId>".... > > Maybe we can assign the commitId atomically when the connectorSubtask polls an event. But some events are created inside transfer() when parsing pattern, do they need a commitId? Well, to extreme, after a connector subtask polls an event, it may just sleep until another subtask has commited the subsequent events... Besides, the events created inside transfer do not need commitId, though they(rawTablets) typically do nothing if you commit them, thus arbitrary order is allowed. In a nutshell, I suggest that we assign the Id before the events are put in to the pending queue, like in PipeEventCollector. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
