StephanEwen commented on pull request #15397:
URL: https://github.com/apache/flink/pull/15397#issuecomment-810420862
Thanks for the answer.
# Removing Split Assignment Tracker from State
Sounds like there is consensus.
@becketqin There is already a method `readFully(byte[])` - are you looking
for something in addition?
We could also decide to not switch to `DataInputDeserializer`, because
`ByteArrayInputStream` and `DataInputStream` together give us `available()` as
well (which isn't guaranteed to work, it is best-effort, but we also use it
only for sanity check, we don't rely on it for correctness).
# Removing Kafka Assignment Tracking
Ok, let's decouple that refactoring from this PR.
I see the point raised by @PatrickRen and think we have either of the
following options:
1. Leave it as is (which I believe is fragile)
2. Change it such that only the main coordinator thread updates
`assignedPartitions` and `pendingSplits`. In that case, the worker thread would
forward all discovered partitions to the enumerator thread, which would filter.
(I think this is quite feasible)
3. Copy the set before passing it to the worker thread (seems like
overkill and might be inefficient).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
