[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

2018-05-17 Thread makeyang
Github user makeyang commented on the issue:

https://github.com/apache/flink/pull/5908
  
@StefanRRichter  should jira be close too?


---


[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

2018-05-17 Thread StefanRRichter
Github user StefanRRichter commented on the issue:

https://github.com/apache/flink/pull/5908
  
@makeyang can you also please close this PR?


---


[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

2018-05-10 Thread makeyang
Github user makeyang commented on the issue:

https://github.com/apache/flink/pull/5908
  
@StefanRRichter I definitely like u ideas about flat copy priority queue in 
sync and then handler key-group thing in async. so I'll get rid of my solution 
and try to implement u solution and then give it a new pr


---


[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

2018-05-10 Thread makeyang
Github user makeyang commented on the issue:

https://github.com/apache/flink/pull/5908
  
@StefanRRichter sorry for the late answer. just take a surgery few days ago 
and come back now.
I'll take close look at you comments and then answer back.


---


[GitHub] flink issue #5908: [FLINK-9182]async checkpoints for timer service

2018-05-03 Thread StefanRRichter
Github user StefanRRichter commented on the issue:

https://github.com/apache/flink/pull/5908
  
Maybe let me add some more. First, about introducing a separate new state 
handle. Our long term plan is actually to integrate timers more closely with 
the backends, so that we can also have a timer service in RocksDB. In general, 
I would target for reducing the handles(e.g. timers could then be considered 
also as managed keyed stated) instead of adding more. Second, I would probably 
suggest a simpler model for the async snapshots. You dropped the idea of making 
flat copies, but I wonder if this was premature. I can see that maybe calling 
`set.toArray(...)` per keygroup could (maybe) turn out a bit slow because it 
has to potentially iterate and flatten linked entries. However, with async 
snapshots, we could get rid of the key-group partitioning of sets, and instead 
do a flat copy of the internal array of the priority queue. This would 
translate to just a single memcopy call internally, which is very efficient. In 
the async part, we can still partition the timers by key-group in a 
 similar way as the copy-on-write state table does. This would avoid slowing 
down the event processing path (in fact improving it be unifying the sets) and 
also keep the approach very straight forward and less invasive.


---