Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-12-24 Thread Yun Gao
Hi all, I tested the previous PoC with the current tests and I found some new issues that might cause divergence, and sorry for there might also be some reversal for some previous problems: 1. Which operators should wait for one more checkpoint before close ? One

[jira] [Created] (FLINK-20767) add nested field support for SupportsFilterPushDown

2020-12-24 Thread Jun Zhang (Jira)
Jun Zhang created FLINK-20767: - Summary: add nested field support for SupportsFilterPushDown Key: FLINK-20767 URL: https://issues.apache.org/jira/browse/FLINK-20767 Project: Flink Issue Type:

[jira] [Created] (FLINK-20766) Separate the implementation of stream sort nodes

2020-12-24 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-20766: --- Summary: Separate the implementation of stream sort nodes Key: FLINK-20766 URL: https://issues.apache.org/jira/browse/FLINK-20766 Project: Flink Issue Type:

[jira] [Created] (FLINK-20765) ScalarOperatorGens doesn't set proper nullability for result type of generated expressions

2020-12-24 Thread Rui Li (Jira)
Rui Li created FLINK-20765: -- Summary: ScalarOperatorGens doesn't set proper nullability for result type of generated expressions Key: FLINK-20765 URL: https://issues.apache.org/jira/browse/FLINK-20765

Re: Insufficient number of network buffers after restarting

2020-12-24 Thread Yangze Guo
Hi, Yufei. Can you reproduce this issue in 1.10.0? The deterministic slot sharing introduced in 1.12.0 is one possible reason. Before 1.12.0, the distribution of tasks in slots is not determined. Even if the network buffers are enough from the perspective of the cluster. Bad distribution of tasks

Insufficient number of network buffers after restarting

2020-12-24 Thread Yufei Liu
Hey, I’ve found that job will throw “java.io.IOException: Insufficient number of network buffers: required 51, but only 1 available” after job retstart, and I’ve observed TM use much more network buffers than before. My internal branch is under 1.10.0 can easily reproduce, but I use 1.12.0

[jira] [Created] (FLINK-20763) canal format parse update record with null value get wrong result

2020-12-24 Thread WangRuoQi (Jira)
WangRuoQi created FLINK-20763: - Summary: canal format parse update record with null value get wrong result Key: FLINK-20763 URL: https://issues.apache.org/jira/browse/FLINK-20763 Project: Flink

[jira] [Created] (FLINK-20762) Remove unused checkpointStorage path after job finished

2020-12-24 Thread Liu (Jira)
Liu created FLINK-20762: --- Summary: Remove unused checkpointStorage path after job finished Key: FLINK-20762 URL: https://issues.apache.org/jira/browse/FLINK-20762 Project: Flink Issue Type:

[jira] [Created] (FLINK-20761) Cannot read hive table/partition whose location path contains comma

2020-12-24 Thread Rui Li (Jira)
Rui Li created FLINK-20761: -- Summary: Cannot read hive table/partition whose location path contains comma Key: FLINK-20761 URL: https://issues.apache.org/jira/browse/FLINK-20761 Project: Flink