Danny Chen created HUDI-1624:
--------------------------------
Summary: The state based index should bootstrap from existing base
files
Key: HUDI-1624
URL: https://issues.apache.org/jira/browse/HUDI-1624
Project: Apache Hudi
Issue Type: Sub-task
Components: Flink Integration
Reporter: Danny Chen
The index should bootstrap from existing base files if there are, in the
design, we load all the keys for one partition if we found that the key does
not exist in the index for {{processElement}}, if there are many records for
this partition, the processing may block and trigger back pressure. When all
the records are loaded, we only need to check the state each time a record is
tagged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
