JingsongLi commented on PR #67: URL: https://github.com/apache/paimon-vector-index/pull/67#issuecomment-5163710109
1. **The configured memory limit will be ignored**: The resident residual-L2 path in `core/src/ivfpq.rs:1965` unconditionally builds `all_ip_tables` and does not validate `reuse_max_bytes`. Even if the configured value is very small, calling `optimize_for_search()` may still allocate `nq * m * ksub * 4` bytes. 2. **The default behavior results in an 8-fold increase in memory consumption**: `core/src/ivfpq.rs:937` sets the default budget to 512 MiB and applies it to the existing residual-L2 temporary tables in `core/src/ivfpq.rs:2143`; the limit in the old implementation was 64 MiB. This contradicts the statement in the PR description that “Residual L2 paths are unchanged”; existing Auto queries may consume up to an additional 512 MiB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
