lxy-9602 commented on code in PR #583: URL: https://github.com/apache/paimon-cpp/pull/583#discussion_r4651659647
########## docs/source/user_guide/realtime.rst: ########## @@ -437,6 +437,33 @@ filtering. Keep predicate fields in the read schema and enable In-memory and disk indexes are configured independently. Use ``file-index.<type>.columns`` to enable indexes for committed files. +Real-time Vector Search +~~~~~~~~~~~~~~~~~~~~~~ + +Build with ``-DPAIMON_ENABLE_LUMINA=ON`` for ``LIST<FLOAT>`` vector indexes +using ``bruteforce`` or ``flatnsw`` with ``rawf32`` encoding: + +.. code-block:: text + + realtime.index.lumina.columns = embedding + realtime.index.lumina.embedding.index.type = bruteforce + realtime.index.lumina.embedding.index.dimension = 2 + realtime.index.lumina.embedding.distance.metric = l2 + realtime.index.lumina.embedding.encoding.type = rawf32 + +For disk indexes, repeat these settings with the ``file-index.lumina`` prefix, Review Comment: Here what I mean is that Lumina’s real-time and disk side need to be configured separately. For example, `realtime.index.lumina.columns = xxx` only controls the real-time index. If you also need it for disk, you must additionally configure `file-index.lumina.columns = xxx`. (In general, if real-time vector queries are needed, these two are configured together; otherwise, the disk-side vector search will error out due to missing indexes.) The specific values depend on the desired real-time vs. disk configuration. Some settings—like dimension—should be the same, while other parameters should be chosen separately based on the algorithm. If you think this is ambiguous, I can adjust the wording. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
