SteNicholas commented on code in PR #504: URL: https://github.com/apache/paimon-cpp/pull/504#discussion_r4457197797
########## docs/source/user_guide/realtime.rst: ########## @@ -398,6 +403,34 @@ context's reclaimed progress for relevant active stores. It does not restore an old in-memory state and is not unrestricted historical time travel. For a disk-only historical query, do not attach the real-time context. +In-memory Predicate Indexes +-------------------------- + +The default Arrow store supports optional predicate indexes on key or value fields +in append and append deduplicate modes. PK modes are unsupported. For example: + +.. code-block:: text + + realtime.enabled = true + realtime.index.<index_name>.columns = id + realtime.store.stats-mode = full + +Replace ``<index_name>`` with the registered index plugin name. Use a +comma-separated list such as ``id,payload`` to index multiple fields. Supported +field types and predicates depend on the plugin; indexing ``_REALTIME_OFFSET`` +is unsupported. Review Comment: **Document the built-in hash plugin and its supported operations**: This asks users to substitute a registered plugin name, but the guide never identifies the built-in `hash` plugin introduced by this PR or describes its capabilities. Please provide a usable example such as `realtime.index.hash.columns=id,payload`, list equality/IN support and the accepted field types, and explicitly mention the current DECIMAL, TIMESTAMP, and nested-type restrictions. It would also help to state that the hash index returns candidates rather than exact matches: postings are keyed only by `Literal::HashCode()`, so hash collisions can retain nonmatching rows. Residual filtering is needed for exact results even when batch statistics are disabled. The explanation below currently motivates filtering mainly in terms of mixed batches and statistics. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
