xiangfu0 opened a new pull request, #20169: URL: https://github.com/apache/pinot/pull/20169
## Summary A principal restricted by row-level security could use `IN_PARTITIONED_SUBQUERY` to test, and with `GROUP BY` enumerate, which values exist in the rows its row filters hide, while the response still reported `rlsFiltersApplied: true`. The broker ANDs the principal's row filters into the WHERE clause and stamps them in the query options, but the subquery of `IN_PARTITIONED_SUBQUERY` reaches the server as a string literal that carries neither. `ServerQueryExecutorV1Impl` compiled that literal and ran it on the table's segments as is, so the `ID_SET` it built covered every row. `IN_SUBQUERY` is not affected: the broker runs it as a nested request under the same identity. The multi-stage engine is not affected: it does not register the function and applies row filters on the server per leaf scan. ## Changes - **Server** (`ServerQueryExecutorV1Impl`): the literal is compiled to a `PinotQuery`, its data source renamed to the queried table (the subquery runs on that table's segments whatever its FROM clause names), the table's row filter added to the options the subquery was written with, and `RlsFiltersRewriter` run on it before it becomes a `QueryContext`. A subquery nested in the subquery is filtered the same way through the recursion. A query that carries row filters but none for the queried table fails instead of running the subquery unfiltered. - **Broker** (`BaseSingleStageBrokerRequestHandler`): when row filters apply to the table, the query is rejected with `403` before the filters are added. A broker cannot tell a server of this release from an older one, so it fails closed rather than forward a subquery an older server would run unfiltered. The new broker config `pinot.broker.rls.reject.in.partitioned.subquery` (default `true`) turns the rejection off once every server runs this release. Principals without row filters keep the function. - **Shared matcher** (`RequestUtils.isInPartitionedSubquery`): broker and server match the function name the same way (underscores removed, case folded per character), so a spelling the server resolves cannot slip past the broker. ## Upgrade matrix | Broker | Server | Behavior for a principal with row filters | |---|---|---| | this release | older | rejected with 403 | | older | this release | subquery filtered, through the `rlsFilters-<table>` option older brokers already send | | this release | this release | rejected by default; `pinot.broker.rls.reject.in.partitioned.subquery=false` lets the server filter it instead | No wire, serialization, or segment format change. The default of `pinot.broker.enable.row.column.level.auth` (`false`) is unchanged, so clusters without row/column-level auth see no difference. ## Release notes - With `pinot.broker.enable.row.column.level.auth=true`, a principal whose access control returns row filters for a table now gets `403 Permission denied` for a single-stage query that calls `IN_PARTITIONED_SUBQUERY` on that table. New broker config `pinot.broker.rls.reject.in.partitioned.subquery` (default `true`): set it to `false` once every server runs this release to let such principals use the function, with the server applying the row filters to the subquery. ## Testing - `QueryExecutorTest#testInPartitionedSubqueryAppliesRowFilters`: with the row filter stamped, a subquery that selects the hidden rows yields an empty `ID_SET` (count 0), also when its FROM clause names another table and when it is nested in another subquery; without row filters the function behaves as before. Fails on the code before this change (returned the full count). - `BaseSingleStageBrokerRequestHandlerTest`: `containsInPartitionedSubquery` under every spelling, including the dotless-i look-alike, in WHERE, CASE and HAVING; the rejection returns 403 before any server request; without row filters the query is forwarded unchanged; with the config off the query is forwarded with the row filters applied and the function left for the servers. Fails on the code before this change. - `RequestUtilsTest#testIsInPartitionedSubquery`. - `RowLevelSecurityIntegrationTest#testInPartitionedSubqueryRejectedForPrincipalWithRowFilters`: end to end, admin keeps the function, a user with row filters is rejected. ## Documentation Follow-up in pinot-docs: the row-level security page and the `IN_PARTITIONED_SUBQUERY` function entry should mention the rejection and the new config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
