Akash3121 commented on code in PR #11144:
URL: https://github.com/apache/paimon/pull/11144#discussion_r4570407006


##########
docs/docs/pypaimon/writing.md:
##########
@@ -90,9 +90,15 @@ the same method. Changing a builder affects only operations 
created afterward.
 
 ### Native Write and Commit
 
-With the `pypaimon-rust` extension installed, set `write.native.enabled=true`
-to use Rust for Parquet/BLOB writes and `commit.native.enabled=true` to use
-Rust for REST snapshot publication. Batch and stream writers retain the same
+On Python 3.10+, `pypaimon-rust>=0.5.0` is installed by default.
+`write.native.enabled` and `commit.native.enabled` both default to `true`,
+selecting Rust for supported Parquet/BLOB writes and REST snapshot publication.
+Set either option to `false` to select its Python execution path.

Review Comment:
   With `write.native.enabled`, `read.native.enabled`, and 
`scan.native-plan.enabled` all defaulting to `True` now, setting 
`write.native.enabled=false` alone does not select the Python execution path in 
`create_row_id_plan_execution()` 
(`paimon-python/pypaimon/write/backend_dispatch.py:351-360`):
    
    ```python
    def create_row_id_plan_execution(table, plan):
        """Select pure routing once; execution errors never replay another 
backend."""
        if (table.options.native_write_enabled() or 
table.options.native_read_enabled()
                or table.options.native_plan_enabled()):
            from pypaimon.execution.native.write.row_id_plan import 
create_native_row_id_plan, NativeRowIdPlan
            native = create_native_row_id_plan(table, plan)
            if native is not None:
                return NativeRowIdPlan(table, plan, native=native)
        from pypaimon.execution.python.write.row_id_plan import PythonRowIdPlan
        return PythonRowIdPlan(plan)
    ```
    
    Previously, when all three options defaulted to `False`, 
`native_write_enabled() or native_read_enabled() or native_plan_enabled()` 
worked as an opt-in check. Now that all three default to `True`, setting 
`write.native.enabled=false` leaves `native_read_enabled()` and 
`native_plan_enabled()` as `True`, so `create_row_id_plan_execution()` still 
routes row-ID updates (`update_by_row_id`) to `NativeRowIdPlan` unless the user 
disables all three options.
    
    Should `create_row_id_plan_execution()` check `if 
table.options.native_write_enabled():` (or respect an explicit 
`write.native.enabled=false`) so that setting `write.native.enabled=false` 
selects `PythonRowIdPlan` as documented here?



##########
paimon-python/pypaimon/common/options/core_options.py:
##########
@@ -723,7 +723,7 @@ class CoreOptions:
     SCAN_NATIVE_PLAN_ENABLED: ConfigOption[bool] = (
         ConfigOptions.key("scan.native-plan.enabled")

Review Comment:
   Flipping both `SCAN_NATIVE_PLAN_ENABLED` and `READ_NATIVE_ENABLED` defaults 
to `True` here breaks single-option opt-out (`scan.native-plan.enabled=false`) 
in `native_plan_requested()` 
(`paimon-python/pypaimon/read/read_dispatch.py:408-415`):
    
    ```python
    def native_plan_requested(scan) -> bool:
        """Resolve Native planning preference without loading a disabled 
backend."""
        if scan.table.options.native_plan_enabled():
            return True
        if not scan.table.options.native_read_enabled():
            return False
        from pypaimon.execution.native.runtime import native_runtime_available
        return native_runtime_available()
    ```
    
    Previously, when both options defaulted to `False`, falling through to 
`scan.table.options.native_read_enabled()` allowed `read.native.enabled=true` 
to also opt into native split planning. Now that `READ_NATIVE_ENABLED` defaults 
to `True`, if a user explicitly sets `scan.native-plan.enabled=false` (without 
also setting `read.native.enabled=false`), 
`scan.table.options.native_plan_enabled()` returns `False`, but 
`scan.table.options.native_read_enabled()` still returns `True` from its new 
default—so `native_plan_requested(scan)` still returns `True` and ignores 
`scan.native-plan.enabled=false` (which is also why the tests in this PR had to 
add `'read.native.enabled': 'false'` alongside `'scan.native-plan.enabled': 
'false'`).
    
    Similarly, `configure_indexed_read(table)` in 
`paimon-python/pypaimon/read/read_dispatch.py:250-259` still checks `if not 
any(options.contains(option) for option in (CoreOptions.READ_NATIVE_ENABLED, 
CoreOptions.SCAN_NATIVE_PLAN_ENABLED)):` and copies 
`{CoreOptions.READ_NATIVE_ENABLED.key(): 'true'}` into `table.options.options`, 
which overrides the `PYPAIMON_TEST_EXECUTION=python` / 
`@pytest.mark.python_read` test-lane default in `conftest.py:246-252` whenever 
`pypaimon_rust` is installed.
    
    If `SCAN_NATIVE_PLAN_ENABLED` is explicitly configured in 
`scan.table.options`, `native_plan_requested(scan)` should respect it directly:
    
    ```python
    def native_plan_requested(scan) -> bool:
        """Resolve Native planning preference without loading a disabled 
backend."""
        if 
scan.table.options.options.contains(CoreOptions.SCAN_NATIVE_PLAN_ENABLED):
            return scan.table.options.native_plan_enabled()
        if not (scan.table.options.native_plan_enabled() or 
scan.table.options.native_read_enabled()):
            return False
        from pypaimon.execution.native.runtime import native_runtime_available
        return native_runtime_available()
    ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to