amoeba commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928652618
I ran the tests in ./r/tests/testthat/test-python-flight.R with reticulate
1.34.0 and 1.35.0 and get _different_ failures so I think this needs a deeper
investigation.
<details>
<summary>reticulate 1.34.0</summary>
```r
==> Testing R file using 'testthat'
ℹ Loading arrow
[ FAIL 6 | WARN 0 | SKIP 1 | PASS 1 ]
── Error (test-python-flight.R:28:5): flight_path_exists
───────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_iterate(it, f)`: pyarrow.lib.ArrowInvalid: invalid literal for
int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_false(flight_obj %in% list_flights(client)) at
test-python-flight.R:28:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. ├─flight_obj %in% list_flights(client)
5. └─arrow::list_flights(client)
6. └─reticulate::iterate(generator, function(x)
as.character(x$descriptor$path[[1]])) at r/R/flight.R:121:3
7. └─reticulate:::py_iterate(it, f)
── Error (test-python-flight.R:33:5): flight_put
───────────────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail:
Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_true(flight_path_exists(client, flight_obj)) at
test-python-flight.R:33:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. └─arrow::flight_path_exists(client, flight_obj)
5. ├─base::tryCatch(...) at r/R/flight.R:128:3
6. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
7. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
8. │ └─base (local) doTryCatch(return(expr), name, parentenv,
handler)
9. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:130:7
10. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:43:5): flight_put with max_chunksize
────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail:
Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_true(flight_path_exists(client, flight_obj)) at
test-python-flight.R:43:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. └─arrow::flight_path_exists(client, flight_obj)
5. ├─base::tryCatch(...) at r/R/flight.R:128:3
6. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
7. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
8. │ └─base (local) doTryCatch(return(expr), name, parentenv,
handler)
9. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:130:7
10. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:56:5): flight_get
───────────────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail:
Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj), example_data)
at test-python-flight.R:56:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at
tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at
tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:62:5): flight_put with RecordBatch
──────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail:
Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj2), example_data)
at test-python-flight.R:62:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at
tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at
tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj2)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:66:5): flight_put with overwrite = FALSE
────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail:
Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_error(...) at test-python-flight.R:66:5
2. │ └─testthat:::expect_condition_matching(...)
3. │ └─testthat:::quasi_capture(...)
4. │ ├─testthat (local) .capture(...)
5. │ │ └─base::withCallingHandlers(...)
6. │ └─rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
7. └─arrow::flight_put(...)
8. └─arrow::flight_path_exists(client, path) at r/R/flight.R:66:3
9. ├─base::tryCatch(...) at r/R/flight.R:128:3
10. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
11. │ └─base (local) tryCatchOne(expr, names, parentenv,
handlers[[1L]])
12. │ └─base (local) doTryCatch(return(expr), name, parentenv,
handler)
13. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:130:7
14. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Skipped tests (1)
───────────────────────────────────────────────────────────────────────────────────────────────
• empty test (1): test-python-flight.R:75:3
[ FAIL 6 | WARN 0 | SKIP 1 | PASS 1 ]
Test complete
```
</details>
<details>
<summary>reticulate 1.34.0</summary>
```r
==> Testing R file using 'testthat'
ℹ Loading arrow
[ FAIL 5 | WARN 0 | SKIP 1 | PASS 2 ]
── Error (test-python-flight.R:32:5): flight_put
───────────────────────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj) at
test-python-flight.R:32:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.Table(data)
4. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE) at
r/R/python.R:111:3
5. └─arrow:::r_to_py.RecordBatchReader(...)
6.
└─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
at r/R/python.R:227:3
7. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:42:5): flight_put with max_chunksize
────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj,
max_chunksize = 1) at test-python-flight.R:42:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.Table(data)
4. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE) at
r/R/python.R:111:3
5. └─arrow:::r_to_py.RecordBatchReader(...)
6.
└─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
at r/R/python.R:227:3
7. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:56:5): flight_get
───────────────────────────────
<pyarrow.lib.ArrowKeyError/python.builtin.KeyError/python.builtin.LookupError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
pyarrow.lib.ArrowKeyError: 'Flight not found.'. Detail: Python exception:
KeyError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj), example_data)
at test-python-flight.R:56:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at
tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at
tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at
r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:61:5): flight_put with RecordBatch
──────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, RecordBatch$create(example_data), path =
flight_obj2) at test-python-flight.R:61:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.RecordBatch(data)
4. └─pa$RecordBatch$`_import_from_c`(...) at r/R/python.R:84:3
5. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Error (test-python-flight.R:66:5): flight_put with overwrite = FALSE
────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`:
ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_error(...) at test-python-flight.R:66:5
2. │ └─testthat:::expect_condition_matching(...)
3. │ └─testthat:::quasi_capture(...)
4. │ ├─testthat (local) .capture(...)
5. │ │ └─base::withCallingHandlers(...)
6. │ └─rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
7. └─arrow::flight_put(...)
8. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
9. └─arrow:::r_to_py.Table(data)
10. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE)
at r/R/python.R:111:3
11. └─arrow:::r_to_py.RecordBatchReader(...)
12.
└─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
at r/R/python.R:227:3
13. └─reticulate:::py_call_impl(callable, call_args$unnamed,
call_args$named)
── Skipped tests (1)
───────────────────────────────────────────────────────────────────────────────────────────────
• empty test (1): test-python-flight.R:75:3
[ FAIL 5 | WARN 0 | SKIP 1 | PASS 2 ]
Test complete
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
