Re: [I] Unable to build on MacOSX [arrow]

2024-04-22 Thread via GitHub
matejsp commented on issue #41322: URL: https://github.com/apache/arrow/issues/41322#issuecomment-2068778423 Ouch ... tnx! that is the problem (invalid brew arrow version ... closing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] feat(glib): Add garrow_connection_get_statistics() [arrow-adbc]

2024-04-22 Thread via GitHub
kou commented on PR #1744: URL: https://github.com/apache/arrow-adbc/pull/1744#issuecomment-2068883450 Can we install PostgreSQL to test the GLib bindings? ```diff diff --git a/ci/conda_env_glib.txt b/ci/conda_env_glib.txt index cd937ee22..883ff7f92 100644 ---

Re: [I] Unable to build on MacOSX [arrow]

2024-04-22 Thread via GitHub
kou commented on issue #41322: URL: https://github.com/apache/arrow/issues/41322#issuecomment-2068733950 If you want to build from source, you should remove Apache Arrow C++ installed by Homebrew and build Apache Arrow C++ by yourself. Or you should wait for

Re: [PR] MINOR: [CMake] Remove unused CMAKE_SKIP_INSTALL_ALL_DEPENDENCY [arrow]

2024-04-22 Thread via GitHub
wgtmac commented on PR #41332: URL: https://github.com/apache/arrow/pull/41332#issuecomment-2068985710 I found this is not cleaned up when reading the CMake code. cc @kou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] MINOR: [CMake] Remove unused CMAKE_SKIP_INSTALL_ALL_DEPENDENCY [arrow]

2024-04-22 Thread via GitHub
wgtmac opened a new pull request, #41332: URL: https://github.com/apache/arrow/pull/41332 ### Rationale for this change CMAKE_SKIP_INSTALL_ALL_DEPENDENCY was removed in https://github.com/apache/arrow/pull/75 but it seems that there is still one line remaining. ### What

[PR] GH-41329: [C++][Gandiva] Fix gandiva cache size env var [arrow]

2024-04-22 Thread via GitHub
zanmato1984 opened a new pull request, #41330: URL: https://github.com/apache/arrow/pull/41330 ### Rationale for this change See #41329. ### What changes are included in this PR? 1. Fix checking gandiva cache size env var. 2. Make cache size static so

Re: [PR] GH-41329: [C++][Gandiva] Fix gandiva cache size env var [arrow]

2024-04-22 Thread via GitHub
zanmato1984 commented on PR #41330: URL: https://github.com/apache/arrow/pull/41330#issuecomment-2068650385 cc @pitrou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] GH-41329: [C++][Gandiva] Fix gandiva cache size env var [arrow]

2024-04-22 Thread via GitHub
github-actions[bot] commented on PR #41330: URL: https://github.com/apache/arrow/pull/41330#issuecomment-2068650745 :warning: GitHub issue #41329 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Arrow Flight SQL example JDBC driver incompatibility [arrow-rs]

2024-04-22 Thread via GitHub
Jefffrey commented on code in PR #5666: URL: https://github.com/apache/arrow-rs/pull/5666#discussion_r1574410918 ## arrow-flight/examples/flight_sql_server.rs: ## @@ -184,7 +188,21 @@ impl FlightSqlService for FlightSqlServiceImpl { }; let result = Ok(result);

Re: [PR] GH-41329: [C++][Gandiva] Fix gandiva cache size env var [arrow]

2024-04-22 Thread via GitHub
zanmato1984 commented on code in PR #41330: URL: https://github.com/apache/arrow/pull/41330#discussion_r1574249283 ## cpp/src/gandiva/cache.h: ## @@ -25,6 +25,13 @@ namespace gandiva { +namespace internal { +// Only called once by GetCapacity(). +// Do the actual work of

[PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc opened a new pull request, #1742: URL: https://github.com/apache/arrow-adbc/pull/1742 Hey! Here is the second PR of the Rust implementation containing full FFI bindings. I've decided to manually implement bindings instead of relying on

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
mbrobbel commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574370882 ## rust/core/src/ffi/methods.rs: ## @@ -0,0 +1,93 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] [Python] Broken unit test: Segmentation fault in test_make_write_options_error [arrow]

2024-04-22 Thread via GitHub
llama90 commented on issue #41312: URL: https://github.com/apache/arrow/issues/41312#issuecomment-2068658290 Hello @AlenkaF I followed the suggested documentation and cleaned all builds, then created a Python virtual environment using `python3 -m venv` instead of conda. I confirmed

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
jorisvandenbossche commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2068785659 Hmm, I don't directly see an option for the command line to allow overwriting existing files (I know that for scientific python index at

Re: [PR] MINOR: [C++][Parquet] change page static_pointer_cast to static_cast raw pointer [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on PR #41304: URL: https://github.com/apache/arrow/pull/41304#issuecomment-2068941982 cc @wgtmac @pitrou Would you mind take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Update object_store 0.10.0 [arrow-rs]

2024-04-22 Thread via GitHub
tustvold opened a new pull request, #5675: URL: https://github.com/apache/arrow-rs/pull/5675 # Which issue does this PR close? Closes #. # Rationale for this change # What changes are included in this PR? # Are there any user-facing

Re: [I] [Python] Broken unit test: Segmentation fault in test_make_write_options_error [arrow]

2024-04-22 Thread via GitHub
AlenkaF commented on issue #41312: URL: https://github.com/apache/arrow/issues/41312#issuecomment-2068575753 I tried reproducing on my M1 with Python 3.11 but no luck. The tests is run, not skipped, and it passes: ```python (pyarrow-dev) alenkafrim@Alenkas-MacBook-Pro arrow %

[PR] feat(glib): Add garrow_connection_get_statistics() [arrow-adbc]

2024-04-22 Thread via GitHub
kou opened a new pull request, #1744: URL: https://github.com/apache/arrow-adbc/pull/1744 Fixes #1743. We use PostgreSQL instead of SQLite for testing. Because the SQLite driver doesn't support GetStatistics yet. -- This is an automated message from the Apache Git Service. To

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
jorisvandenbossche commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2068857641 One problem is that we don't have tags on he main branch, so `git describe` based approaches don't work here -- This is an automated message from the Apache Git

Re: [I] [Python] Broken unit test: Segmentation fault in test_make_write_options_error [arrow]

2024-04-22 Thread via GitHub
AlenkaF commented on issue #41312: URL: https://github.com/apache/arrow/issues/41312#issuecomment-2068859092 Thanks for trying it out and confirming! Maybe it would be worth testing with new conda env also. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Move ffi stream and utils from arrow to arrow-array [arrow-rs]

2024-04-22 Thread via GitHub
tustvold commented on code in PR #5670: URL: https://github.com/apache/arrow-rs/pull/5670#discussion_r1574503228 ## arrow-array/src/ffi_stream.rs: ## @@ -63,14 +63,16 @@ use std::{ sync::Arc, }; +use arrow_data::ffi::FFI_ArrowArray; +use

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069300971 Oh I've reproduce the problem, let me fix it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] [C++][Flight] Downstream MINGW builds are failing to build arrow flight do to undefined reference [arrow]

2024-04-22 Thread via GitHub
lidavidm commented on issue #41333: URL: https://github.com/apache/arrow/issues/41333#issuecomment-2069302263 Seems mingw has `mingw-w64-x86_64-abseil-cpp-20240116.1-1` which _should_ be new enough by the description on that page. Maybe we're forgetting to link to something? -- This is

[I] object_store: visibility over maybe_spawn_blocking method, PutPayload object and MultiPartUpload [arrow-rs]

2024-04-22 Thread via GitHub
Silemo opened a new issue, #5677: URL: https://github.com/apache/arrow-rs/issues/5677 **Which part is this question about** Hi there! Young dev here. I recently opened an issue about having HDFS support, and after your suggestions I am now working on a refactor of

Re: [PR] GH-41329: [C++][Gandiva] Fix gandiva cache size env var [arrow]

2024-04-22 Thread via GitHub
zanmato1984 commented on code in PR #41330: URL: https://github.com/apache/arrow/pull/41330#discussion_r1574529527 ## cpp/src/gandiva/cache.cc: ## @@ -23,23 +23,38 @@ namespace gandiva { -static const size_t DEFAULT_CACHE_SIZE = 5000; +static const int DEFAULT_CACHE_SIZE =

Re: [I] [C++][Flight] Downstream MINGW builds are failing to build arrow flight do to undefined reference [arrow]

2024-04-22 Thread via GitHub
raulcd commented on issue #41333: URL: https://github.com/apache/arrow/issues/41333#issuecomment-2069172251 @kou @lidavidm any idea what the issue is here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[I] Avoid using `arrow_schema` in `downcast_dictionary_array` [arrow-rs]

2024-04-22 Thread via GitHub
jayzhan211 opened a new issue, #5676: URL: https://github.com/apache/arrow-rs/issues/5676 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** I found that I need to import `arrow-schema` for this macro although I have import

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
lidavidm commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2069297299 ADBC doesn't actually rebase on the release commit, we just have a script that manually bumps a version string in a bunch of places -- This is an automated message from

Re: [I] [C++] Acero benchmarks should present a items/s or bytes/s metric [arrow]

2024-04-22 Thread via GitHub
westonpace commented on issue #39664: URL: https://github.com/apache/arrow/issues/39664#issuecomment-2069363713 Issue resolved by pull request 40884 https://github.com/apache/arrow/pull/40884 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Arrow Flight SQL example JDBC driver incompatibility [arrow-rs]

2024-04-22 Thread via GitHub
istvan-fodor commented on code in PR #5666: URL: https://github.com/apache/arrow-rs/pull/5666#discussion_r1574837679 ## arrow-flight/examples/flight_sql_server.rs: ## @@ -725,9 +742,8 @@ impl FlightSqlService for FlightSqlServiceImpl { /// This example shows how to run a

Re: [PR] Arrow Flight SQL example JDBC driver incompatibility [arrow-rs]

2024-04-22 Thread via GitHub
istvan-fodor commented on code in PR #5666: URL: https://github.com/apache/arrow-rs/pull/5666#discussion_r1574850757 ## arrow-flight/examples/flight_sql_server.rs: ## @@ -190,14 +190,8 @@ impl FlightSqlService for FlightSqlServiceImpl { let output =

Re: [PR] add cudf-flight-ucx example [arrow-experiments]

2024-04-22 Thread via GitHub
zeroshade commented on code in PR #28: URL: https://github.com/apache/arrow-experiments/pull/28#discussion_r1574884794 ## dissociated-ipc/cudf-flight-poc.cc: ## @@ -0,0 +1,799 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

[PR] feat(python): Add user-facing ArrayStream class [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot opened a new pull request, #439: URL: https://github.com/apache/arrow-nanoarrow/pull/439 This class provides an interface to the ArrowArrayStream whose methods return `Schema`s and `Array`s. It also provides a more ergonomic interface to the `ipc.Stream` interface.

Re: [PR] Update object_store depedency in arrow to `0.10.0` [arrow-rs]

2024-04-22 Thread via GitHub
tustvold merged PR #5675: URL: https://github.com/apache/arrow-rs/pull/5675 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574512095 ## python/CMakeLists.txt: ## @@ -590,28 +616,42 @@ endif() # Acero if(PYARROW_BUILD_ACERO) - if(PYARROW_BUNDLE_ARROW_CPP) -

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574555323 ## rust/core/src/ffi/mod.rs: ## @@ -16,11 +16,22 @@ // under the License. pub mod constants; +pub(crate) mod methods; pub(crate) mod types; +pub use

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
mbrobbel commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574573423 ## rust/core/src/ffi/mod.rs: ## @@ -16,11 +16,22 @@ // under the License. pub mod constants; +pub(crate) mod methods; pub(crate) mod types; +pub use types::{

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574625991 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,606 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069173994 I've check the C++ using sanitizer: ``` parquet-reader --debug crash-34fd88d625cc5fef893bcba62aad402883d98f47.parquet ``` This raise the error but doesn't cause

Re: [I] [C++][Flight] Downstream MINGW builds are failing to build arrow flight do to undefined reference [arrow]

2024-04-22 Thread via GitHub
assignUser commented on issue #41333: URL: https://github.com/apache/arrow/issues/41333#issuecomment-2069237645 Protobuf added a dependency on abseil in v22+ and iirc we are using 21, could it be that mingw has a more recent version? -- This is an automated message from the Apache Git

Re: [PR] MINOR: [C++][CMake] Remove unused CMAKE_SKIP_INSTALL_ALL_DEPENDENCY [arrow]

2024-04-22 Thread via GitHub
wgtmac merged PR #41332: URL: https://github.com/apache/arrow/pull/41332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
rouault commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069086314 @mapleFU Reproducable with v15.0.0 and latest master at time of writing (https://github.com/apache/arrow/commit/16e20b712ab8dafa9d1ec6154415fe5748fe6135) > have you select

Re: [PR] Move ffi stream and utils from arrow to arrow-array [arrow-rs]

2024-04-22 Thread via GitHub
alexandreyc commented on PR #5670: URL: https://github.com/apache/arrow-rs/pull/5670#issuecomment-2069131518 CI is green now! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574602705 ## python/CMakeLists.txt: ## @@ -763,8 +813,12 @@ foreach(module ${CYTHON_EXTENSIONS}) set_target_properties(${module_name} PROPERTIES COMPILE_DEFINITIONS

Re: [I] [C++] Build failure due to `orc_ep` in IDE [arrow]

2024-04-22 Thread via GitHub
wgtmac commented on issue #41331: URL: https://github.com/apache/arrow/issues/41331#issuecomment-2069215006 @kou Thanks for pinging me! @llama90 Did you use the default CMake presets? How can I reproduce this? -- This is an automated message from the Apache Git Service. To respond

Re: [I] [C++] Build failure due to `orc_ep` in IDE [arrow]

2024-04-22 Thread via GitHub
llama90 commented on issue #41331: URL: https://github.com/apache/arrow/issues/41331#issuecomment-2069228756 Sure, it is my preset. ``` --preset ninja-debug-maximal \ -G Ninja \ -DCMAKE_INSTALL_PREFIX=build \ -DARROW_AZURE:BOOL=OFF \ -DARROW_CUDA=OFF \

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on PR #37822: URL: https://github.com/apache/arrow/pull/37822#issuecomment-2069229762 Oh, also, about the `get_env_options` stuff in setup.py. The problem with your proposed solution is that setup.py depends on those options internally. So if we do everything in

Re: [PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]

2024-04-22 Thread via GitHub
zanmato1984 commented on PR #41335: URL: https://github.com/apache/arrow/pull/41335#issuecomment-2069305289 cc @westonpace @pitrou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] feat(go/adbc/driver/bigquery): add support for Google BigQuery [arrow-adbc]

2024-04-22 Thread via GitHub
cocoa-xu commented on PR #1722: URL: https://github.com/apache/arrow-adbc/pull/1722#issuecomment-2069309031 > all those TODOs are fine to split into later PRs Got it! Then we probably can merge this first once we're happy about with it. I'll do separate PRs for the left bits. :)

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
mbrobbel commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574718776 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,622 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069594570 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] GH-41179: [Docs] Documentation for Dissociated IPC Protocol [arrow]

2024-04-22 Thread via GitHub
bkietz commented on code in PR #41180: URL: https://github.com/apache/arrow/pull/41180#discussion_r1574895346 ## docs/source/format/DissociatedIPC.rst: ## @@ -0,0 +1,335 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license

Re: [I] [C++] Build failure due to `orc_ep` in IDE [arrow]

2024-04-22 Thread via GitHub
kou commented on issue #41331: URL: https://github.com/apache/arrow/issues/41331#issuecomment-2069006615 I'll explain details tomorrow but here are workarounds: * Uninstall Protobuf installed by Homebrew * Use Protobuf installed by Homebrew not bundled Protobuf Cc: @wgtmac

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574522230 ## python/CMakeLists.txt: ## @@ -365,6 +388,9 @@ if(NOT PYARROW_CPP_LINK_LIBS) endif() add_library(arrow_python SHARED ${PYARROW_CPP_SRCS})

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069093626 Ah nice, that's probably a problem about exception safety, I'll dive into it -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
raulcd commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2069177304 On the main repo after bumping versions we push the `dev` tag: https://github.com/apache/arrow/blob/main/dev/release/post-11-bump-versions.sh#L124-L129 -- This is an

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069221185 Emmm would you mind check `poRecordBatchReader->ReadNext` and break when detect error? I try the code below and it still not leak: ``` std::unique_ptr arrow_reader;

Re: [PR] GH-41317: [C++] Fix crash on invalid Parquet file [arrow]

2024-04-22 Thread via GitHub
wgtmac commented on PR #41320: URL: https://github.com/apache/arrow/pull/41320#issuecomment-2069225126 cc @mapleFU -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] GH-41112: [C++] Clean up unused parameter warnings [arrow]

2024-04-22 Thread via GitHub
WillAyd commented on PR #4: URL: https://github.com/apache/arrow/pull/4#issuecomment-2069234103 Unfortunate...so I guess back to the macros? Unless I am reading this wrong I am surprised by that lack of support - this says gcc 7 started support for maybe_unused?

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069297036 do you mean the code here: https://github.com/OSGeo/gdal/blob/27b5611353ae0cfe6d4e0244ef3272723845bd14/ogr/ogrsf_frmts/parquet/ogrparquetlayer.cpp Let me try it -- This is

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574709258 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,622 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574709872 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,606 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [PR] GH-39664: [C++][Acero] Ensure Acero benchmarks present a metric for identifying throughput [arrow]

2024-04-22 Thread via GitHub
westonpace merged PR #40884: URL: https://github.com/apache/arrow/pull/40884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574508173 ## python/CMakeLists.txt: ## @@ -763,8 +813,12 @@ foreach(module ${CYTHON_EXTENSIONS}) set_target_properties(${module_name} PROPERTIES COMPILE_DEFINITIONS

Re: [I] [C++] Build failure due to `orc_ep` in IDE [arrow]

2024-04-22 Thread via GitHub
llama90 commented on issue #41331: URL: https://github.com/apache/arrow/issues/41331#issuecomment-2069036406 @kou You are awesome. I've spent a lot of time trying to solve the problem. 﫠 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574553207 ## rust/core/src/ffi/methods.rs: ## @@ -0,0 +1,93 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] [Python] `tests/test_feather.py::test_roundtrip`: `hypothesis.errors.FailedHealthCheck: Data generation is extremely slow` […] [arrow]

2024-04-22 Thread via GitHub
mgorny commented on issue #41318: URL: https://github.com/apache/arrow/issues/41318#issuecomment-2069100122 Our CI caught one more: ``` FAILED tests/test_pandas.py::test_array_to_pandas_roundtrip - hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Onl...

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574598450 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,622 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574628447 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,606 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
rouault commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069205619 > I guess the problem is in PyArrow wrapper no, it is not PyArrow specific. It can also be reproduced using plain C++ Parquet Arrow API

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2069288546 This is all me (for managing my tags improperly!). I think ADBC rebases on the release commit, which I never quite got to. The heuristic I use for the changelog is to

Re: [PR] GH-41323: [R] Redo how summarize() evaluates expressions [arrow]

2024-04-22 Thread via GitHub
nealrichardson commented on PR #41223: URL: https://github.com/apache/arrow/pull/41223#issuecomment-2069506515 This is ready to go AFAIK @jonkeane @thisisnic, LMK if you have any more feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069554288 The root cause of this memory access is clear, it doesn't happen during reading a "valid" parquet file. During decompressing the "corrupt" file, this file has two row-groups:

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
mapleFU commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069557275 @rouault I've find out the reason here https://github.com/apache/arrow/issues/41321#issuecomment-2069554288 . I'm a bit tired today and will fix it tomorrow. This will not happen

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on PR #37822: URL: https://github.com/apache/arrow/pull/37822#issuecomment-2069199736 @kou I've fixed most of those things you mentioned above - there are a couple of points where it looks weird as per your comments but is right in this case - 1) I don't

[PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]

2024-04-22 Thread via GitHub
zanmato1984 opened a new pull request, #41335: URL: https://github.com/apache/arrow/pull/41335 ### Rationale for this change See #41334 . ### What changes are included in this PR? Add env var `ACERO_TEMP_STACK_SIZE` to augment the size of temp stack in

Re: [PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]

2024-04-22 Thread via GitHub
github-actions[bot] commented on PR #41335: URL: https://github.com/apache/arrow/pull/41335#issuecomment-2069303054 :warning: GitHub issue #41334 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] [Java][JPMS][IDE] Fix developer docs for building Arrow Java JPMS in IntelliJ IDE [arrow]

2024-04-22 Thread via GitHub
lidavidm commented on issue #40087: URL: https://github.com/apache/arrow/issues/40087#issuecomment-2069309242 Basically, the IDE intelligence will be semi broken and you can only build on command line (or tell IntelliJ to invoke Maven which is basically the same thing in as far as you

Re: [PR] Arrow Flight SQL example JDBC driver incompatibility [arrow-rs]

2024-04-22 Thread via GitHub
istvan-fodor commented on code in PR #5666: URL: https://github.com/apache/arrow-rs/pull/5666#discussion_r1574835054 ## arrow-flight/examples/flight_sql_server.rs: ## @@ -999,15 +1028,6 @@ mod tests { .to_string() .contains("Invalid

Re: [PR] add cudf-flight-ucx example [arrow-experiments]

2024-04-22 Thread via GitHub
zeroshade commented on PR #28: URL: https://github.com/apache/arrow-experiments/pull/28#issuecomment-2069777958 @kou so all of the things related directly to the protocol are in the `cudf-flight-poc.cc` file. Everything else is just wrapping the underlying UCX primitives with nicer classes

Re: [I] Allow adding existing arrays into structs [arrow]

2024-04-22 Thread via GitHub
zeroshade commented on issue #41284: URL: https://github.com/apache/arrow/issues/41284#issuecomment-2069760332 Because of the need for the offsets buffer and so on, it's tough to think of what the interface for such helper functions for Lists and Maps would be. What do you think would make

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574510160 ## python/CMakeLists.txt: ## @@ -683,14 +725,22 @@ if(PYARROW_BUILD_SUBSTRAIT) message(FATAL_ERROR "You must build Arrow C++ with ARROW_SUBSTRAIT=ON")

Re: [PR] GH-23221: [Python] python changes for pyodide build [arrow]

2024-04-22 Thread via GitHub
joemarshall commented on code in PR #37822: URL: https://github.com/apache/arrow/pull/37822#discussion_r1574511165 ## python/CMakeLists.txt: ## @@ -590,28 +616,42 @@ endif() # Acero if(PYARROW_BUILD_ACERO) - if(PYARROW_BUNDLE_ARROW_CPP) -

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
mbrobbel commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574567409 ## rust/core/src/ffi/methods.rs: ## @@ -0,0 +1,93 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] [Java][JPMS][IDE] Fix developer docs for building Arrow Java JPMS in IntelliJ IDE [arrow]

2024-04-22 Thread via GitHub
llama90 commented on issue #40087: URL: https://github.com/apache/arrow/issues/40087#issuecomment-2069103643 Hello. Hello. I am interested in Java in addition to the Arrow code, and I'm wondering if there is a way to set up an environment and perform code using IntelliJ? From

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r157457 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,622 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [PR] feat(rust): add complete FFI bindings [arrow-adbc]

2024-04-22 Thread via GitHub
alexandreyc commented on code in PR #1742: URL: https://github.com/apache/arrow-adbc/pull/1742#discussion_r1574625991 ## rust/core/src/ffi/types.rs: ## @@ -17,4 +17,606 @@ #![allow(non_camel_case_types, non_snake_case)] +use std::ffi::{CStr, CString}; +use

Re: [I] [C++][Parquet] Crash / heap-use-after-free in ByteArrayChunkedRecordReader::ReadValuesSpaced() on a corrupted Parquet file [arrow]

2024-04-22 Thread via GitHub
rouault commented on issue #41321: URL: https://github.com/apache/arrow/issues/41321#issuecomment-2069228166 > I try the code below by chance, can you share the source code of the standalone .cpp you use, so I can start from that ? That will make it easier for me to tune it to

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2069353280 Git describe seems to work for ADBC: https://github.com/apache/arrow-adbc/blob/257420a69f9215de12c8fcb8d90c95a10e510380/.github/workflows/packaging.yml#L85-L90

Re: [I] [C++][Flight] Downstream MINGW builds are failing to build arrow flight do to undefined reference [arrow]

2024-04-22 Thread via GitHub
assignUser commented on issue #41333: URL: https://github.com/apache/arrow/issues/41333#issuecomment-2069368784 When I cross compiled arrow r in a mingw based thing there was an issue with abseil dependencies not being properly forwarded through pkg config (I think it was re2). -- This

Re: [PR] add cudf-flight-ucx example [arrow-experiments]

2024-04-22 Thread via GitHub
ianmcook commented on code in PR #28: URL: https://github.com/apache/arrow-experiments/pull/28#discussion_r1574980334 ## dissociated-ipc/README.md: ## @@ -0,0 +1,56 @@ + + +# Arrow Dissociated IPC Protocol Example + +This directory contains a reference example implementation of

Re: [I] python: Nightly wheel upload is failing [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot commented on issue #437: URL: https://github.com/apache/arrow-nanoarrow/issues/437#issuecomment-2070323482 I'm game for either wiring up the logic in `changelog.py` or by pushing a dev tag (but one of you might have to point me to where this happens in the Arrow repo). --

Re: [PR] feat(python): function to inspect a single-chunk Array [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot commented on code in PR #436: URL: https://github.com/apache/arrow-nanoarrow/pull/436#discussion_r1575119448 ## python/src/nanoarrow/_repr_utils.py: ## @@ -248,3 +248,46 @@ def device_repr(device): device_type = f"- device_type: {device.device_type.name}

Re: [PR] add some range-for helpers [arrow-nanoarrow]

2024-04-22 Thread via GitHub
paleolimbot commented on code in PR #404: URL: https://github.com/apache/arrow-nanoarrow/pull/404#discussion_r1574933882 ## src/nanoarrow/nanoarrow.hpp: ## @@ -548,6 +550,312 @@ class VectorArrayStream { /// @} +struct Nothing {}; Review Comment: I know it is a bit

Re: [PR] add cudf-flight-ucx example [arrow-experiments]

2024-04-22 Thread via GitHub
ianmcook commented on code in PR #28: URL: https://github.com/apache/arrow-experiments/pull/28#discussion_r1574974330 ## data/taxi-data/train.parquet: ## Review Comment: Can you please add a `README.md` file in `data/taxi-data` briefly explaining what this data represents

Re: [PR] fix(go/adbc/driver/snowflake): handle quotes properly [arrow-adbc]

2024-04-22 Thread via GitHub
zeroshade commented on PR #1738: URL: https://github.com/apache/arrow-adbc/pull/1738#issuecomment-2070087202 @lidavidm Ultimately this comes down to the question of who is responsible for ensuring we avoid sql injection vs ease of use. Because we provide this whole bulk ingestion API

Re: [PR] MINOR: [C++][CMake] Remove unused CMAKE_SKIP_INSTALL_ALL_DEPENDENCY [arrow]

2024-04-22 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41332: URL: https://github.com/apache/arrow/pull/41332#issuecomment-2070120343 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 08aefc395d5763e405c736c35231706f6f9634a2. There were

[PR] MINOR: [Go] Bump modernc.org/sqlite from 1.29.6 to 1.29.8 in /go [arrow]

2024-04-22 Thread via GitHub
dependabot[bot] opened a new pull request, #41337: URL: https://github.com/apache/arrow/pull/41337 Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.29.6 to 1.29.8. Commits

Re: [PR] GH-41179: [Docs] Documentation for Dissociated IPC Protocol [arrow]

2024-04-22 Thread via GitHub
zeroshade commented on code in PR #41180: URL: https://github.com/apache/arrow/pull/41180#discussion_r1575009495 ## docs/source/format/DissociatedIPC.rst: ## @@ -0,0 +1,335 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license

Re: [PR] MINOR: [Go] Bump modernc.org/sqlite from 1.29.5 to 1.29.7 in /go [arrow]

2024-04-22 Thread via GitHub
dependabot[bot] closed pull request #41236: MINOR: [Go] Bump modernc.org/sqlite from 1.29.5 to 1.29.7 in /go URL: https://github.com/apache/arrow/pull/41236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

  1   2   3   >