Re: [I] [Python] Support pyarrow.Table.cast with CastOptions [arrow]

2025-04-15 Thread via GitHub
AlenkaF commented on issue #46138: URL: https://github.com/apache/arrow/issues/46138#issuecomment-2808513620 I will try to investigate this today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]

2025-04-15 Thread via GitHub
AlenkaF commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2808500280 Thank you for opening up the issue @vadimkantorov. Pandas is not a required dependency for PyArrow, but a lot of functionality is designed to work seamlessly when both PyArrow

Re: [PR] GH-46130: [Python] Remove `use_legacy_format` in favour of setting `IpcWriteOptions` [arrow]

2025-04-15 Thread via GitHub
AlenkaF commented on code in PR #46131: URL: https://github.com/apache/arrow/pull/46131#discussion_r2046136210 ## python/pyarrow/ipc.py: ## @@ -150,9 +140,8 @@ def _ensure_default_ipc_read_options(options): return options or IpcReadOptions() -def new_stream(sink, schema

Re: [PR] Add support for file row numbers in Parquet readers [arrow-rs]

2025-04-15 Thread via GitHub
scovich commented on PR #7307: URL: https://github.com/apache/arrow-rs/pull/7307#issuecomment-2808130256 > I think we need to be very careful to balance adding new features in the parquet reader with keeping it fast and maintainable. I haven't had a chance to look at this PR yet, but I do w

Re: [I] object_store: Automatically fall back to use_azure_cli=True if no other credentials can be found [arrow-rs-object-store]

2025-04-15 Thread via GitHub
kylebarron commented on issue #290: URL: https://github.com/apache/arrow-rs-object-store/issues/290#issuecomment-2808114816 I think the creation of `ImdsManagedIdentityProvider` **never fails**. It's only at _runtime_, _after creation_, when the user makes the first request, that the insta

Re: [PR] [PATHFINDING] Parse json as variant [arrow-rs]

2025-04-15 Thread via GitHub
scovich commented on PR #7403: URL: https://github.com/apache/arrow-rs/pull/7403#issuecomment-2808107096 > > TBD Where/how to expose this functionality through a public API > > In my mind this functionality feels like a "computation kernel" (aka similarly to the functions in https://

Re: [I] object_store: Automatically fall back to use_azure_cli=True if no other credentials can be found [arrow-rs-object-store]

2025-04-15 Thread via GitHub
daviewales commented on issue #290: URL: https://github.com/apache/arrow-rs-object-store/issues/290#issuecomment-2807942096 My expectation is that Azure CLI would be tried _after_ the `ImdsManagedIdentityProvider`. Only if creation of `ImdsManagedIdentityProvider` fails should Azure CLI

Re: [PR] Default wasm32-unknown-unknown HttpConnector [arrow-rs-object-store]

2025-04-15 Thread via GitHub
H-Plus-Time commented on PR #329: URL: https://github.com/apache/arrow-rs-object-store/pull/329#issuecomment-2807878631 I tested through object-store-wasm (which provided all the wasm-bindgen bindings to create object store instances and so on from JS), but a reasonable approach would be:

[PR] MINOR: [C++][Parquet] Fix incorrect comments about dictionary encoding fallback behaviour [arrow]

2025-04-15 Thread via GitHub
adamreeve opened a new pull request, #46158: URL: https://github.com/apache/arrow/pull/46158 ### Rationale for this change Prevent confusion of developers reading the Parquet source code or generated API documentation. ### What changes are included in this PR? Updates co

Re: [PR] GH-45829: [C++] Add compute directory to Meson configuration [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on PR #45830: URL: https://github.com/apache/arrow/pull/45830#issuecomment-2807782359 I don't believe the test failure is related to the changes made in this PR as it is reproducible with CMake. I opened an issue for it in https://github.com/apache/arrow/issues/46157 --

Re: [I] Arrow 55 not building with other versions of Arrow [arrow-rs]

2025-04-15 Thread via GitHub
EricFecteau commented on issue #7410: URL: https://github.com/apache/arrow-rs/issues/7410#issuecomment-2807721860 Unfortunately, this does not work because the latest version of arrow depends on chrono `^0.4.40`. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] GH-46075: [Release][CI] Fix binary verification [arrow]

2025-04-15 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46076: URL: https://github.com/apache/arrow/pull/46076#issuecomment-2807634786 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 4937cf5721bd4912438964377361c4ec49fd5e80. There was 1 b

Re: [PR] GH-45829: [C++] Add compute directory to Meson configuration [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #45830: URL: https://github.com/apache/arrow/pull/45830#issuecomment-2807620623 Revision: db6f5568b4159f0db7e0ed5ec9f05e4f5c4cc2f7 Submitted crossbow builds: [ursacomputing/crossbow @ actions-b82155104d](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45829: [C++] Add compute directory to Meson configuration [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on PR #45830: URL: https://github.com/apache/arrow/pull/45830#issuecomment-2807616768 @github-actions crossbow submit *meson -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat(csharp/src/Drivers/Apache): enhance GetColumns with BASE_TYPE_NAME column [arrow-adbc]

2025-04-15 Thread via GitHub
birschick-bq commented on code in PR #2695: URL: https://github.com/apache/arrow-adbc/pull/2695#discussion_r2045534408 ## csharp/src/Drivers/Apache/Hive2/HiveServer2Statement.cs: ## @@ -406,7 +407,36 @@ private async Task GetColumnsAsync(CancellationToken cancellationTo

Re: [PR] feat(csharp/src/Drivers/Apache): enhance GetColumns with BASE_TYPE_NAME column [arrow-adbc]

2025-04-15 Thread via GitHub
birschick-bq commented on code in PR #2695: URL: https://github.com/apache/arrow-adbc/pull/2695#discussion_r2045527580 ## csharp/src/Drivers/Apache/Hive2/HiveServer2Statement.cs: ## @@ -426,12 +427,128 @@ private async Task GetQueryResult(TSparkDirectResults? directResult

Re: [PR] feat(csharp/src/Drivers/Apache): enhance GetColumns with BASE_TYPE_NAME column [arrow-adbc]

2025-04-15 Thread via GitHub
birschick-bq commented on code in PR #2695: URL: https://github.com/apache/arrow-adbc/pull/2695#discussion_r2045527580 ## csharp/src/Drivers/Apache/Hive2/HiveServer2Statement.cs: ## @@ -426,12 +427,128 @@ private async Task GetQueryResult(TSparkDirectResults? directResult

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#discussion_r2045487830 ## src/client/builder.rs: ## @@ -183,7 +183,8 @@ impl HttpRequestBuilder { let mut error = None; if let Ok(ref mut req) = self.request {

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#discussion_r2045487830 ## src/client/builder.rs: ## @@ -183,7 +183,8 @@ impl HttpRequestBuilder { let mut error = None; if let Ok(ref mut req) = self.request {

Re: [PR] feat: Add `SpawnService` and `SpawnedReqwestConnector` for running requests on a different runtime [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2045452723 ## src/aws/mod.rs: ## @@ -820,4 +821,54 @@ mod tests { store.delete(location).await.unwrap(); } } + +/// Integration test that

Re: [PR] GH-45829: [C++] Add compute directory to Meson configuration [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #45830: URL: https://github.com/apache/arrow/pull/45830#issuecomment-2807464048 Revision: c97bec990621dc463d2c249d95626bc24d994311 Submitted crossbow builds: [ursacomputing/crossbow @ actions-3bb4e7a944](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45829: [C++] Add compute directory to Meson configuration [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on PR #45830: URL: https://github.com/apache/arrow/pull/45830#issuecomment-2807457087 @github-actions crossbow submit *meson -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [C++] FixedSizeListBuilder should have UnsafeAppend methods [arrow]

2025-04-15 Thread via GitHub
pitrou commented on issue #45723: URL: https://github.com/apache/arrow/issues/45723#issuecomment-2807440037 @gauravrp Do you still want to work on this as you proposed above? If not, then @cramosme can take it up. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] GH-46155: [C++] Implement Tensorflow directory in Meson [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46156: URL: https://github.com/apache/arrow/pull/46156#issuecomment-2807433236 :warning: GitHub issue #46155 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-46155: [C++] Implement Tensorflow directory in Meson [arrow]

2025-04-15 Thread via GitHub
WillAyd opened a new pull request, #46156: URL: https://github.com/apache/arrow/pull/46156 ### Rationale for this change This continues improving the coverage of the Meson build configuration ### What changes are included in this PR? Added the Tensorflow directory

[PR] GH-46152: [C++] Implement acero directory in Meson [arrow]

2025-04-15 Thread via GitHub
WillAyd opened a new pull request, #46154: URL: https://github.com/apache/arrow/pull/46154 ### Rationale for this change Improves coverage of the Meson build system configuration ### What changes are included in this PR? This adds the Acero directory to Meson ### A

Re: [PR] GH-46152: [C++] Implement acero directory in Meson [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on PR #46154: URL: https://github.com/apache/arrow/pull/46154#issuecomment-2807405916 Draft for now - this will require a rebase after https://github.com/apache/arrow/pull/45830 gets merged, but the overall structure should not change -- This is an automated message fro

Re: [PR] GH-46152: [C++] Implement acero directory in Meson [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46154: URL: https://github.com/apache/arrow/pull/46154#issuecomment-2807406235 :warning: GitHub issue #46152 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-46132: [C++][Parquet] Remove deprecated parquet APIs from 19.0.0 [arrow]

2025-04-15 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46133: URL: https://github.com/apache/arrow/pull/46133#issuecomment-2807401225 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit f7bc27132ea84c4639528d73a9c289c7a1db154f. There were no

Re: [PR] feat: spawn-service [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2045392668 ## src/client/http/connection.rs: ## @@ -244,6 +245,67 @@ impl HttpConnector for ReqwestConnector { } } +/// [`reqwest::Client`] connector that perfo

Re: [PR] feat: spawn-service [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2045350077 ## src/client/http/mod.rs: ## @@ -0,0 +1,41 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
AdamGS commented on PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#issuecomment-2807303161 merged `main` so we can run tests here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] CI doesn't run on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb closed issue #335: CI doesn't run on PRs URL: https://github.com/apache/arrow-rs-object-store/issues/335 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb merged PR #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

Re: [PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on PR #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336#issuecomment-2807265879 We run tests, sure we do. Thank you very much @AdamGS ![image](https://github.com/user-attachments/assets/88b5d973-1037-415f-a2e9-9351e1de8c97) -- This is an auto

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
AdamGS commented on code in PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#discussion_r2045308481 ## src/client/builder.rs: ## @@ -183,7 +183,8 @@ impl HttpRequestBuilder { let mut error = None; if let Ok(ref mut req) = self.request {

Re: [PR] feat: spawn-service [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#issuecomment-2807251994 Note due to https://github.com/apache/arrow-rs-object-store/issues/335 the CI was not running. I plan to merge up from main shortly -- This is an automated message from the A

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#issuecomment-2807250737 FYI @crepererum and @tustvold since you approved / authored https://github.com/apache/arrow-rs/pull/7183 I would appreciate it if you could also give this PR a look -- This

Re: [PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on PR #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336#issuecomment-2807245775 🤞 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on PR #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336#issuecomment-2807242955 ![Screenshot 2025-04-15 at 3 18 46  PM](https://github.com/user-attachments/assets/77959789-5921-426b-bf67-d1396438a2aa) ❤️ -- This is an automated message from the Ap

Re: [PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336#discussion_r2045299015 ## .github/workflows/ci.yml: ## @@ -30,7 +30,8 @@ on: - main pull_request: paths: - - object_store/** + - src/** Review Comment:

Re: [I] CI doesn't run on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on issue #335: URL: https://github.com/apache/arrow-rs-object-store/issues/335#issuecomment-2807236586 Well, that is embarrasing 🤦 -- will fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
alamb commented on code in PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#discussion_r2045293398 ## src/client/builder.rs: ## @@ -183,7 +183,8 @@ impl HttpRequestBuilder { let mut error = None; if let Ok(ref mut req) = self.request {

[PR] Run CI on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
AdamGS opened a new pull request, #336: URL: https://github.com/apache/arrow-rs-object-store/pull/336 # Which issue does this PR close? Closes #335 Closes #. # Rationale for this change # What changes are included in this PR? # Are there an

[I] CI doesn't run on PRs [arrow-rs-object-store]

2025-04-15 Thread via GitHub
AdamGS opened a new issue, #335: URL: https://github.com/apache/arrow-rs-object-store/issues/335 Seems like the CI workflow was migrated when the repo was split but it still points at old file paths preventing it from being triggered on PRs. -- This is an automated message from the Apache

Re: [PR] MINOR: [C++][Docs] Fix doxygen issue in ree_util.h [arrow]

2025-04-15 Thread via GitHub
zanmato1984 commented on PR #46012: URL: https://github.com/apache/arrow/pull/46012#issuecomment-2807222820 No problem at all. Just learning how we are organizing the doc. Thank you @amoeba ! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-15 Thread via GitHub
kou commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2807183435 @github-actions crossbow submit --group verify-rc-source --param release=20.0.0 --param rc=1 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2807190690 Revision: 8e83a99e8399abd3c5bc539941b6bb9bfefae0e0 Submitted crossbow builds: [ursacomputing/crossbow @ actions-b0123f432f](https://github.com/ursacomputing/crossbow/bra

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-15 Thread via GitHub
kou commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2807190141 Revision: apache-arrow-20.0.0-rc1 Submitted crossbow builds: [ursacomputing/crossbow @ release-20.0.0-rc1-0](https://github.com/ursacomputing/crossbow/branches/all?query=release-20.0.0-

Re: [PR] feat(csharp/src/Drivers/Databricks): Support server side property passthrough [arrow-adbc]

2025-04-15 Thread via GitHub
alexguo-db commented on PR #2692: URL: https://github.com/apache/arrow-adbc/pull/2692#issuecomment-2807185357 > Is there some type of test that can be included to demonstrate this is working correctly? I added an E2E test for this -- This is an automated message from the Apache Git

[PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-15 Thread via GitHub
kou opened a new pull request, #46152: URL: https://github.com/apache/arrow/pull/46152 PR to verify Release Candidate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
zanmato1984 commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045208022 ## cpp/src/arrow/compute/row/compare_internal.cc: ## @@ -276,12 +276,13 @@ void KeyCompare::CompareVarBinaryColumnToRowHelper( int32_t tail_length = length -

Re: [I] [C++] arrow::SchemaBuidler::AddMetadata() replaces metadata not adds metadata [arrow]

2025-04-15 Thread via GitHub
bkietz commented on issue #46146: URL: https://github.com/apache/arrow/issues/46146#issuecomment-2807122134 This change seems fine to me; it doesn't seem to be used anywhere except type_test and it doesn't seem to be in any of the bindings. Just in case, I'd recommend not requiring `impl_->

Re: [PR] MINOR: [C++][Docs] Fix doxygen issue in ree_util.h [arrow]

2025-04-15 Thread via GitHub
amoeba commented on PR #46012: URL: https://github.com/apache/arrow/pull/46012#issuecomment-2807103886 Hi @zanmato1984, it's not documented, no. It didn't look like something we need/want to document, what do you think? -- This is an automated message from the Apache Git Service. To respo

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045192318 ## dev/tasks/tasks.yml: ## @@ -1087,13 +1087,25 @@ tasks: image: ubuntu-r-sanitizer timeout: 120 - test-r-clang-sanitizer: + test-r-clang-asan:

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
zanmato1984 commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045188499 ## dev/tasks/tasks.yml: ## @@ -1087,13 +1087,25 @@ tasks: image: ubuntu-r-sanitizer timeout: 120 - test-r-clang-sanitizer: + test-r-clang-asan:

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
zanmato1984 commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045188499 ## dev/tasks/tasks.yml: ## @@ -1087,13 +1087,25 @@ tasks: image: ubuntu-r-sanitizer timeout: 120 - test-r-clang-sanitizer: + test-r-clang-asan:

[PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-15 Thread via GitHub
AdamGS opened a new pull request, #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334 # Which issue does this PR close? Closes #320 Closes #. # Rationale for this change https://github.com/apache/arrow-rs/pull/7183 Changed the way URLs are

Re: [PR] MINOR: [C++][Docs] Fix doxygen issue in ree_util.h [arrow]

2025-04-15 Thread via GitHub
zanmato1984 commented on PR #46012: URL: https://github.com/apache/arrow/pull/46012#issuecomment-2807026451 Aha, hit the same one after upgrading to Apple clang 17.0.0. Thank you @amoeba for fixing this! I have one side questions though: is the piece of doc in question accessible in

Re: [PR] GH-41672: [Python][Doc] Clarify docstring of FixedSizeListArray.values that it ignores the offset [arrow]

2025-04-15 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46144: URL: https://github.com/apache/arrow/pull/46144#issuecomment-2807023702 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 4d6b3a973850c63c2a3638d6a708de0e40579378. There were no

Re: [I] Can you clarify what BufferedStreamEnabled does? [arrow-go]

2025-04-15 Thread via GitHub
stevbear commented on issue #354: URL: https://github.com/apache/arrow-go/issues/354#issuecomment-2802799098 @zeroshade FYI since we've discussed optimizing object store latencies before. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-698: Improve and fix Avro read consumers [arrow-java]

2025-04-15 Thread via GitHub
martin-traverse commented on PR #718: URL: https://github.com/apache/arrow-java/pull/718#issuecomment-2803951943 > I think in the interest of trying to keep semver, we should avoid breaking changes if possible. Any thoughts @jbonofre @laurentgo? Or we could just call the next release 19.0.0

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-2803480854 @github-actions crossbow submit test-r-m1-san -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] fix(python/adbc_driver_postgresql): handle kwargs in dbapi connect [arrow-adbc]

2025-04-15 Thread via GitHub
lidavidm commented on PR #2700: URL: https://github.com/apache/arrow-adbc/pull/2700#issuecomment-2800231285 Thanks for the PR. Let me debug what's going on and I'll file a new PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [I] Add end-user logging and tracing for drivers [arrow-adbc]

2025-04-15 Thread via GitHub
birschick-bq commented on issue #2210: URL: https://github.com/apache/arrow-adbc/issues/2210#issuecomment-2806970076 Observations: - ADBC drivers should adopt OpenTelemetry instrumentation for tracing as is provides for a good variety of [export targets](https://opentelemetry.io/ecos

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
amoeba commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045098899 ## docker-compose.yml: ## @@ -1745,6 +1746,34 @@ services: /bin/bash -c " /arrow/ci/scripts/r_sanitize.sh /arrow" + r-clang-ubsan: +image: ${REPO}

Re: [PR] Odbc tests folder [arrow]

2025-04-15 Thread via GitHub
rscales closed pull request #46150: Odbc tests folder URL: https://github.com/apache/arrow/pull/46150 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gi

Re: [PR] GH-39294: [C++][Python] DLPack on Tensor class [arrow]

2025-04-15 Thread via GitHub
jjerphan commented on PR #42118: URL: https://github.com/apache/arrow/pull/42118#issuecomment-2805045957 @jjerphan: I appreciate your ping, unfortunately I don't have time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] feat(go/adbc/driver/flightsql): Add OAuth Support to Flight Client [arrow-adbc]

2025-04-15 Thread via GitHub
howareyouman commented on code in PR #2651: URL: https://github.com/apache/arrow-adbc/pull/2651#discussion_r2044144115 ## go/adbc/driver/flightsql/flightsql_database.go: ## @@ -384,24 +418,30 @@ func getFlightClient(ctx context.Context, loc string, d *databaseImpl, authMiddl

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2044389136 ## ci/scripts/r_sanitize.sh: ## @@ -36,34 +36,36 @@ ncores=$(${R_BIN} -s -e 'cat(parallel::detectCores())') echo "MAKEFLAGS=-j${ncores}" >> ${rhome}/etc/Renviron.site

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on code in PR #46124: URL: https://github.com/apache/arrow/pull/46124#discussion_r2045083661 ## docker-compose.yml: ## @@ -1745,6 +1746,34 @@ services: /bin/bash -c " /arrow/ci/scripts/r_sanitize.sh /arrow" + r-clang-ubsan: +image: ${REP

Re: [PR] Add benchmark for parquet reader with row_filter and project settings [arrow-rs]

2025-04-15 Thread via GitHub
alamb merged PR #7401: URL: https://github.com/apache/arrow-rs/pull/7401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] Add benchmark for parquet reader with row_filter and project settings [arrow-rs]

2025-04-15 Thread via GitHub
alamb commented on PR #7401: URL: https://github.com/apache/arrow-rs/pull/7401#issuecomment-2806880649 > And still no performance improvement compare the page cache PR to main branch. I am confused why datafusion benchmark will be improved but the benchmark here will not show performance im

[PR] Odbc tests folder [arrow]

2025-04-15 Thread via GitHub
rscales opened a new pull request, #46150: URL: https://github.com/apache/arrow/pull/46150 Thanks for opening a pull request! If this is your first pull request you can find detailed information on how to contribute here: * [New Contributor's Guide](https://arrow.apache.org/

Re: [PR] Odbc tests folder [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46150: URL: https://github.com/apache/arrow/pull/46150#issuecomment-2806859783 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue f

Re: [PR] GH-41672: [Python][Doc] Clarify docstring of FixedSizeListArray.values that it ignores the offset [arrow]

2025-04-15 Thread via GitHub
thisisnic commented on PR #46144: URL: https://github.com/apache/arrow/pull/46144#issuecomment-2804351636 @AlenkaF - just wondering; do I need to also rebuild the docs and include an updated correpsonding `.rst` file as part of this PR? -- This is an automated message from the Apache Git

Re: [PR] chore(r): Update CI setup to silence warnings for Go-based drivers [arrow-adbc]

2025-04-15 Thread via GitHub
paleolimbot commented on PR #2708: URL: https://github.com/apache/arrow-adbc/pull/2708#issuecomment-2805309942 That sounds hard, but good to know there's an option! The driver packages have very little R code and so the WARNINGs aren't really catching anything anyway (the mostly likely thin

Re: [PR] Variant Support for Arrow and Parquet [DRAFT] [arrow-rs]

2025-04-15 Thread via GitHub
PinkCrow007 commented on code in PR #7404: URL: https://github.com/apache/arrow-rs/pull/7404#discussion_r2045042520 ## arrow-schema/src/extension/canonical/variant.rs: ## @@ -0,0 +1,233 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lic

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-280309 @github-actions crossbow submit test-r-m1-san -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] GH-46092: [C++] Add filesystem option to Meson [arrow]

2025-04-15 Thread via GitHub
kou commented on code in PR #46101: URL: https://github.com/apache/arrow/pull/46101#discussion_r2041292188 ## cpp/src/arrow/filesystem/meson.build: ## @@ -0,0 +1,87 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See t

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-2800166287 Revision: d33a710ba2d6d96ebb1cda6e2596b1d3380872d2 Submitted crossbow builds: [ursacomputing/crossbow @ actions-42aba23e69](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-2803638776 Revision: e1d844f5d4a62417d69b20d7258720da1535cf89 Submitted crossbow builds: [ursacomputing/crossbow @ actions-ac9b480ec4](https://github.com/ursacomputing/crossbow/bra

[I] Support FixedSizeList in JSON reader [arrow-rs]

2025-04-15 Thread via GitHub
H2CO3 opened a new issue, #7414: URL: https://github.com/apache/arrow-rs/issues/7414 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** I'm trying to use LanceDB through FFI. I am transfering the initial data as serialized JSON acros

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-2803099487 Revision: 98c63b381ddc26a4ad0a4f317f5e7b2d2b42837d Submitted crossbow builds: [ursacomputing/crossbow @ actions-c47ffba7ee](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-46134: [CI][C++] Explicit conversion of possible `absl::string_view` on protobuf methods to `std::string` [arrow]

2025-04-15 Thread via GitHub
kou merged PR #46136: URL: https://github.com/apache/arrow/pull/46136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-46123: [C++] Undefined behavior in `compare_internal.cc` and `light_array_internal.cc` [arrow]

2025-04-15 Thread via GitHub
jonkeane commented on PR #46124: URL: https://github.com/apache/arrow/pull/46124#issuecomment-2803684505 @github-actions crossbow submit test-r-m1-san This is the m1-san job after the fix -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Variant Support for Arrow and Parquet [DRAFT] [arrow-rs]

2025-04-15 Thread via GitHub
PinkCrow007 commented on code in PR #7404: URL: https://github.com/apache/arrow-rs/pull/7404#discussion_r2043575052 ## arrow-array/src/array/variant_array.rs: ## @@ -0,0 +1,628 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [I] [Python][Doc] Clarify docstring of FixedSizeListArray.values that it ignores the offset [arrow]

2025-04-15 Thread via GitHub
thisisnic commented on issue #41672: URL: https://github.com/apache/arrow/issues/41672#issuecomment-2804306938 Hey @PeopleMakeCulture - as this has been open a while without a PR, I'm gonna grab it, hope you don't mind! (Also, hey Jing, didn't realise this is you, see you at RC in a few wee

Re: [PR] GH-46092: [C++] Add filesystem related options to Meson [arrow]

2025-04-15 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46101: URL: https://github.com/apache/arrow/pull/46101#issuecomment-2803816784 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 6b64396dfedfb42e00a484f7ade21c099914e0db. There were no

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-15 Thread via GitHub
mgorny commented on code in PR #45854: URL: https://github.com/apache/arrow/pull/45854#discussion_r2044645547 ## python/pyproject.toml: ## @@ -17,19 +17,21 @@ [build-system] requires = [ +"cmake < 4", Review Comment: Why do you need to add a dependency on CMake? Such

Re: [PR] GH-38903: [R][Docs] Improve documentation of col_types [arrow]

2025-04-15 Thread via GitHub
github-actions[bot] commented on PR #46145: URL: https://github.com/apache/arrow/pull/46145#issuecomment-2804474066 :warning: GitHub issue #38903 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on code in PR #45854: URL: https://github.com/apache/arrow/pull/45854#discussion_r2044767032 ## python/pyproject.toml: ## @@ -17,19 +17,21 @@ [build-system] requires = [ +"cmake < 4", "cython >= 3", # Starting with NumPy 1.25, NumPy is (by def

Re: [PR] GH-46132: [C++][Parquet] Remove deprecated parquet APIs from 19.0.0 [arrow]

2025-04-15 Thread via GitHub
kszucs merged PR #46133: URL: https://github.com/apache/arrow/pull/46133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-45653: [Python] Scalar subclasses should implement Python protocols [arrow]

2025-04-15 Thread via GitHub
AlenkaF commented on code in PR #45818: URL: https://github.com/apache/arrow/pull/45818#discussion_r2044537069 ## python/pyarrow/scalar.pxi: ## @@ -887,7 +887,29 @@ cdef class BinaryScalar(Scalar): def __getbuffer__(self, cp.Py_buffer* buffer, int flags): cdef Bu

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-15 Thread via GitHub
WillAyd commented on code in PR #45854: URL: https://github.com/apache/arrow/pull/45854#discussion_r2044740300 ## python/pyproject.toml: ## @@ -17,19 +17,21 @@ [build-system] requires = [ +"cmake < 4", Review Comment: Ah nice catch - I think this is a relic from the

Re: [I] [Python] Support pyarrow.Table.cast with CastOptions [arrow]

2025-04-15 Thread via GitHub
raulcd commented on issue #46138: URL: https://github.com/apache/arrow/issues/46138#issuecomment-2804290837 cc @AlenkaF @rok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Support round tripping extension types in parquet [arrow-rs]

2025-04-15 Thread via GitHub
paleolimbot commented on issue #7063: URL: https://github.com/apache/arrow-rs/issues/7063#issuecomment-2805716506 > extension types automatically "just work" This is definitely true for some operations (e.g., arrow-select), but for others (e.g., cast, parse, print, write to CSV, arith

[PR] GH-698: Improve and fix Avro read consumers [arrow-java]

2025-04-15 Thread via GitHub
martin-traverse opened a new pull request, #718: URL: https://github.com/apache/arrow-java/pull/718 Hi @lidavidm - here is part 2 in my Avro series, apologies for the delay, it's the usual work / contention story! ## What's Changed This PR relates to #698 and is the second in a

Re: [PR] ci: build and test JNI module [arrow-adbc]

2025-04-15 Thread via GitHub
lidavidm merged PR #2686: URL: https://github.com/apache/arrow-adbc/pull/2686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] Implement `Default` for `ScalarBuffer` [arrow-rs]

2025-04-15 Thread via GitHub
mbrobbel commented on code in PR #7413: URL: https://github.com/apache/arrow-rs/pull/7413#discussion_r2042091031 ## arrow-buffer/src/buffer/scalar.rs: ## @@ -48,6 +48,13 @@ pub struct ScalarBuffer { phantom: PhantomData, } +impl Default for ScalarBuffer { +#[inline]

  1   2   >