Re: [I] [Python] Schema inference reorders fields in nested structs [arrow]

2025-04-16 Thread via GitHub
tooptoop4 commented on issue #34250: URL: https://github.com/apache/arrow/issues/34250#issuecomment-2810780132 any chance of getting schema from all rows? so the new tag is not lost in example above -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] feat(parquet): add variant encoder/decoder [arrow-go]

2025-04-16 Thread via GitHub
sfc-gh-mbojanczyk commented on PR #344: URL: https://github.com/apache/arrow-go/pull/344#issuecomment-2811034930 > Made an initial pass on this and left a bunch of comments. I'll try to take another look later on Thanks so much! I'll get to addressing the comments here shortly (was ou

Re: [I] Error casting strings with large dates to Timestamp [arrow-rs]

2025-04-16 Thread via GitHub
swanandx commented on issue #7208: URL: https://github.com/apache/arrow-rs/issues/7208#issuecomment-2811774150 facing same issue when using delta lake. Able to write the files and stats, but when parsing i get: `Error parsing timestamp from '+57216-08-06T00:59:13Z': error parsing dat

Re: [PR] GH-46167: [R][CI] Update Artifacts for R 4.5 in task.yml [arrow]

2025-04-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46168: URL: https://github.com/apache/arrow/pull/46168#issuecomment-2811401028 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit b80d62b531c768cf7a162a7676461a0c2d811e6a. There were no

Re: [I] object_store: Automatically fall back to use_azure_cli=True if no other credentials can be found [arrow-rs-object-store]

2025-04-16 Thread via GitHub
daviewales commented on issue #290: URL: https://github.com/apache/arrow-rs-object-store/issues/290#issuecomment-2811746255 Hmm... I guess the subsequent creation of the `TokenCredentialProvider` succeeds too? https://github.com/apache/arrow-rs-object-store/blob/3e12e1a229754fc7a2cb20ff

[PR] fix(go): Use arrow-go in templates instead of arrow/go [arrow-adbc]

2025-04-16 Thread via GitHub
felipecrv opened a new pull request, #2712: URL: https://github.com/apache/arrow-adbc/pull/2712 This makes the templates consistent with the generated (then manually edited code). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] rust: Let the returned RecordBatchReader outlive & parameters [arrow-adbc]

2025-04-16 Thread via GitHub
felipecrv commented on issue #2694: URL: https://github.com/apache/arrow-adbc/issues/2694#issuecomment-2811700735 > OK, sounds good to me. Though it sounds like Felipe's only concern was outliving the `str` parameters and not necessarily everything else, now that I look over everything agai

Re: [PR] Add benchmark for parquet reader with row_filter and project settings [arrow-rs]

2025-04-16 Thread via GitHub
zhuqi-lucas commented on PR #7401: URL: https://github.com/apache/arrow-rs/pull/7401#issuecomment-2811691474 Thank you @alamb @XiangpengHao for double checking! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] GH-44366: [Python][Acero] RecordBatch.filter on expression raises error if result set is empty [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #46057: URL: https://github.com/apache/arrow/pull/46057#issuecomment-2811039063 @github-actions crossbow submit -g python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] GH-463: Improve TZ support for JDBC driver [arrow-java]

2025-04-16 Thread via GitHub
lidavidm commented on code in PR #464: URL: https://github.com/apache/arrow-java/pull/464#discussion_r2048031976 ## flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/accessor/impl/calendar/ArrowFlightJdbcTimeStampVectorAccessor.java: ## @@ -102,7 +160,7 @@ p

[PR] Add user agent entry for spark http connections [arrow-adbc]

2025-04-16 Thread via GitHub
toddmeng-db opened a new pull request, #2711: URL: https://github.com/apache/arrow-adbc/pull/2711 Enables user agent entry to be passed in, such as "powerbi" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #45854: URL: https://github.com/apache/arrow/pull/45854#issuecomment-2811070577 The minimal job failures I think are from a regression introduced by https://github.com/apache/arrow/pull/46057#issuecomment-2811068182 -- This is an automated message from the Apache G

Re: [PR] GH-44366: [Python][Acero] RecordBatch.filter on expression raises error if result set is empty [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46057: URL: https://github.com/apache/arrow/pull/46057#issuecomment-2811046768 Revision: b5ef33038f5bbb6b848edf600a23072d9082c1b9 Submitted crossbow builds: [ursacomputing/crossbow @ actions-85876e4ee6](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45853: [C++][Dev] Fix Meson compilation issues in Docker builds [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #45858: URL: https://github.com/apache/arrow/pull/45858#issuecomment-2810505704 Revision: 8f646a868e5309f47e0e95f614b0bff8aa42ef41 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1b31b845a1](https://github.com/ursacomputing/crossbow/bra

Re: [PR] feat(csharp/src/Drivers/Apache): Add prefetch functionality to CloudFetch in Spark ADBC driver [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2678: URL: https://github.com/apache/arrow-adbc/pull/2678#discussion_r2047511703 ## csharp/src/Drivers/Databricks/DatabricksParameters.cs: ## @@ -42,6 +42,30 @@ public class DatabricksParameters : SparkParameters /// Default valu

Re: [PR] GH-46172: [C++] Add SIMD support in Meson configuration [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #46173: URL: https://github.com/apache/arrow/pull/46173#issuecomment-2810666762 Note that there are also SIMD modules to be used in the compute and acero directories, but those are not in main yet I noticed that the CSV module also uses some macros around SSE4

Re: [PR] GH-46172: [C++] Add SIMD support in Meson configuration [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46173: URL: https://github.com/apache/arrow/pull/46173#issuecomment-2810664375 :warning: GitHub issue #46172 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-46172: [C++] Add SIMD support in Meson configuration [arrow]

2025-04-16 Thread via GitHub
WillAyd opened a new pull request, #46173: URL: https://github.com/apache/arrow/pull/46173 ### Rationale for this change This gets the Meson configuration closer to feature parity with CMake ### What changes are included in this PR? Adds SIMD support to the top level arro

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #45854: URL: https://github.com/apache/arrow/pull/45854#issuecomment-2810562790 Revision: 3b907a666f9f8e40bdb44f798bf0ce1a3b03581c Submitted crossbow builds: [ursacomputing/crossbow @ actions-2b1639a26c](https://github.com/ursacomputing/crossbow/bra

Re: [I] [C++] FixedSizeListBuilder should have UnsafeAppend methods [arrow]

2025-04-16 Thread via GitHub
gauravrp commented on issue #45723: URL: https://github.com/apache/arrow/issues/45723#issuecomment-2810509734 Hey @cramosme Please feel free to pick this issue, and apologies for late replies. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #45854: URL: https://github.com/apache/arrow/pull/45854#issuecomment-2810557307 @github-actions crossbow submit -g python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
raulcd opened a new pull request, #46171: URL: https://github.com/apache/arrow/pull/46171 ### Rationale for this change The [debian-12-docs](https://github.com/ursacomputing/crossbow/actions/runs/14477216662/job/40605633697) are failing due to R 4.4 missing from Debian repositories a

Re: [PR] GH-46061: [Python] Replace setup.py invocations with modern commands [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #46062: URL: https://github.com/apache/arrow/pull/46062#issuecomment-2810479031 @github-actions crossbow submit wheel-windows-* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] GH-45853: [C++][Dev] Fix Meson compilation issues in Docker builds [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #45858: URL: https://github.com/apache/arrow/pull/45858#issuecomment-2810499226 @github-actions crossbow submit *meson -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-46061: [Python] Replace setup.py invocations with modern commands [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46062: URL: https://github.com/apache/arrow/pull/46062#issuecomment-2810500576 Revision: 032050ca2a8bf73ac08178820bfd01e8402cb48f Submitted crossbow builds: [ursacomputing/crossbow @ actions-2a2ee27448](https://github.com/ursacomputing/crossbow/bra

Re: [PR] Variant Support for Arrow and Parquet [DRAFT] [arrow-rs]

2025-04-16 Thread via GitHub
adriangb commented on code in PR #7404: URL: https://github.com/apache/arrow-rs/pull/7404#discussion_r2047543959 ## parquet/src/arrow/arrow_reader/mod.rs: ## @@ -4431,4 +4431,258 @@ mod tests { assert_eq!(c0.len(), c1.len()); c0.iter().zip(c1.iter()).for_each(|

Re: [PR] GH-39294: [C++][Python] DLPack on Tensor class [arrow]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #42118: URL: https://github.com/apache/arrow/pull/42118#discussion_r2047495330 ## cpp/src/arrow/c/dlpack.cc: ## @@ -130,4 +131,71 @@ Result ExportDevice(const std::shared_ptr& arr) { } } +struct TensorManagerCtx { + std::shared_ptr t; +

Re: [PR] GH-39294: [C++][Python] DLPack on Tensor class [arrow]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #42118: URL: https://github.com/apache/arrow/pull/42118#discussion_r2047491267 ## cpp/src/arrow/c/dlpack.cc: ## @@ -130,4 +131,71 @@ Result ExportDevice(const std::shared_ptr& arr) { } } +struct TensorManagerCtx { + std::shared_ptr t; +

Re: [PR] GH-39294: [C++][Python] DLPack on Tensor class [arrow]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #42118: URL: https://github.com/apache/arrow/pull/42118#discussion_r2047472486 ## cpp/src/arrow/c/dlpack.cc: ## @@ -66,15 +67,15 @@ struct ManagerCtx { } // namespace Result ExportArray(const std::shared_ptr& arr) { - // Define DLDevice st

Re: [PR] GH-46159: [CI][C++] Stop using possibly missing boost/process/v2.hpp on boost 1.88 and use individual includes [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46160: URL: https://github.com/apache/arrow/pull/46160#issuecomment-2810367902 Revision: 3ade5dd8c29e1ab43d72b7fbd1e68ca1941203f1 Submitted crossbow builds: [ursacomputing/crossbow @ actions-4e85a276e1](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-46159: [CI][C++] Stop using possibly missing boost/process/v2.hpp on boost 1.88 and use individual includes [arrow]

2025-04-16 Thread via GitHub
raulcd commented on PR #46160: URL: https://github.com/apache/arrow/pull/46160#issuecomment-2810362977 @github-actions crossbow submit verify-rc-source-integration-macos-amd64 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] [R][CI] r-binary-packages job is failing due to R 4.5 release [arrow]

2025-04-16 Thread via GitHub
assignUser commented on issue #46167: URL: https://github.com/apache/arrow/issues/46167#issuecomment-2810329373 Issue resolved by pull request 46168 https://github.com/apache/arrow/pull/46168 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] GH-46167: [R][CI] Update Artifacts for R 4.5 in task.yml [arrow]

2025-04-16 Thread via GitHub
assignUser merged PR #46168: URL: https://github.com/apache/arrow/pull/46168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.ap

[PR] Fix GCP signing token [arrow-rs-object-store]

2025-04-16 Thread via GitHub
jackm-mimica opened a new pull request, #338: URL: https://github.com/apache/arrow-rs-object-store/pull/338 * refresh token replaced by access token # Which issue does this PR close? https://github.com/apache/arrow-rs-object-store/issues/337 Closes #. # What change

Re: [I] Incorrect token sent as part of url signing function. [arrow-rs-object-store]

2025-04-16 Thread via GitHub
jackm-mimica commented on issue #337: URL: https://github.com/apache/arrow-rs-object-store/issues/337#issuecomment-2810272414 Fix here https://github.com/jackm-mimica/rs-object-store-arrow/commit/6ba38fa367f7143c213ae4b1153507add56bd8d7 -- This is an automated message from the Apache Git

Re: [PR] feat(go/adbc/driver/flightsql): Add OAuth Support to Flight Client [arrow-adbc]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #2651: URL: https://github.com/apache/arrow-adbc/pull/2651#discussion_r2047412891 ## go/adbc/driver/flightsql/flightsql_oauth.go: ## @@ -0,0 +1,149 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [I] Incorrect token sent as part of url signing function. [arrow-rs-object-store]

2025-04-16 Thread via GitHub
jackm-mimica commented on issue #337: URL: https://github.com/apache/arrow-rs-object-store/issues/337#issuecomment-2810246451 Confirmed this is only when using ADC and with a service account it functions correctly. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
raulcd commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810241704 test-r-arrow-backwards-compatibility failures are unrelated, they are due to: > This is a scheduled Ubuntu 20.04 retirement. Ubuntu 20.04 LTS runner will be removed on 2025-04-15

Re: [PR] Default wasm32-unknown-unknown HttpConnector [arrow-rs-object-store]

2025-04-16 Thread via GitHub
alamb commented on PR #329: URL: https://github.com/apache/arrow-rs-object-store/pull/329#issuecomment-2810235262 Looks like there are some CI failures: https://github.com/apache/arrow-rs-object-store/actions/runs/14495449031/job/40672770867?pr=329 -- This is an automated message from th

Re: [PR] GH-46167: [R][CI] Update Artifacts for R 4.5 in task.yml [arrow]

2025-04-16 Thread via GitHub
assignUser commented on PR #46168: URL: https://github.com/apache/arrow/pull/46168#issuecomment-2810045377 @github-actions crossbow submit r-binary-packages -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Default wasm32-unknown-unknown HttpConnector [arrow-rs-object-store]

2025-04-16 Thread via GitHub
alamb commented on PR #329: URL: https://github.com/apache/arrow-rs-object-store/pull/329#issuecomment-2810225288 kicked off the tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Default wasm32-unknown-unknown HttpConnector [arrow-rs-object-store]

2025-04-16 Thread via GitHub
alamb commented on code in PR #329: URL: https://github.com/apache/arrow-rs-object-store/pull/329#discussion_r2047391347 ## .github/workflows/ci.yml: ## @@ -192,6 +192,10 @@ jobs: run: rustup target add wasm32-wasip1 - name: Build wasm32-wasip1 run: car

Re: [PR] feat(csharp/src/Drivers/Databricks): Make Cloud Fetch options configurable at the connection level [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2691: URL: https://github.com/apache/arrow-adbc/pull/2691#discussion_r2047325192 ## csharp/src/Drivers/Databricks/DatabricksConnection.cs: ## @@ -29,10 +29,50 @@ namespace Apache.Arrow.Adbc.Drivers.Databricks { internal class Databr

Re: [PR] Skip page should also support skip dict page [arrow-rs]

2025-04-16 Thread via GitHub
alamb commented on PR #7409: URL: https://github.com/apache/arrow-rs/pull/7409#issuecomment-2810189170 Thank you @zhuqi-lucas and @tustvold and @XiangpengHao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Skip page should also support skip dict page [arrow-rs]

2025-04-16 Thread via GitHub
alamb merged PR #7409: URL: https://github.com/apache/arrow-rs/pull/7409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810188557 Revision: 48349203e121c5c19d56bc8bbb5c4673fafcc34a Submitted crossbow builds: [ursacomputing/crossbow @ actions-a82c5e1f51](https://github.com/ursacomputing/crossbow/bra

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-16 Thread via GitHub
raulcd commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2810183450 I am unsure on the macOS verification failures due to Gtest, I can try and investigate tomorrow. As per the debian-docs failure I've opened an issue and pushed a PR for it https://githu

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-16 Thread via GitHub
alamb commented on code in PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334#discussion_r2047362742 ## src/client/builder.rs: ## @@ -283,4 +293,38 @@ mod tests { "https://f...@example.com/bananas?foo=1&bingo=foo&auth=test&t1=funky+shenanigans&

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
raulcd commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810182607 @github-actions crossbow submit test-r-* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810181407 Revision: 48349203e121c5c19d56bc8bbb5c4673fafcc34a Submitted crossbow builds: [ursacomputing/crossbow @ actions-a61cebecff](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810179606 ``` Unable to match any tasks for `ubuntu-r-*` The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/14498249532 ``` -- This is an autom

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810176620 :warning: GitHub issue #46169 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-46169: [CI][R] Update R version to 4.5 due to 4.4 not being on APT repositories anymore [arrow]

2025-04-16 Thread via GitHub
raulcd commented on PR #46171: URL: https://github.com/apache/arrow/pull/46171#issuecomment-2810176198 @github-actions crossbow submit test-debian-12-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] GH-45531: [Python] Add the `dim_names` argument to `from_numpy_ndarray` [arrow]

2025-04-16 Thread via GitHub
yyossy5 commented on PR #46170: URL: https://github.com/apache/arrow/pull/46170#issuecomment-2810155991 Should the document be revised as well? https://arrow.apache.org/docs/python/generated/pyarrow.FixedShapeTensorArray.html#pyarrow.FixedShapeTensorArray.from_numpy_ndarray -- This

[PR] GH-45531: [Python] Add the `dim_names` argument to from_numpy_ndarray [arrow]

2025-04-16 Thread via GitHub
yyossy5 opened a new pull request, #46170: URL: https://github.com/apache/arrow/pull/46170 ### Rationale for this change The `FixedShapeTensorArray.from_numpy_ndarray` method did not pass `dim_names` to the `fixed_shape_tensor` constructor, which resulted in dimension names being los

Re: [PR] feat(csharp/src/Drivers/BigQuery): support evaluation kind and statement type setting [arrow-adbc]

2025-04-16 Thread via GitHub
davidhcoe commented on PR #2698: URL: https://github.com/apache/arrow-adbc/pull/2698#issuecomment-2810130958 > (Also, it would be nice if we tried to maintain some alignment between the C# BigQuery driver and the Go BigQuery driver -- though it may already be a little late for that :(. CC:

Re: [PR] feat(csharp/src/Drivers/Databricks): Make Cloud Fetch options configurable at the connection level [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2691: URL: https://github.com/apache/arrow-adbc/pull/2691#discussion_r2047276895 ## csharp/src/Drivers/Databricks/DatabricksConnection.cs: ## @@ -29,10 +29,50 @@ namespace Apache.Arrow.Adbc.Drivers.Databricks { internal class Databr

Re: [PR] GH-46167: [R][CI] Update Artifacts for R 4.5 in task.yml [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46168: URL: https://github.com/apache/arrow/pull/46168#issuecomment-2810051479 Revision: 2ddd8185a1f1902725a92811fb3e3c242067d464 Submitted crossbow builds: [ursacomputing/crossbow @ actions-71e72c038c](https://github.com/ursacomputing/crossbow/bra

Re: [PR] feat(csharp/src/Drivers/Databricks): Support server side property passthrough [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2692: URL: https://github.com/apache/arrow-adbc/pull/2692#discussion_r2047234125 ## csharp/src/Drivers/Databricks/DatabricksConnection.cs: ## @@ -86,9 +100,70 @@ protected override TOpenSessionReq CreateSessionRequest() C

[PR] GH-46167: [R][CI] Update Artifacts for R 4.5 in task.yml [arrow]

2025-04-16 Thread via GitHub
assignUser opened a new pull request, #46168: URL: https://github.com/apache/arrow/pull/46168 ### Rationale for this change R 4.5 was released, we use rel and old-rel for nightlies ### What changes are included in this PR? Update the artifacts ### Are these changes tested?

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
lemire commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2810028915 @pitrou See PR https://github.com/simdjson/simdjson/pull/2365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Azure Gen2 broken on latest [arrow-rs-object-store]

2025-04-16 Thread via GitHub
crepererum closed issue #320: Azure Gen2 broken on latest URL: https://github.com/apache/arrow-rs-object-store/issues/320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Fix query parameter signing in Azure [arrow-rs-object-store]

2025-04-16 Thread via GitHub
crepererum merged PR #334: URL: https://github.com/apache/arrow-rs-object-store/pull/334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] GH-46130: [Python] Remove `use_legacy_format` in favour of setting `IpcWriteOptions` [arrow]

2025-04-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46131: URL: https://github.com/apache/arrow/pull/46131#issuecomment-2810014928 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 586ed925f4bb4d333f8e2a0beb07564bade355e8. There were no

[I] Incorrect token sent as part of url signing function. [arrow-rs-object-store]

2025-04-16 Thread via GitHub
jackm-mimica opened a new issue, #337: URL: https://github.com/apache/arrow-rs-object-store/issues/337 **Describe the bug** Creating a signed url for a gcp blob throws the error ``` Generic GCS error: Error performing token request: Error performing GET https://oauth2.googleapis.com

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
lemire commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809987618 @pitrou Point taken. You are correct. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
pitrou commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809985297 > The simdjson library is used by major systems like the Node.js JavaScript runtime, [ClickHouse](https://github.com/ClickHouse/ClickHouse), [StarRocks](https://github.com/StarRocks/s

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
lemire commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809976587 @pitrou > it doesn't reduce the current platform compatibility (...) we currently need to be compatible with 32-bit systems. It's not obvious that simdjson allows that.

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-16 Thread via GitHub
assignUser commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2809941918 Hm, it's just the nightly package binaries... we should probably make them optional for releases or something? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] feat(csharp): Add retry-after behavior for 503 responses in Spark ADBC driver [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2664: URL: https://github.com/apache/arrow-adbc/pull/2664#discussion_r2047128798 ## csharp/src/Drivers/Databricks/DatabricksConnection.cs: ## @@ -89,6 +110,35 @@ protected override TOpenSessionReq CreateSessionRequest() retur

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-16 Thread via GitHub
assignUser commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2809903253 > We may need the following for the r-binary-packages failure: Ah, how unlucky, yes R 4.5 was just released. I'll have a look -- This is an automated message from the Apache G

Re: [PR] Variant Support for Arrow and Parquet [DRAFT] [arrow-rs]

2025-04-16 Thread via GitHub
thinkharderdev commented on code in PR #7404: URL: https://github.com/apache/arrow-rs/pull/7404#discussion_r2047155023 ## arrow-array/src/array/variant_array.rs: ## @@ -0,0 +1,628 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license a

Re: [PR] feat(go/adbc/driver/flightsql): Add OAuth Support to Flight Client [arrow-adbc]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #2651: URL: https://github.com/apache/arrow-adbc/pull/2651#discussion_r2047149164 ## go/adbc/driver/flightsql/flightsql_database.go: ## @@ -146,10 +147,12 @@ func (d *databaseImpl) SetOptions(cnOptions map[string]string) error {

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
pitrou commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809866592 And, by the way, we currently need to be compatible with 32-bit systems. It's not obvious that simdjson allows that. -- This is an automated message from the Apache Git Service. To

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
pitrou commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809862168 [This part](https://github.com/simdjson/simdjson/blob/master/doc/performance.md#free-padding), however, will require additional care: > For performance reasons, the simdjson librar

Re: [PR] Skip page should also support skip dict page [arrow-rs]

2025-04-16 Thread via GitHub
XiangpengHao commented on PR #7409: URL: https://github.com/apache/arrow-rs/pull/7409#issuecomment-2809848164 This is a nice catch, thank you @zhuqi-lucas -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] [C++] can we use simdjson to replace rapidjson [arrow]

2025-04-16 Thread via GitHub
pitrou commented on issue #35460: URL: https://github.com/apache/arrow/issues/35460#issuecomment-2809849645 I agree we need to do something about our RapidJSON dependency. simdjson is a reasonable contender. We should just have to check it doesn't reduce the current platform compatibility (

Re: [PR] Default wasm32-unknown-unknown HttpConnector [arrow-rs-object-store]

2025-04-16 Thread via GitHub
H-Plus-Time commented on PR #329: URL: https://github.com/apache/arrow-rs-object-store/pull/329#issuecomment-2809839782 That should do it I think - that one test covers a sizable chunk of the wasm-specific code; put/post requests would be the only other aspect worth testing I think. --

Re: [PR] Add benchmark for parquet reader with row_filter and project settings [arrow-rs]

2025-04-16 Thread via GitHub
XiangpengHao commented on PR #7401: URL: https://github.com/apache/arrow-rs/pull/7401#issuecomment-2809810111 This is great work, thank you @zhuqi-lucas > And still no performance improvement compare the page cache PR to main branch. I am confused why datafusion benchmark will be imp

Re: [PR] feat(csharp/src/Drivers/BigQuery): support evaluation kind and statement type setting [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2698: URL: https://github.com/apache/arrow-adbc/pull/2698#discussion_r2047088929 ## csharp/test/Drivers/BigQuery/DriverTests.cs: ## @@ -346,5 +346,43 @@ public void QueryTimeoutTest() } } } + +

Re: [PR] feat(csharp/src/Drivers/BigQuery): support evaluation kind and statement type setting [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on code in PR #2698: URL: https://github.com/apache/arrow-adbc/pull/2698#discussion_r2047082128 ## csharp/src/Drivers/BigQuery/BigQueryStatement.cs: ## @@ -51,52 +51,60 @@ public BigQueryStatement(BigQueryClient client, GoogleCredential credential)

Re: [PR] feat(csharp/src/Drivers/BigQuery): support evaluation kind and statement type setting [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on PR #2698: URL: https://github.com/apache/arrow-adbc/pull/2698#issuecomment-2809790484 (Also, it would be nice if we tried to maintain some alignment between the C# BigQuery driver and the Go BigQuery driver -- though it may already be a little late for that :(.

Re: [PR] feat(csharp/src/Drivers/BigQuery): support evaluation kind and statement type setting [arrow-adbc]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on PR #2698: URL: https://github.com/apache/arrow-adbc/pull/2698#issuecomment-2809785069 I feel like I don't entirely understand this change, so it would be nice to get a little more explanation. Today, there's a limitation in ADBC which prevents a single exe

Re: [PR] WIP: [Release] Verify release-20.0.0-rc1 [arrow]

2025-04-16 Thread via GitHub
kou commented on PR #46152: URL: https://github.com/apache/arrow/pull/46152#issuecomment-2809772625 @assignUser @raulcd Could you take a look at the r-binary-packages failure https://github.com/ursacomputing/crossbow/actions/runs/14477217372/job/40614786493#step:10:63 ```tet

Re: [PR] GH-46157: [C++] Disable util test when ARROW_COMPUTE=OFF [arrow]

2025-04-16 Thread via GitHub
WillAyd closed pull request #46162: GH-46157: [C++] Disable util test when ARROW_COMPUTE=OFF URL: https://github.com/apache/arrow/pull/46162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] GH-46157: [C++] Disable util test when ARROW_COMPUTE=OFF [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on code in PR #46162: URL: https://github.com/apache/arrow/pull/46162#discussion_r2046983251 ## cpp/src/arrow/testing/gtest_util_test.cc: ## @@ -292,6 +292,7 @@ TEST(AssertTestWithinUlp, Basics) { EXPECT_FATAL_FAILURE(AssertWithinUlp(123.456f, 123.456085f, 1

Re: [PR] GH-46165: [C++] Add cuda option to Meson configuration [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46166: URL: https://github.com/apache/arrow/pull/46166#issuecomment-2809647571 :warning: GitHub issue #46165 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-46165: [C++] Add cuda option to Meson configuration [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #46166: URL: https://github.com/apache/arrow/pull/46166#issuecomment-2809649089 I am unable to fully test this locally as I do not have a CUDA-enabled device. I can configure the project and compile, but at runtime I get errors like: ```sh /usr/bin/ld: src/

[PR] GH-46165: [C++] Add cuda option to Meson configuration [arrow]

2025-04-16 Thread via GitHub
WillAyd opened a new pull request, #46166: URL: https://github.com/apache/arrow/pull/46166 ### Rationale for this change This continues building out support for Meson as a build system gnerator ### What changes are included in this PR? Adds a cuda option that pulls in the

Re: [PR] feat(go/adbc/driver/flightsql): Add OAuth Support to Flight Client [arrow-adbc]

2025-04-16 Thread via GitHub
zeroshade commented on code in PR #2651: URL: https://github.com/apache/arrow-adbc/pull/2651#discussion_r2046977641 ## docs/source/driver/flight_sql.rst: ## @@ -159,6 +159,12 @@ few optional authentication schemes: header will then be sent back as the ``authorization`` header

Re: [PR] GH-46157: [C++] Disable util test when ARROW_COMPUTE=OFF [arrow]

2025-04-16 Thread via GitHub
raulcd commented on code in PR #46162: URL: https://github.com/apache/arrow/pull/46162#discussion_r2046952253 ## cpp/src/arrow/testing/gtest_util_test.cc: ## @@ -292,6 +292,7 @@ TEST(AssertTestWithinUlp, Basics) { EXPECT_FATAL_FAILURE(AssertWithinUlp(123.456f, 123.456085f, 10

Re: [I] Serialized polars DataFrame cannot be processed properly [arrow-nanoarrow]

2025-04-16 Thread via GitHub
paleolimbot commented on issue #743: URL: https://github.com/apache/arrow-nanoarrow/issues/743#issuecomment-2809586694 I believe this is because of flatbuffers, which we had to fix to get arrow-rs to work with nanoarrow! Upgrading flatbuffers to 24.12.23 and regenerating the generated code

Re: [PR] GH-46163: [C++] Add vendored directory to Meson [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46164: URL: https://github.com/apache/arrow/pull/46164#issuecomment-2809520604 :warning: GitHub issue #46163 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-46163: [C++] Add vendored directory to Meson [arrow]

2025-04-16 Thread via GitHub
WillAyd opened a new pull request, #46164: URL: https://github.com/apache/arrow/pull/46164 ### Rationale for this change This continues adding support for Meson as a build system generator ### What changes are included in this PR? This adds the vendored directory to the M

Re: [PR] Handle compressed empty DataPage v2 [arrow-rs]

2025-04-16 Thread via GitHub
EnricoMi commented on PR #7389: URL: https://github.com/apache/arrow-rs/pull/7389#issuecomment-2809509911 Waiting for https://github.com/apache/parquet-testing/pull/74 to be merged. Will then point parquet-testing sub-module back to main branch. -- This is an automated message from the Ap

Re: [PR] GH-45833: [C++] Add JSON directory to Meson configuration [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #45834: URL: https://github.com/apache/arrow/pull/45834#issuecomment-2809495742 Revision: c77c6d4e52dcf7e58eae41e898cebf37efc8a9fa Submitted crossbow builds: [ursacomputing/crossbow @ actions-c92d9bf32b](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45833: [C++] Add JSON directory to Meson configuration [arrow]

2025-04-16 Thread via GitHub
WillAyd commented on PR #45834: URL: https://github.com/apache/arrow/pull/45834#issuecomment-2809490206 @github-actions crossbow submit *meson -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH46157: [C++] Disable util test when ARROW_COMPUTE=OFF [arrow]

2025-04-16 Thread via GitHub
github-actions[bot] commented on PR #46162: URL: https://github.com/apache/arrow/pull/46162#issuecomment-2809463635 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue f

[PR] GH46157: [C++] Disable util test when ARROW_COMPUTE=OFF [arrow]

2025-04-16 Thread via GitHub
WillAyd opened a new pull request, #46162: URL: https://github.com/apache/arrow/pull/46162 ### Rationale for this change The test suite currently fails when ARROW_COMPUTE is disabled ### What changes are included in this PR? This disables the particular util test when ARR

Re: [I] [C#] BitUtility.cs performance improvement [arrow]

2025-04-16 Thread via GitHub
CurtHagenlocher commented on issue #45559: URL: https://github.com/apache/arrow/issues/45559#issuecomment-2809459697 Are you @alexdegroot, are you asking about the existing PR which doesn't appear to have any performance improvements or about the proposed alternative which seems to have a ~

  1   2   >