Re: [PR] GH-45185: [C++][Parquet] Raise an error for invalid repetition levels when delimiting records [arrow]

2025-03-27 Thread via GitHub
mapleFU commented on PR #45186: URL: https://github.com/apache/arrow/pull/45186#issuecomment-2760365645 Oh my I found this is not merged...I'll rebase and try to merge this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[I] object_store: Uploading empty file to S3 results in "411 Length Required" [arrow-rs-object-store]

2025-03-27 Thread via GitHub
Turbo87 opened a new issue, #155: URL: https://github.com/apache/arrow-rs-object-store/issues/155 **Describe the bug** When using the S3 implementation of `object_store` a `put()` call with an empty content fails on the server side with a "411 Length Required" error: > Gene

Re: [I] object store: retry / recover after partially reading a streaming response ( fix timeout errors / `error decoding response body` ) [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #7242: URL: https://github.com/apache/arrow-rs/issues/7242#issuecomment-2738319028 Copying some discussion with @ryzhyk from https://github.com/apache/arrow-rs/issues/7251#issuecomment-2735415879: In terms of rety the idea is that the rety doesn't retry the *

Re: [PR] test(c/integration/duckdb): update DuckDB integration test [arrow-adbc]

2025-03-27 Thread via GitHub
lidavidm merged PR #2654: URL: https://github.com/apache/arrow-adbc/pull/2654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [I] [C++] Reduce number of internal APIs that define default_memory_pool() as default argument value [arrow]

2025-03-27 Thread via GitHub
kapoisu commented on issue #36360: URL: https://github.com/apache/arrow/issues/36360#issuecomment-2760325161 Does this issue still require some effort? Is arrow/ipc/metadata_internal.h a sample that can be changed? I've checked the codebase and most other occurrences are within public

[PR] GH-45961: [Release][Docs] Upload generated docs to GitHub Releases not apache.jfrog.io [arrow]

2025-03-27 Thread via GitHub
kou opened a new pull request, #45963: URL: https://github.com/apache/arrow/pull/45963 This is based on #45962 . We should merge this after #45962 . ### Rationale for this change We want to stop using apache.jfrog.io. See also: #40760 ### What changes are included in this

Re: [PR] GH-45961: [Release][Docs] Upload generated docs to GitHub Releases not apache.jfrog.io [arrow]

2025-03-27 Thread via GitHub
github-actions[bot] commented on PR #45963: URL: https://github.com/apache/arrow/pull/45963#issuecomment-2760317133 :warning: GitHub issue #45961 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] S3 endpoint and trailing slash result in weird/invalid requests [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #6580: URL: https://github.com/apache/arrow-rs/issues/6580#issuecomment-2741680838 Migrating from arrow-rs issue [#6580](https://github.com/apache/arrow-rs/issues/6580) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Tweaks: homepage and fix RAT [arrow-rs-object-store]

2025-03-27 Thread via GitHub
alamb commented on code in PR #7: URL: https://github.com/apache/arrow-rs-object-store/pull/7#discussion_r2006340822 ## dev/release/check-rat-report.py: ## @@ -0,0 +1,59 @@ +#!/usr/bin/python Review Comment: I found this by looking at the license check on main: https://

Re: [I] [C++][Compute][Acero] Poor aggregate performance when there is a large number of batches on the build side [arrow]

2025-03-27 Thread via GitHub
zanmato1984 commented on issue #45847: URL: https://github.com/apache/arrow/issues/45847#issuecomment-2742563375 Yes, this matches what I've been suspecting. BTW, what value of `kNumRowsPerScanTask` were you using for this recent experiment? -- This is an automated message from the

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017865043 ## python/pyarrow/_parquet.pyx: ## @@ -319,6 +319,84 @@ cdef _box_flba(ParquetFLBA val, uint32_t len): return cp.PyBytes_FromStringAndSize( val.ptr, len)

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017865379 ## python/pyarrow/_parquet.pyx: ## @@ -319,6 +319,84 @@ cdef _box_flba(ParquetFLBA val, uint32_t len): return cp.PyBytes_FromStringAndSize( val.ptr, len)

[I] socks5 proxy support for the object_store crate [arrow-rs-object-store]

2025-03-27 Thread via GitHub
sum12 opened a new issue, #218: URL: https://github.com/apache/arrow-rs-object-store/issues/218 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** object_store crate cannot be used when the application is supposed to be runni

Re: [PR] GH-45920: [Release][Python] Upload sdist and wheels to GitHub Releases not apache.jfrog.io [arrow]

2025-03-27 Thread via GitHub
github-actions[bot] commented on PR #45962: URL: https://github.com/apache/arrow/pull/45962#issuecomment-2760187365 :warning: GitHub issue #45920 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-45920: [Release][Python] Upload sdist and wheels to GitHub Releases not apache.jfrog.io [arrow]

2025-03-27 Thread via GitHub
kou opened a new pull request, #45962: URL: https://github.com/apache/arrow/pull/45962 ### Rationale for this change We want to stop using apache.jfrog.io. See also: #40760 ### What changes are included in this PR? Use GitHub Release instead of apache.jfrog.io. Use

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017866549 ## cpp/src/parquet/test_util.h: ## @@ -39,6 +39,7 @@ #include "parquet/column_reader.h" #include "parquet/column_writer.h" #include "parquet/encoding.h" +#includ

[PR] test(c/integration/duckdb): update DuckDB integration test [arrow-adbc]

2025-03-27 Thread via GitHub
lidavidm opened a new pull request, #2654: URL: https://github.com/apache/arrow-adbc/pull/2654 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017867021 ## cpp/src/parquet/geospatial_util_json_internal.h: ## @@ -0,0 +1,46 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licen

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017867509 ## cpp/src/parquet/geospatial_statistics_test.cc: ## @@ -0,0 +1,317 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017867141 ## cpp/src/parquet/geospatial_util_json_internal.h: ## @@ -0,0 +1,46 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licen

Re: [PR] GH-40760: [Release] Use repository.apache.org [arrow]

2025-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45903: URL: https://github.com/apache/arrow/pull/45903#issuecomment-2760107629 After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit cc10aa5da9738144d0c591ebca1fc2ac8ec6679b. None of the s

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017866390 ## cpp/src/parquet/types.cc: ## @@ -1619,6 +1658,209 @@ class LogicalType::Impl::Float16 final : public LogicalType::Impl::Incompatible, GENERATE_MAKE(Float16)

Re: [PR] GH-45930: [C++] Don't use ICU C++ API in Azure SDK C++ [arrow]

2025-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45952: URL: https://github.com/apache/arrow/pull/45952#issuecomment-2760107613 After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit f4b7a4ee0a52777f4a696ad3c8d6f62f48cb4867. There were no

Re: [PR] fix(c/driver/postgresql): avoid crash if closing invalidated result [arrow-adbc]

2025-03-27 Thread via GitHub
lidavidm merged PR #2653: URL: https://github.com/apache/arrow-adbc/pull/2653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017860210 ## python/pyarrow/_parquet.pyx: ## @@ -319,6 +319,84 @@ cdef _box_flba(ParquetFLBA val, uint32_t len): return cp.PyBytes_FromStringAndSize( val.ptr, len)

Re: [PR] GH-45893: [Ruby] Unify test for map array in raw_records and each_raw_record [arrow]

2025-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45955: URL: https://github.com/apache/arrow/pull/45955#issuecomment-2760096396 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 2f4f68aed64b99a01cada6070b69cadc256f55da. There were no

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017853950 ## cpp/src/parquet/CMakeLists.txt: ## @@ -171,6 +171,9 @@ set(PARQUET_SRCS exception.cc file_reader.cc file_writer.cc +geospatial_statistics.cc

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017854863 ## cpp/src/parquet/api/reader.h: ## @@ -22,6 +22,7 @@ #include "parquet/column_scanner.h" #include "parquet/exception.h" #include "parquet/file_reader.h" +#inclu

Re: [I] [R] CRAN packaging checklist for version 19.0.1.1 [arrow]

2025-03-27 Thread via GitHub
jonkeane commented on issue #45950: URL: https://github.com/apache/arrow/issues/45950#issuecomment-2760054160 Commits to pull: * https://github.com/apache/arrow/commit/629e05fb75a07bbb4f9d5d04c3797f15b07291dd * the merge of #45958 * the merge of #45951 -- This is an automated

Re: [PR] GH-40760: [Release] Use repository.apache.org [arrow]

2025-03-27 Thread via GitHub
kou merged PR #45903: URL: https://github.com/apache/arrow/pull/45903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [I] [Release] Provide more ways to publish Binary Artifacts [arrow]

2025-03-27 Thread via GitHub
kou commented on issue #40760: URL: https://github.com/apache/arrow/issues/40760#issuecomment-2760057185 Issue resolved by pull request 45903 https://github.com/apache/arrow/pull/45903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-40760: [Release] Use repository.apache.org [arrow]

2025-03-27 Thread via GitHub
kou commented on PR #45903: URL: https://github.com/apache/arrow/pull/45903#issuecomment-2760055667 I'll merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [I] use GCE_METADATA_HOST and GCE_METADATA_IP env vars in object_store GCP auth [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #6014: URL: https://github.com/apache/arrow-rs/issues/6014#issuecomment-2741681413 Migrating from arrow-rs issue [#6014](https://github.com/apache/arrow-rs/issues/6014) -- This is an automated message from the Apache Git Service. To respond to the message, please

[I] Release object_store `0.12.1` (non breaking) Around April 30 2025 [arrow-rs-object-store]

2025-03-27 Thread via GitHub
alamb opened a new issue, #287: URL: https://github.com/apache/arrow-rs-object-store/issues/287 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Previous release: - https://github.com/apache/arrow-rs-object-store/issues/2

Re: [I] Feature Request: New ingest mode where the bound Arrow array or stream is registered as a view [arrow-adbc]

2025-03-27 Thread via GitHub
ahawker commented on issue #2617: URL: https://github.com/apache/arrow-adbc/issues/2617#issuecomment-2759774303 I did some digging into this before and I believe https://github.com/duckdb/duckdb/blob/v1.2.1/src/common/adbc/adbc.cpp#L793 is what's stopping DuckDB from directly querying arrow

Re: [PR] GH-45959: [C++][CMake] Fix Protobuf dependency in Arrow::arrow_static [arrow]

2025-03-27 Thread via GitHub
github-actions[bot] commented on PR #45960: URL: https://github.com/apache/arrow/pull/45960#issuecomment-2759970857 Revision: e77845974c06a694be84689671c4f2f2a5b64592 Submitted crossbow builds: [ursacomputing/crossbow @ actions-17b26eac16](https://github.com/ursacomputing/crossbow/bra

Re: [PR] fix(c/driver/postgresql): avoid crash if closing invalidated result [arrow-adbc]

2025-03-27 Thread via GitHub
lidavidm commented on code in PR #2653: URL: https://github.com/apache/arrow-adbc/pull/2653#discussion_r2017820924 ## c/driver/postgresql/statement.cc: ## @@ -219,55 +220,84 @@ void TupleReader::Release() { row_id_ = -1; } +struct ExportedTupleReader { + std::weak_ptr sel

Re: [I] Support for AWS Requester Pays buckets [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #6716: URL: https://github.com/apache/arrow-rs/issues/6716#issuecomment-2741680694 Migrating from arrow-rs issue [#6716](https://github.com/apache/arrow-rs/issues/6716) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [EXP] GH-44792: [C++] Require C++20 [arrow]

2025-03-27 Thread via GitHub
github-actions[bot] commented on PR #45445: URL: https://github.com/apache/arrow/pull/45445#issuecomment-2735773869 Revision: 680bf7b3ab9963b9867b37a170187699fbd88e7c Submitted crossbow builds: [ursacomputing/crossbow @ actions-f0fd97206f](https://github.com/ursacomputing/crossbow/bra

[I] Release `object_store` `0.5.2` (next release after`0.5.1`) [arrow-rs-object-store]

2025-03-27 Thread via GitHub
alamb opened a new issue, #212: URL: https://github.com/apache/arrow-rs-object-store/issues/212 Follow on from https://github.com/apache/arrow-rs/issues/2845 * Planned Release Candidate: 2022-12-02 * Planned Release and Publish to crates.io: 2022-12-5 Items: - [x] Update c

Re: [PR] GH-45959: [C++][CMake] Fix Protobuf dependency in Arrow::arrow_static [arrow]

2025-03-27 Thread via GitHub
kou commented on PR #45960: URL: https://github.com/apache/arrow/pull/45960#issuecomment-2759965897 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] GH-45959: [C++][CMake] Fix Protobuf dependency in Arrow::arrow_static [arrow]

2025-03-27 Thread via GitHub
kou commented on PR #45960: URL: https://github.com/apache/arrow/pull/45960#issuecomment-2759966113 @anku94 Could you try this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] GH-45959: [C++][CMake] Fix Protobuf dependency in Arrow::arrow_static [arrow]

2025-03-27 Thread via GitHub
kou opened a new pull request, #45960: URL: https://github.com/apache/arrow/pull/45960 ### Rationale for this change If Protobuf is used only for Flight, Arrow::arrow_static must not depend on Protobuf. Instead, ArrowFlight::arrow_flight_static must depend on Protobuf. ### What

Re: [PR] GH-45959: [C++][CMake] Fix Protobuf dependency in Arrow::arrow_static [arrow]

2025-03-27 Thread via GitHub
github-actions[bot] commented on PR #45960: URL: https://github.com/apache/arrow/pull/45960#issuecomment-2759966073 :warning: GitHub issue #45959 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-40754: [Python] Expose tls_ca_file_path to S3FileSystem [arrow]

2025-03-27 Thread via GitHub
bw513 opened a new pull request, #45881: URL: https://github.com/apache/arrow/pull/45881 ### Rationale for this change Currently, when using the pyarrow.fs.S3FileSystem, it is not immediately obvious how provide an alternative TLS certificate authority when working with (for example)

Re: [PR] MINOR: Bump error_prone_core.version from 2.31.0 to 2.37.0 [arrow-java]

2025-03-27 Thread via GitHub
lidavidm merged PR #682: URL: https://github.com/apache/arrow-java/pull/682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [I] IMDSv1 Fallback for S3 [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #2609: URL: https://github.com/apache/arrow-rs/issues/2609#issuecomment-2741686847 Migrating from arrow-rs issue [#2609](https://github.com/apache/arrow-rs/issues/2609) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Parquet: Support reading Parquet metadata via suffix range requests [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7334: URL: https://github.com/apache/arrow-rs/pull/7334#issuecomment-2759871673 I hope to review this tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] IP clearance for object_store_rs donation [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #2096: URL: https://github.com/apache/arrow-rs/issues/2096#issuecomment-2741687586 Migrating from arrow-rs issue [#2096](https://github.com/apache/arrow-rs/issues/2096) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Relax `WriteMultipart` API to support aborting after completion [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #5977: URL: https://github.com/apache/arrow-rs/issues/5977#issuecomment-2741681439 Migrating from arrow-rs issue [#5977](https://github.com/apache/arrow-rs/issues/5977) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2008350733 ## cpp/src/parquet/geospatial_util_internal.cc: ## @@ -0,0 +1,241 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Add configurable download method to object_store for enhanced usability [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #6837: URL: https://github.com/apache/arrow-rs/pull/6837#issuecomment-2741515932 Thank you for this PR. We are in the process of moving the object_store code to its [own repository](https://github.com/apache/arrow-rs-object-store). Would it be possible for you to creat

Re: [PR] Remove default implementation for `AsyncFileReader::get_metadata_with_options` [arrow-rs]

2025-03-27 Thread via GitHub
corwinjoy commented on PR #7342: URL: https://github.com/apache/arrow-rs/pull/7342#issuecomment-2759782227 Looking through how this function is used and implemented, many times users just want to grab metadata and don't care about options. Furthermore, right now we have a lot of code duplic

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017707076 ## cpp/src/parquet/schema.cc: ## @@ -28,6 +28,7 @@ #include "parquet/exception.h" #include "parquet/schema_internal.h" #include "parquet/thrift_internal.h" +#include

Re: [PR] fix(c/driver/postgresql): avoid crash if closing invalidated result [arrow-adbc]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #2653: URL: https://github.com/apache/arrow-adbc/pull/2653#discussion_r2017584088 ## c/driver/postgresql/statement.cc: ## @@ -219,55 +220,84 @@ void TupleReader::Release() { row_id_ = -1; } +struct ExportedTupleReader { + std::weak_ptr

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017639249 ## cpp/src/parquet/schema.cc: ## @@ -25,6 +25,7 @@ #include #include "arrow/util/logging.h" +#include "arrow/util/windows_fixup.h" // for OPTIONAL Review Comment:

Re: [PR] Add support for file row numbers in Parquet readers [arrow-rs]

2025-03-27 Thread via GitHub
etseidl commented on code in PR #7307: URL: https://github.com/apache/arrow-rs/pull/7307#discussion_r2017697049 ## parquet/src/arrow/array_reader/row_number.rs: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [I] [C++] StructBuilder should have UnsafeAppend methods [arrow]

2025-03-27 Thread via GitHub
virajmavani commented on issue #45722: URL: https://github.com/apache/arrow/issues/45722#issuecomment-2759754454 Would like to pick this up! Can you please share some more reference on how to find candidate call sites for the unsafe operations? -- This is an automated message from the Apa

Re: [I] object_store(aws): percent encoding test fails with latest localstack [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #3379: URL: https://github.com/apache/arrow-rs/issues/3379#issuecomment-2741685930 Migrating from arrow-rs issue [#3379](https://github.com/apache/arrow-rs/issues/3379) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Add Parquet Modular encryption support (write) [arrow-rs]

2025-03-27 Thread via GitHub
adamreeve commented on code in PR #7111: URL: https://github.com/apache/arrow-rs/pull/7111#discussion_r2017690250 ## parquet/src/file/metadata/writer.rs: ## @@ -133,17 +190,53 @@ impl<'a, W: Write> ThriftMetadataWriter<'a, W> { // Write file metadata let star

Re: [I] [Ruby] Unify test for map array in raw_records and each_raw_record [arrow]

2025-03-27 Thread via GitHub
kou commented on issue #45893: URL: https://github.com/apache/arrow/issues/45893#issuecomment-2759439270 Issue resolved by pull request 45955 https://github.com/apache/arrow/pull/45955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017652290 ## cpp/src/parquet/chunker_internal.cc: ## @@ -0,0 +1,319 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [I] Possible integer overflow [arrow-nanoarrow]

2025-03-27 Thread via GitHub
paleolimbot commented on issue #735: URL: https://github.com/apache/arrow-nanoarrow/issues/735#issuecomment-2759430736 Agreed! I don't think the fix for any of these will affect performance and it's worth doing to guarantee a clean compile when we're vendored. Can you share what stat

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017651577 ## cpp/src/parquet/column_chunker.cc: ## @@ -0,0 +1,778 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

Re: [I] Support the AES-GCM-CTR-V1 encryption algorithm for Parquet modular encryption [arrow-rs]

2025-03-27 Thread via GitHub
adamreeve commented on issue #7258: URL: https://github.com/apache/arrow-rs/issues/7258#issuecomment-2759562283 Ring is now in "security maintenance only" mode under new maintainers, so contributing plain CTR might not be possible, see https://rustsec.org/advisories/RUSTSEC-2025-0007.html a

Re: [PR] GH-25025: [C++] Move non core compute kernels into separate shared library [arrow]

2025-03-27 Thread via GitHub
kou commented on code in PR #45618: URL: https://github.com/apache/arrow/pull/45618#discussion_r2017649541 ## cpp/src/arrow/compute/kernels/api.h: ## @@ -0,0 +1,29 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017639249 ## cpp/src/parquet/schema.cc: ## @@ -25,6 +25,7 @@ #include #include "arrow/util/logging.h" +#include "arrow/util/windows_fixup.h" // for OPTIONAL Review Comment:

Re: [I] Support User-Defined Object Metadata [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #4754: URL: https://github.com/apache/arrow-rs/issues/4754#issuecomment-2741683710 Migrating from arrow-rs issue [#4754](https://github.com/apache/arrow-rs/issues/4754) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-03-27 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2017639249 ## cpp/src/parquet/schema.cc: ## @@ -25,6 +25,7 @@ #include #include "arrow/util/logging.h" +#include "arrow/util/windows_fixup.h" // for OPTIONAL Review Comment:

Re: [I] Proposal: `parquet 53.0.0` feature branch [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on issue #6050: URL: https://github.com/apache/arrow-rs/issues/6050#issuecomment-2741681353 Migrating from arrow-rs issue [#6050](https://github.com/apache/arrow-rs/issues/6050) -- This is an automated message from the Apache Git Service. To respond to the message, please

[I] Handle Body Errors in AWS CompleteMultipartUpload [arrow-rs-object-store]

2025-03-27 Thread via GitHub
tustvold opened a new issue, #122: URL: https://github.com/apache/arrow-rs-object-store/issues/122 **Describe the bug** CompleteMultipartUpload has an _interesting_ approach to error handling > Processing of a Complete Multipart Upload request could take several minutes to

Re: [PR] Add Parquet Modular encryption support (write) [arrow-rs]

2025-03-27 Thread via GitHub
adamreeve commented on PR #7111: URL: https://github.com/apache/arrow-rs/pull/7111#issuecomment-2759518123 Yes I think this is ready for review. There are a couple of comments I'd like some input on (https://github.com/apache/arrow-rs/pull/7111/files#r2015196618 and https://github.com/apac

Re: [PR] GH-45848: [C++][Python][R] Remove deprecated PARQUET_2_0 [arrow]

2025-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #45849: URL: https://github.com/apache/arrow/pull/45849#issuecomment-2759517690 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 7c18001f0d7bd97471237719702c33165858bba7. There were no

Re: [PR] GH-43573: [C++] Copy bitmap when casting from string-view to offset string and binary types [arrow]

2025-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #44822: URL: https://github.com/apache/arrow/pull/44822#issuecomment-2759510561 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit a42edc05b707251d648c6a1ceb5406434f4b8287. There were no

Re: [PR] GH-45930: [C++] Don't use ICU C++ API in Azure SDK C++ [arrow]

2025-03-27 Thread via GitHub
kou merged PR #45952: URL: https://github.com/apache/arrow/pull/45952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [I] [CI][Packaging] debian-trixie packaging jobs fail to build Azure SDK [arrow]

2025-03-27 Thread via GitHub
kou commented on issue #45930: URL: https://github.com/apache/arrow/issues/45930#issuecomment-2759487570 Issue resolved by pull request 45952 https://github.com/apache/arrow/pull/45952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-45893: [Ruby] Unify test for map array in raw_records and each_raw_record [arrow]

2025-03-27 Thread via GitHub
kou merged PR #45955: URL: https://github.com/apache/arrow/pull/45955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] Remove default implementation for `AsyncFileReader::get_metadata_with_options` [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7342: URL: https://github.com/apache/arrow-rs/pull/7342#issuecomment-2759437985 > If we don't have both we could keep the get_metadata name but change the signature to I think changing the signature would also be a good choice -- This is an automated message

Re: [PR] Encapsulate encryption code more in readers [arrow-rs]

2025-03-27 Thread via GitHub
alamb merged PR #7337: URL: https://github.com/apache/arrow-rs/pull/7337 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] Encapsulate encryption code more in readers [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7337: URL: https://github.com/apache/arrow-rs/pull/7337#issuecomment-2759433064 Thank you for the review @etseidl and @adamreeve -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Add Parquet Modular encryption support (write) [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7111: URL: https://github.com/apache/arrow-rs/pull/7111#issuecomment-2759435510 How is this PR doing, btw? Is it ready for another review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add Parquet Modular encryption support (write) [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7111: URL: https://github.com/apache/arrow-rs/pull/7111#issuecomment-2759435126 > But that might be quite a large and painful breaking change, and also might come with a performance hit... We would have to try it I suspect to test -- This is an automated mess

Re: [PR] Upgrade to twox-hash 2.0 [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #6583: URL: https://github.com/apache/arrow-rs/pull/6583#issuecomment-2759428060 Closed in https://github.com/apache/arrow-rs/pull/6583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Upgrade to twox-hash 2.0 [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7347: URL: https://github.com/apache/arrow-rs/pull/7347#issuecomment-2759427373 thanks @mbrobbel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] MINOR: [R] Don't print that we're cleaning up static libs [arrow]

2025-03-27 Thread via GitHub
nealrichardson opened a new pull request, #45958: URL: https://github.com/apache/arrow/pull/45958 ### Rationale for this change Suggestion from @gaborcsardi to be more discreet about #45870. ### What changes are included in this PR? `@` ### Are these changes tested

Re: [PR] Upgrade to twox-hash 2.0 [arrow-rs]

2025-03-27 Thread via GitHub
alamb closed pull request #6583: Upgrade to twox-hash 2.0 URL: https://github.com/apache/arrow-rs/pull/6583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] Upgrade to twox-hash 2.0 [arrow-rs]

2025-03-27 Thread via GitHub
alamb merged PR #7347: URL: https://github.com/apache/arrow-rs/pull/7347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] Merge changelog and version from 54.3.1 into main [arrow-rs]

2025-03-27 Thread via GitHub
alamb commented on PR #7340: URL: https://github.com/apache/arrow-rs/pull/7340#issuecomment-2759426600 > DOH! My original PR never was merged into main! It was approved, but not merged. No it was merged I just doubled checked 😅 - https://github.com/apache/arrow-rs/pull/7318

Re: [PR] GH-41246: [Docs][C++][Python] Improve docs on column encryption for nested fields [arrow]

2025-03-27 Thread via GitHub
EnricoMi commented on code in PR #45411: URL: https://github.com/apache/arrow/pull/45411#discussion_r2013651600 ## docs/source/cpp/parquet.rst: ## @@ -585,6 +585,51 @@ More specifically, Parquet C++ supports: * EncryptionWithFooterKey and EncryptionWithColumnKey modes. * Encry

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
pitrou commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2016824600 ## cpp/src/parquet/geospatial_util_json_internal.cc: ## @@ -0,0 +1,227 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] GH-615: Produce Avro core data types out of Arrow VSR [arrow-java]

2025-03-27 Thread via GitHub
martin-traverse commented on PR #638: URL: https://github.com/apache/arrow-java/pull/638#issuecomment-2759141103 Hi - I think I have addressed all the new comments. Also found an issue in the fixed list producer - I updated the test cases to show this up and included a fix. Fairly sure all

Re: [I] Possible integer overflow [arrow-nanoarrow]

2025-03-27 Thread via GitHub
WillAyd commented on issue #735: URL: https://github.com/apache/arrow-nanoarrow/issues/735#issuecomment-2759382426 Thanks for the issue. These are arguably a bit pedantic, but if there's a reasonable way to clean up the warnings I think we'd be open to any contribution. Just to add

Re: [PR] GH-32276: [C++][FlightRPC] Add option to align RecordBatch buffers given to IPC reader [arrow]

2025-03-27 Thread via GitHub
EnricoMi commented on PR #44279: URL: https://github.com/apache/arrow/pull/44279#issuecomment-2759369667 I have also added the 64-byte alignment, as it was easy to add. fda0101fd5f00e36e3e7572c6fcb468e4f66aebe -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] SEGFAULT when using ADBC Postgres driver with Go FlightServer [arrow-adbc]

2025-03-27 Thread via GitHub
acuitymd-filip commented on issue #2629: URL: https://github.com/apache/arrow-adbc/issues/2629#issuecomment-2759206305 @lidavidm Amazing, thank you! My issue is fixed and everything works correctly now as far as I can tell. -- This is an automated message from the Apache Git Service. To r

Re: [PR] feat(go/adbc/driver/flightsql): Add OAuth Support to Flight Client [arrow-adbc]

2025-03-27 Thread via GitHub
zeroshade commented on code in PR #2651: URL: https://github.com/apache/arrow-adbc/pull/2651#discussion_r2017494290 ## go/adbc/driver/flightsql/flightsql_database.go: ## @@ -149,7 +151,7 @@ func (d *databaseImpl) SetOptions(cnOptions map[string]string) error { if u, ok

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017528679 ## cpp/src/parquet/geospatial_util_internal.cc: ## @@ -0,0 +1,237 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] ARROW-15384: [Python] Wheel task for python 3.7 M1 [arrow]

2025-03-27 Thread via GitHub
kszucs commented on PR #12195: URL: https://github.com/apache/arrow/pull/12195#issuecomment-2759309843 Closing it since python 3.7 has reached EOL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] ARROW-15384: [Python] Wheel task for python 3.7 M1 [arrow]

2025-03-27 Thread via GitHub
kszucs closed pull request #12195: ARROW-15384: [Python] Wheel task for python 3.7 M1 URL: https://github.com/apache/arrow/pull/12195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] GH-45522: [Parquet][C++] Parquet GEOMETRY and GEOGRAPHY logical type implementations [arrow]

2025-03-27 Thread via GitHub
paleolimbot commented on code in PR #45459: URL: https://github.com/apache/arrow/pull/45459#discussion_r2017496308 ## cpp/src/parquet/column_writer.cc: ## @@ -1104,9 +1107,18 @@ int64_t ColumnWriterImpl::Close() { if (rows_written_ > 0 && chunk_statistics.is_set()) {

Re: [PR] GH-45937: [C++][Parquet] Variant logical type definition [arrow]

2025-03-27 Thread via GitHub
neilechao commented on code in PR #45375: URL: https://github.com/apache/arrow/pull/45375#discussion_r2017491219 ## cpp/src/parquet/arrow/variant.h: ## @@ -0,0 +1,65 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

  1   2   3   4   >