Re: [I] Add ObjectStore::list_opts [arrow-rs-object-store]

2025-04-04 Thread via GitHub
tustvold commented on issue #295: URL: https://github.com/apache/arrow-rs-object-store/issues/295#issuecomment-2780346442 The other APIs use https://docs.rs/http/1.2.0/http/struct.Extensions.html which implements Clone -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Remove default implementation for `AsyncFileReader::get_metadata_with_options` [arrow-rs]

2025-04-04 Thread via GitHub
adamreeve commented on code in PR #7342: URL: https://github.com/apache/arrow-rs/pull/7342#discussion_r2020324846 ## parquet/src/arrow/async_reader/mod.rs: ## @@ -172,37 +159,14 @@ impl AsyncFileReader for T { let footer = ParquetMetaDataReader::decode_footer_tai

Re: [PR] GH-45937: [C++][Parquet] Variant logical type definition [arrow]

2025-04-04 Thread via GitHub
neilechao commented on code in PR #45375: URL: https://github.com/apache/arrow/pull/45375#discussion_r2025232060 ## cpp/src/parquet/arrow/variant.h: ## @@ -0,0 +1,70 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. Se

Re: [I] Support Parquet key management tools [arrow-rs]

2025-04-04 Thread via GitHub
AudriusButkevicius commented on issue #7256: URL: https://github.com/apache/arrow-rs/issues/7256#issuecomment-2773759627 I guess this is not closed as needs to be weaved in for encryption? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-04 Thread via GitHub
kou commented on PR #45854: URL: https://github.com/apache/arrow/pull/45854#issuecomment-2771107586 > * C Glib & Ruby - appears to be an environment setup issue with CMake; likely unrelated Yes. It's unrelated: https://github.com/apache/arrow/issues/45994 -- This is an automate

Re: [I] object_store unit test might be flaky [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #2141: URL: https://github.com/apache/arrow-rs/issues/2141#issuecomment-2741687560 Migrating from arrow-rs issue [#2141](https://github.com/apache/arrow-rs/issues/2141) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] [C++] Define an official support policy [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #46002: URL: https://github.com/apache/arrow/issues/46002#issuecomment-2771103662 +1 Can we define the followings too? * How to change the tier for platform X (e.g.: tier 2 -> tier 1, tier 2 -> tier 2, ...) * FYI: We have https://arrow.apache.org/doc

Re: [I] [Parquet] Support Variant Encoding for Parquet [arrow-go]

2025-04-04 Thread via GitHub
sfc-gh-mbojanczyk commented on issue #310: URL: https://github.com/apache/arrow-go/issues/310#issuecomment-2780180299 Alright, I sidestepped the precommit hooks for now to get some feedback on my approach in [this draft PR](https://github.com/apache/arrow-go/pull/344). This only has Variant

Re: [I] c/driver/sqlite: Column type is always int64 with empty table [arrow-adbc]

2025-04-04 Thread via GitHub
davlee1972 commented on issue #581: URL: https://github.com/apache/arrow-adbc/issues/581#issuecomment-2780182171 I think adding a schema option like how pyarrow dataset also accepts a schema would be the best approach.. This could also support casting data types.. SQLite for example r

[PR] feat(parquet): add variant encoder/decoder [arrow-go]

2025-04-04 Thread via GitHub
sfc-gh-mbojanczyk opened a new pull request, #344: URL: https://github.com/apache/arrow-go/pull/344 ### Rationale for this change This adds a basic Variant encoder/decoder to start the process of supporting the new [Variant encoding spec](https://github.com/apache/parquet-format/blob/mas

Re: [I] Use `chrono` version 0.4.39 [arrow-rs]

2025-04-04 Thread via GitHub
matthewmturner commented on issue #7386: URL: https://github.com/apache/arrow-rs/issues/7386#issuecomment-2780168182 Posted this in wrong repo, sorry -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Use `chrono` version 0.4.39 [arrow-rs]

2025-04-04 Thread via GitHub
matthewmturner closed issue #7386: Use `chrono` version 0.4.39 URL: https://github.com/apache/arrow-rs/issues/7386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[I] Use `chrono` version 0.4.39 [arrow-rs]

2025-04-04 Thread via GitHub
matthewmturner opened a new issue, #7386: URL: https://github.com/apache/arrow-rs/issues/7386 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Because of [this](https://github.com/apache/arrow-rs/issues/7196) issue I have chrono pi

Re: [I] [Parquet] Support Variant Encoding for Parquet [arrow-go]

2025-04-04 Thread via GitHub
sfc-gh-mbojanczyk commented on issue #310: URL: https://github.com/apache/arrow-go/issues/310#issuecomment-2780155833 It's quite a load of output, but for example: ``` golangci-lint-full-internal..Failed - hook id: golangci-lint-full

Re: [I] Add ObjectStore::list_opts [arrow-rs-object-store]

2025-04-04 Thread via GitHub
stayrascal commented on issue #295: URL: https://github.com/apache/arrow-rs-object-store/issues/295#issuecomment-2780147007 I'm trying to add the `list_opts`, but meet a problem about `Extensions` during passing it into a closure, which need the `Extensions` implemented `std::marker::Copy`

Re: [PR] GH-40343: [C++] Move S3FileSystem to the registry [arrow]

2025-04-04 Thread via GitHub
bkietz commented on code in PR #41559: URL: https://github.com/apache/arrow/pull/41559#discussion_r2029652585 ## cpp/src/arrow/filesystem/s3fs.cc: ## @@ -3082,6 +3085,30 @@ Result S3FileSystem::PathFromUri(const std::string& uri_string) con

Re: [PR] GH-46023: [CI][MATLAB] libmexclass doesn't work with CMake 4.0.0 [arrow]

2025-04-04 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46033: URL: https://github.com/apache/arrow/pull/46033#issuecomment-2780130041 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit bfaa61286aee54528f0a342e906df549eaf42b94. There were no

Re: [PR] GH-40343: [C++] Move S3FileSystem to the registry [arrow]

2025-04-04 Thread via GitHub
kou commented on code in PR #41559: URL: https://github.com/apache/arrow/pull/41559#discussion_r2029632486 ## cpp/src/arrow/filesystem/s3fs.cc: ## @@ -3082,6 +3085,30 @@ Result S3FileSystem::PathFromUri(const std::string& uri_string) con i

Re: [I] [CI][Crossbow] Use apache/arrow instead of separated repository (e.g. ursacomputing/crossbow) [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #46014: URL: https://github.com/apache/arrow/issues/46014#issuecomment-2780005425 Wow! I didn't know it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [I] [C++] Bump bundled crc32c version [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #45989: URL: https://github.com/apache/arrow/issues/45989#issuecomment-2779985063 > https://github.com/googleapis/google-cloud-cpp/blob/04483120a960dcd63073261824995b1d43b44cc8/google/cloud/storage/internal/crc32c.cc#L23 It seems that we don't need to use google

Re: [I] [C++] Bump bundled utf8proc version [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #45992: URL: https://github.com/apache/arrow/issues/45992#issuecomment-2779963532 Issue resolved by pull request 46032 https://github.com/apache/arrow/pull/46032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-45992: [C++] Bump bundled utf8proc version to 2.10.0 [arrow]

2025-04-04 Thread via GitHub
kou merged PR #46032: URL: https://github.com/apache/arrow/pull/46032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [I] [CI][MATLAB] libmexclass doesn't work with CMake 4.0.0 [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #46023: URL: https://github.com/apache/arrow/issues/46023#issuecomment-2779953451 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [I] [CI][R] test-r-offline-maximal failed with install_pyarrow [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #46024: URL: https://github.com/apache/arrow/issues/46024#issuecomment-2779961678 Ah, we may need to rename `/etc/resolv.conf` while running tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [R][CI] two linux-as-cran jobs fail with leftover files [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #45714: URL: https://github.com/apache/arrow/issues/45714#issuecomment-2779950658 @assignUser You may want to add this to 20.0.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [R][CI] two linux-as-cran jobs fail with leftover files [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #45714: URL: https://github.com/apache/arrow/issues/45714#issuecomment-2779950170 Issue resolved by pull request 46026 https://github.com/apache/arrow/pull/46026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-45714: [CI][R] Don't run tests that use reticulate on CRAN [arrow]

2025-04-04 Thread via GitHub
kou merged PR #46026: URL: https://github.com/apache/arrow/pull/46026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [I] [C++] DCHECK macros conflicts with glog [arrow]

2025-04-04 Thread via GitHub
kou commented on issue #46011: URL: https://github.com/apache/arrow/issues/46011#issuecomment-2779946380 Because it's shorter than `ARROW_DCHECK*`. In general, (reasonable) shorter names are easier to write and read. -- This is an automated message from the Apache Git Service. To respond

[PR] fix(parquet/file): restore goroutine safety for reader [arrow-go]

2025-04-04 Thread via GitHub
zeroshade opened a new pull request, #343: URL: https://github.com/apache/arrow-go/pull/343 ### Rationale for this change Fixes #342 ### What changes are included in this PR? use `sync.OnceValues` to initialize the row group page index reader on demand, but safely -- Thi

Re: [PR] feat(csharp/src/Drivers/Apache): Regenerate Thrift classes based on a newer TCLIService.thrift [arrow-adbc]

2025-04-04 Thread via GitHub
CurtHagenlocher merged PR #2611: URL: https://github.com/apache/arrow-adbc/pull/2611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [I] [Parquet] Support Variant Encoding for Parquet [arrow-go]

2025-04-04 Thread via GitHub
zeroshade commented on issue #310: URL: https://github.com/apache/arrow-go/issues/310#issuecomment-2779899532 Hmm, can you put the issues you're seeing in this issue? We run the `golangci-lint` on the entire repo as part of CI, so there shouldn't be any issues for directories you haven't to

Re: [PR] GH-40343: [C++] Move S3FileSystem to the registry [arrow]

2025-04-04 Thread via GitHub
bkietz commented on code in PR #41559: URL: https://github.com/apache/arrow/pull/41559#discussion_r2029547333 ## ci/docker/conda-cpp.dockerfile: ## @@ -71,6 +71,7 @@ ENV ARROW_ACERO=ON \ ARROW_ORC=ON \ ARROW_PARQUET=ON \ ARROW_S3=ON \ +ARROW_S3_MODULE=ON \ Re

Re: [I] [Parquet] Support Variant Encoding for Parquet [arrow-go]

2025-04-04 Thread via GitHub
sfc-gh-mbojanczyk commented on issue #310: URL: https://github.com/apache/arrow-go/issues/310#issuecomment-2779895372 Got sniped by some things around here, but finally dusted off the change and I think I've got an initial stab at the base of Variant encoding (haven't yet plumbed this into

Re: [I] New clippy failures in code base with release of rustc 1.86 [arrow-rs]

2025-04-04 Thread via GitHub
westonpace closed issue #7381: New clippy failures in code base with release of rustc 1.86 URL: https://github.com/apache/arrow-rs/issues/7381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] [CI][Python] A new version (77.0.2) of setuptools seems to have broken some of our builds [arrow]

2025-04-04 Thread via GitHub
raulcd commented on issue #45867: URL: https://github.com/apache/arrow/issues/45867#issuecomment-2742790701 A new release of setuptools has been done which temporarily solves the issue (there is a Warning of deprecation being raised). We should investigate whether we have to do any change

Re: [PR] GH-45614: [C++] Use Boost's CMake packages instead of FindBoost.cmake in CMake [arrow]

2025-04-04 Thread via GitHub
github-actions[bot] commented on PR #45623: URL: https://github.com/apache/arrow/pull/45623#issuecomment-2742609293 Revision: 471114c569731d795ff15bab77c6b224fbbfa505 Submitted crossbow builds: [ursacomputing/crossbow @ actions-a49eb69369](https://github.com/ursacomputing/crossbow/bra

Re: [I] [EPIC] Port object_store content from arrow-rs repository [arrow-rs-object-store]

2025-04-04 Thread via GitHub
alamb commented on issue #2: URL: https://github.com/apache/arrow-rs-object-store/issues/2#issuecomment-2748781351 I think this issue is now done 🎉 Thanks everyone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] There is no way to pass object store client options as environment variables [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #6333: URL: https://github.com/apache/arrow-rs/issues/6333#issuecomment-2741681008 Migrating from arrow-rs issue [#6333](https://github.com/apache/arrow-rs/issues/6333) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] JSON writer - write null structs as null, instead of struct with nulls [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #5066: URL: https://github.com/apache/arrow-rs/issues/5066#issuecomment-2741682718 Migrating from arrow-rs issue [#5066](https://github.com/apache/arrow-rs/issues/5066) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Bump `object_store` to `0.12.0` [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on PR #7328: URL: https://github.com/apache/arrow-rs/pull/7328#issuecomment-2766430986 > > Is this PR ready to go? Looks like > > ``` > > * [Change Parquet API interaction for u64 #7252](https://github.com/apache/arrow-rs/pull/7252) > > has feedback but no action f

Re: [PR] GH-45732: [C++][Compute] Accept more pivot key types [arrow]

2025-04-04 Thread via GitHub
pitrou commented on code in PR #45945: URL: https://github.com/apache/arrow/pull/45945#discussion_r2024469418 ## cpp/src/arrow/compute/kernels/aggregate_pivot.cc: ## @@ -42,47 +44,56 @@ struct PivotImpl : public ScalarAggregator { values_.push_back(MakeNullScalar(value_ty

[PR] MINOR: [C++] Fix doxygen issue in ree_util.h [arrow]

2025-04-04 Thread via GitHub
amoeba opened a new pull request, #46012: URL: https://github.com/apache/arrow/pull/46012 ### Rationale for this change clang 19.1.0 or higher now warns (or errors under `-Wdocumentation`) about improper use of the `\par` directive and we have one such instance in our docs in https:/

Re: [PR] GH-46011: [C++] Hide DCHECK family from public headers [arrow]

2025-04-04 Thread via GitHub
github-actions[bot] commented on PR #46015: URL: https://github.com/apache/arrow/pull/46015#issuecomment-2775931985 Revision: ecc5934545f2e4dbe1ce56b340250dc01a8ac602 Submitted crossbow builds: [ursacomputing/crossbow @ actions-cc97883a32](https://github.com/ursacomputing/crossbow/bra

Re: [PR] Clean up `ArrowReaderMetadata::load_async` [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on PR #7369: URL: https://github.com/apache/arrow-rs/pull/7369#issuecomment-2779826199 Thanks again @etseidl @adamreeve and @kylebarron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Improve documentation for AsyncFileReader::get_metadata [arrow-rs]

2025-04-04 Thread via GitHub
alamb merged PR #7380: URL: https://github.com/apache/arrow-rs/pull/7380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [I] ArrowReaderMetadata API makes it too easy to (accidentally) make an additional object store request [arrow-rs]

2025-04-04 Thread via GitHub
alamb closed issue #6476: ArrowReaderMetadata API makes it too easy to (accidentally) make an additional object store request URL: https://github.com/apache/arrow-rs/issues/6476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Clean up `ArrowReaderMetadata::load_async` [arrow-rs]

2025-04-04 Thread via GitHub
alamb merged PR #7369: URL: https://github.com/apache/arrow-rs/pull/7369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] Clean up `ArrowReaderMetadata::load_async` [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on PR #7369: URL: https://github.com/apache/arrow-rs/pull/7369#issuecomment-2779826076 Let's do it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [I] [Python] test_dtypes hypotesis test fails sporadically [arrow]

2025-04-04 Thread via GitHub
pitrou commented on issue #45292: URL: https://github.com/apache/arrow/issues/45292#issuecomment-2777838591 Well, @kszucs could probably look into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] GH-36411: [C++][Python] Use meson-python for PyArrow build system [arrow]

2025-04-04 Thread via GitHub
WillAyd commented on PR #45854: URL: https://github.com/apache/arrow/pull/45854#issuecomment-2738713615 @kou I have made some offline progress on this, but one of the things I am getting stuck on is how the pyarrow C++ modules are being compiled. From what I understand, the current build pr

Re: [I] [object-store]: Implement credential_process support for S3 [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #6422: URL: https://github.com/apache/arrow-rs/issues/6422#issuecomment-2741680933 Migrating from arrow-rs issue [#6422](https://github.com/apache/arrow-rs/issues/6422) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] [Python] Improve the pyarrow build system [arrow]

2025-04-04 Thread via GitHub
vyasr commented on issue #36411: URL: https://github.com/apache/arrow/issues/36411#issuecomment-2738185657 > > I think that this question means whether `meson install` can install `XXXConfig.cmake` or not (downstream projects can find a project by `find_package(XXX)` or not). I think that t

Re: [PR] GH-46004: [C++] Add ODBC build option to arrow root cmake file [arrow]

2025-04-04 Thread via GitHub
github-actions[bot] commented on PR #46005: URL: https://github.com/apache/arrow/pull/46005#issuecomment-2770336541 :warning: GitHub issue #46004 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] [Python] pyarrow._s3fs.S3FileSystem credentials issue with assumerole profile config [arrow]

2025-04-04 Thread via GitHub
fehtemam commented on issue #39304: URL: https://github.com/apache/arrow/issues/39304#issuecomment-2767380312 @jafournier did you ever find a solution for this? Is there a recommended way to use Pyarrow's dataset with assume role authentication? -- This is an automated message from the Ap

Re: [I] [EPIC] Port object_store content from arrow-rs repository [arrow-rs-object-store]

2025-04-04 Thread via GitHub
ByteBaker commented on issue #2: URL: https://github.com/apache/arrow-rs-object-store/issues/2#issuecomment-2743273862 > Thanks [@ByteBaker](https://github.com/ByteBaker) Ohh, I meant I can't modify since I don't have permissions. 😅 @alamb -- This is an automated message from the

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
pw42020 commented on PR #45973: URL: https://github.com/apache/arrow/pull/45973#issuecomment-2779780643 Just did! Thanks so much for your help on this PR! Looking forward to helping out more. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[I] Support auth using AssumeRoleWithWebIdentity for non-AWS S3-compatible implementations [arrow-rs-object-store]

2025-04-04 Thread via GitHub
Muon opened a new issue, #283: URL: https://github.com/apache/arrow-rs-object-store/issues/283 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** We're using `object_store` for S3 access. We're trying to get OIDC auth working

Re: [PR] GH-45886: [CI] Upload and publish test results [arrow]

2025-04-04 Thread via GitHub
EnricoMi commented on code in PR #45887: URL: https://github.com/apache/arrow/pull/45887#discussion_r2008744095 ## .github/workflows/test-results.yml: ## @@ -0,0 +1,68 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. Se

[I] Multipart upload can leave futures unpolled, leading to timeout [arrow-rs-object-store]

2025-04-04 Thread via GitHub
wjones127 opened a new issue, #93: URL: https://github.com/apache/arrow-rs-object-store/issues/93 **Describe the bug** Instead of waiting until the data passed to the writer is uploaded to return ready, we buffer it until there is enough data and then put the request future in `Futur

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on PR #45973: URL: https://github.com/apache/arrow/pull/45973#issuecomment-2779749411 Your changes look good! @pw42020 - before I merge this, could you please update the description of this pull request to remove the note about the `RecordBatchFileWriter` being

Re: [I] [EPIC] Port object_store content from arrow-rs repository [arrow-rs-object-store]

2025-04-04 Thread via GitHub
alamb commented on issue #2: URL: https://github.com/apache/arrow-rs-object-store/issues/2#issuecomment-2741704602 Thanks for your help @assignUser -- the commands seem to have worked great! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] [R][CI]: Compile error on opensuse155 [arrow]

2025-04-04 Thread via GitHub
jonkeane commented on issue #45718: URL: https://github.com/apache/arrow/issues/45718#issuecomment-2743490119 Sure, I'm fine to be targeted here and let the C++20 upgrade do cleanup -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Update flatbuffers requirement from 24.12.23 to 25.2.10 [arrow-rs]

2025-04-04 Thread via GitHub
dependabot[bot] commented on PR #7118: URL: https://github.com/apache/arrow-rs/pull/7118#issuecomment-2755167067 Looks like flatbuffers is up-to-date now, so this is no longer needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-04-04 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2025275904 ## cpp/src/parquet/column_writer.cc: ## @@ -892,6 +897,8 @@ class ColumnWriterImpl { std::vector> data_pages_; + internal::ContentDefinedChunker content_defined_

Re: [PR] Test int96 Parquet file from Spark [arrow-rs]

2025-04-04 Thread via GitHub
alamb merged PR #7367: URL: https://github.com/apache/arrow-rs/pull/7367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [I] [C++][CI] Appveyor job fails to build boost [arrow]

2025-04-04 Thread via GitHub
pitrou commented on issue #45966: URL: https://github.com/apache/arrow/issues/45966#issuecomment-2766254212 This may be fixed by https://github.com/apache/arrow/pull/45623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [I] Some clippy errors after updating rust toolchain [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #3011: URL: https://github.com/apache/arrow-rs/issues/3011#issuecomment-2741686406 Migrating from arrow-rs issue [#3011](https://github.com/apache/arrow-rs/issues/3011) -- This is an automated message from the Apache Git Service. To respond to the message, please

[I] Release object store `0.11.0` (breaking API) around Aug 15 2024 [arrow-rs-object-store]

2025-04-04 Thread via GitHub
alamb opened a new issue, #59: URL: https://github.com/apache/arrow-rs-object-store/issues/59 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Following on from https://github.com/apache/arrow-rs/issues/6022 There are several

Re: [PR] GH-45862: Modify FixedSizeListBuilder behavior for null slots [arrow]

2025-04-04 Thread via GitHub
amoeba commented on PR #45889: URL: https://github.com/apache/arrow/pull/45889#issuecomment-2746482670 Thanks for the quick review @trxcllnt. I'll let CI run and I've re-requested a review in case you want another look or just to sign off. -- This is an automated message from the Apache G

[PR] MINOR: [C#] Bump Grpc.AspNetCore.Server from 2.67.0 to 2.70.0 in /csharp [arrow]

2025-04-04 Thread via GitHub
dependabot[bot] opened a new pull request, #45842: URL: https://github.com/apache/arrow/pull/45842 Bumps [Grpc.AspNetCore.Server](https://github.com/grpc/grpc-dotnet) from 2.67.0 to 2.70.0. Release notes Sourced from https://github.com/grpc/grpc-dotnet/releases";>Grpc.AspNetCore.Se

Re: [PR] Deprecated the public expose of test_util package [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on PR #7375: URL: https://github.com/apache/arrow-rs/pull/7375#issuecomment-2775358812 > @alamb Before making the change, could you please help to check something below, thanks. > > 1. The tests in `arrow/src/util/test_util.rs` will show that it uses deprecated code w

[PR] MINOR: [Swift] Bump github.com/apache/arrow-go/v18 from 18.1.0 to 18.2.0 in /swift/CDataWGo [arrow]

2025-04-04 Thread via GitHub
dependabot[bot] opened a new pull request, #45836: URL: https://github.com/apache/arrow/pull/45836 Bumps [github.com/apache/arrow-go/v18](https://github.com/apache/arrow-go) from 18.1.0 to 18.2.0. Release notes Sourced from https://github.com/apache/arrow-go/releases";>github.com/a

[I] Object Store Authorization [arrow-rs-object-store]

2025-04-04 Thread via GitHub
tustvold opened a new issue, #166: URL: https://github.com/apache/arrow-rs-object-store/issues/166 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** The various cloud stores have moderately sophisticated logic for fetching c

Re: [I] ObjectStore: hard to determine the cause of the error thrown from retry [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #5013: URL: https://github.com/apache/arrow-rs/issues/5013#issuecomment-2741682847 Migrating from arrow-rs issue [#5013](https://github.com/apache/arrow-rs/issues/5013) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] object_store: allow setting content-type per request [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #5329: URL: https://github.com/apache/arrow-rs/issues/5329#issuecomment-2741682281 Migrating from arrow-rs issue [#5329](https://github.com/apache/arrow-rs/issues/5329) -- This is an automated message from the Apache Git Service. To respond to the message, please

[I] Add ETag to ObjectMeta [arrow-rs-object-store]

2025-04-04 Thread via GitHub
tustvold opened a new issue, #243: URL: https://github.com/apache/arrow-rs-object-store/issues/243 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** All object stores support a notion of an ETag that uniquely identifies a re

Re: [PR] Add a JSON reader option to ignore type conflicts [arrow-rs]

2025-04-04 Thread via GitHub
scovich commented on PR #7276: URL: https://github.com/apache/arrow-rs/pull/7276#issuecomment-2779674533 bump? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[PR] GH-31603: [C++][Python] Wrap encryption keys in secure string [arrow]

2025-04-04 Thread via GitHub
EnricoMi opened a new pull request, #46017: URL: https://github.com/apache/arrow/pull/46017 ### Rationale for this change Arrow deals with secrets like encryption / decryption keys which must be kept private. One way of leaking such secrets is through memory allocation where another proc

Re: [PR] GH-45735: [C++] Broken tests for extract_regex compute funcion [arrow]

2025-04-04 Thread via GitHub
arashandishgar commented on PR #45900: URL: https://github.com/apache/arrow/pull/45900#issuecomment-2775423674 Thank you for your review. By the way, I have a question about StringView, BinaryView, and FixedSizeBinary. Do you know if there are any plans to port these string types to scalar_

Re: [PR] GH-45732: [C++][Compute] Accept more pivot key types [arrow]

2025-04-04 Thread via GitHub
pitrou commented on code in PR #45945: URL: https://github.com/apache/arrow/pull/45945#discussion_r2021022438 ## cpp/src/arrow/compute/kernels/pivot_internal.cc: ## @@ -18,110 +18,137 @@ #include "arrow/compute/kernels/pivot_internal.h" #include +#include +#include +#in

Re: [I] Add AzureConfigKey::ContainerName [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #4629: URL: https://github.com/apache/arrow-rs/issues/4629#issuecomment-2741683907 Migrating from arrow-rs issue [#4629](https://github.com/apache/arrow-rs/issues/4629) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add Lz4 compression support to arrow batch reader [arrow-adbc]

2025-04-04 Thread via GitHub
CurtHagenlocher commented on code in PR #2669: URL: https://github.com/apache/arrow-adbc/pull/2669#discussion_r2029225879 ## csharp/src/Drivers/Apache/Spark/Lz4Utilities.cs: ## @@ -0,0 +1,57 @@ +/* +* Licensed to the Apache Software Foundation (ASF) under one or more +* contribu

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add Lz4 compression support to arrow batch reader [arrow-adbc]

2025-04-04 Thread via GitHub
CurtHagenlocher merged PR #2669: URL: https://github.com/apache/arrow-adbc/pull/2669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [I] [EPIC] Port object_store content from arrow-rs repository [arrow-rs-object-store]

2025-04-04 Thread via GitHub
assignUser commented on issue #2: URL: https://github.com/apache/arrow-rs-object-store/issues/2#issuecomment-2741558972 ```bash gh issue list -R apache/arrow-rs --label "object-store" -L 1000 --state "all" --json 'number' -q '.[].number' | xargs -I% gh issue transfer % https://github.c

Re: [I] Switch object_store to log crate from tokio-tracing [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #2255: URL: https://github.com/apache/arrow-rs/issues/2255#issuecomment-2741687193 Migrating from arrow-rs issue [#2255](https://github.com/apache/arrow-rs/issues/2255) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] MINOR: Bump parquet.version from 1.15.0 to 1.15.1 [arrow-java]

2025-04-04 Thread via GitHub
jbonofre merged PR #676: URL: https://github.com/apache/arrow-java/pull/676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [I] object_store: Instantiate object store from provided url with store options [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on issue #4047: URL: https://github.com/apache/arrow-rs/issues/4047#issuecomment-2741684827 Migrating from arrow-rs issue [#4047](https://github.com/apache/arrow-rs/issues/4047) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029335367 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -179,22 +180,28 @@ Roughly speaking, local memory sharing workflows can be divided into two catego

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029300769 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -131,33 +119,43 @@ Alternatively, they could directly convert from an existing MATLAB `table` to a

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029336109 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -133,23 +121,36 @@ Alternatively, they could directly convert from an existing MATLAB `table` to a

Re: [I] docs: demonstrate more complex SQLAlchemy integration [arrow-adbc]

2025-04-04 Thread via GitHub
iliya-malecki commented on issue #2079: URL: https://github.com/apache/arrow-adbc/issues/2079#issuecomment-2774023495 @lidavidm im on postgresql+psycopg, i didnt adopt any adbc stuff due to this (general) conversation -- This is an automated message from the Apache Git Service. To respond

Re: [PR] GH-45897: [Ruby] Unify test for table in raw_records and each_raw_record [arrow]

2025-04-04 Thread via GitHub
kou merged PR #45977: URL: https://github.com/apache/arrow/pull/45977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-45750: [C++][Python][Parquet] Implement Content-Defined Chunking for the Parquet writer [arrow]

2025-04-04 Thread via GitHub
kszucs commented on code in PR #45360: URL: https://github.com/apache/arrow/pull/45360#discussion_r2027832305 ## cpp/src/parquet/chunker_internal_test.cc: ## @@ -0,0 +1,1397 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029342822 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -179,47 +177,67 @@ Roughly speaking, local memory sharing workflows can be divided into two catego

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029342822 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -179,47 +177,67 @@ Roughly speaking, local memory sharing workflows can be divided into two catego

Re: [PR] MINOR: [Dev] Add yum build directories to `.gitignore` [arrow]

2025-04-04 Thread via GitHub
pitrou commented on code in PR #46018: URL: https://github.com/apache/arrow/pull/46018#discussion_r2027124973 ## dev/.gitignore: ## @@ -18,3 +18,6 @@ # Python virtual environments for dev tools .venv*/ +# yum-based builds copy a lot of files there +tasks/linux-packages/apach

Re: [PR] Print row, data present, expected type, and row number in error messages for arrow-csv [arrow-rs]

2025-04-04 Thread via GitHub
alamb merged PR #7361: URL: https://github.com/apache/arrow-rs/pull/7361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] [object_store] Support MD5 checksum in attributes [arrow-rs]

2025-04-04 Thread via GitHub
alamb commented on PR #6915: URL: https://github.com/apache/arrow-rs/pull/6915#issuecomment-2741515627 Thank you for this PR. We are in the process of moving the object_store code to its [own repository](https://github.com/apache/arrow-rs-object-store). Would it be possible for you to creat

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029335846 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -133,23 +121,36 @@ Alternatively, they could directly convert from an existing MATLAB `table` to a

Re: [PR] [MATLAB] update README failing example code snippets [arrow]

2025-04-04 Thread via GitHub
kevingurney commented on code in PR #45973: URL: https://github.com/apache/arrow/pull/45973#discussion_r2029334974 ## matlab/doc/matlab_interface_for_apache_arrow_design.md: ## @@ -109,19 +109,7 @@ ans = To serialize MATLAB data to a file on disk (e.g. Feather, Parquet), a MA

  1   2   3   4   >