[I] Artifactory repositories unavailable [arrow]
adampinky85 opened a new issue, #40770: URL: https://github.com/apache/arrow/issues/40770 ### Describe the bug, including details regarding any error messages, version, and platform. Hi Team, The Apache Arrow jfrog artifactory repository does not appear to be available currently. I've tried to access them from Ubuntu, Amazon Linux 2, and Amazon Linux 2023 without success. Retrieving the Ubuntu / Debian package with wget returns javascript code with the message _"MyJFrog doesn't work properly without JavaScript enabled. Please enable it to continue"._ https://arrow.apache.org/install/ ``` curl https://apache.jfrog.io/artifactory/arrow/ubuntu/apache-arrow-apt-source-latest-jammy.deb 302 Found 302 Found nginx ``` ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Java] Cannot create StructVector with dense-union child [arrow]
jarohen opened a new issue, #40773:
URL: https://github.com/apache/arrow/issues/40773
### Describe the bug, including details regarding any error messages,
version, and platform.
I cannot create a StructVector with a dense-union child - it fails with the
following stack trace:
```
java.lang.UnsupportedOperationException: Unknown type: DENSEUNION
at
org.apache.arrow.vector.complex.impl.NullableStructWriter.(NullableStructWriter.java:258)
at
org.apache.arrow.vector.complex.StructVector.(StructVector.java:69)
at xtdb.vector.StructVectorWriterTest.struct with DUV
child(StructVectorWriterTest.kt:101)
```
Test code is essentially:
```java
Field child = new Field("child", FieldType.notNullable(new
ArrowType.Union(UnionMode.Dense, null)), emptyList());
Field struct = new Field("foo",
FieldType.notNullable(ArrowType.Struct.INSTANCE), List.of(child));
try(RootAllocator allocator = new RootAllocator();
StructVector structVec = new StructVector(struct, allocator,
null)) {
System.out.println("working!");
}
```
I suspect it's a case of adding `DENSEUNION` to the `switch` in the
`NullableStructWriter` constructor, with a similar implementation to the sparse
`UNION`?
Cheers,
James
### Component(s)
Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Python][Docs] API docs are broken for dev version [arrow]
AlenkaF closed issue #40623: [Python][Docs] API docs are broken for dev version URL: https://github.com/apache/arrow/issues/40623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] go/adbc/driver/flightsql: support stateless prepared statements [arrow-adbc]
lidavidm opened a new issue, #1657: URL: https://github.com/apache/arrow-adbc/issues/1657 ### What feature or improvement would you like to see? See https://github.com/apache/arrow/pull/40243 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] ci: fix title check again [arrow-adbc]
lidavidm opened a new issue, #1658: URL: https://github.com/apache/arrow-adbc/issues/1658 ### What feature or improvement would you like to see? The original pull_request_target trigger is better since it reacts to changes in the title, while pull_request doesn't (even when manually re-run). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [CI][C++] Google protobuf fails to compile on source verification jobs [arrow]
raulcd opened a new issue, #40772: URL: https://github.com/apache/arrow/issues/40772 ### Describe the bug, including details regarding any error messages, version, and platform. There are several failures on the source verification jobs. Some of the jobs were failing on the past but some of them have started to fail during the last 3 days: [verify-rc-source-cpp-linux-almalinux-8-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407364/job/23026685294) [verify-rc-source-cpp-linux-ubuntu-20.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407583/job/23026686505) [verify-rc-source-cpp-linux-ubuntu-22.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407499/job/23026686123) [verify-rc-source-integration-linux-almalinux-8-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407589/job/23026686542) [verify-rc-source-integration-linux-conda-latest-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407385/job/23026685394) [verify-rc-source-integration-linux-ubuntu-20.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407436/job/23026685741) [verify-rc-source-integration-linux-ubuntu-22.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407468/job/23026685956) [verify-rc-source-integration-macos-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407422/job/23026685648) [verify-rc-source-integration-macos-conda-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407433/job/23026685691) [verify-rc-source-python-linux-almalinux-8-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407466/job/23026685957) [verify-rc-source-python-linux-ubuntu-20.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407735/job/23026700687) [verify-rc-source-python-linux-ubuntu-22.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407444/job/23026685817) [verify-rc-source-ruby-linux-almalinux-8-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407514/job/23026686164) [verify-rc-source-ruby-linux-ubuntu-20.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407609/job/23026686689) [verify-rc-source-ruby-linux-ubuntu-22.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/8409407452/job/23026685860) The changes introduced when the new jobs started failing are the following: https://github.com/apache/arrow/compare/51817917e1436c8799ed382d160798060cd76652...fa36cde1105c8c82ef6407cb991b83d1ca67258a The current stack trace: ``` [190/205] Building CXX object CMakeFiles/libprotobuf-lite.dir/tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/message_lite.cc.o In file included from /usr/include/string.h:495, from /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/stubs/port.h:39, from /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/stubs/common.h:48, from /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/message_lite.h:46, from /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/message_lite.cc:36: In function 'void* memcpy(void*, const void*, size_t)', inlined from 'uint8_t* google::protobuf::io::EpsCopyOutputStream::WriteRaw(const void*, int, uint8_t*)' at /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/io/coded_stream.h:684:16, inlined from 'virtual uint8_t* google::protobuf::internal::ImplicitWeakMessage::_InternalSerialize(uint8_t*, google::protobuf::io::EpsCopyOutputStream*) const' at /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/implicit_weak_message.h:103:28, inlined from 'bool google::protobuf::MessageLite::SerializePartialToZeroCopyStream(google::protobuf::io::ZeroCopyOutputStream*) const' at /tmp/arrow-HEAD.K9cQf/cpp-build/protobuf_ep-prefix/src/protobuf_ep/src/google/protobuf/message_lite.cc:411:30: /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:33: warning: 'void* __builtin___memcpy_chk(void*, const void*, long unsigned int, long unsigned int)' specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=] 34 | return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest)); | ~~~^~~ ``` ### Component(s) C++, Continuous Integration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:
Re: [I] [R][C++] Can't build with jemalloc on RHEL8 [arrow]
assignUser closed issue #32193: [R][C++] Can't build with jemalloc on RHEL8 URL: https://github.com/apache/arrow/issues/32193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] Create an arrow-specific checklist for a CRAN release [arrow]
assignUser closed issue #30136: [R] Create an arrow-specific checklist for a CRAN release URL: https://github.com/apache/arrow/issues/30136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] Document use of arrow 13.0.0 for cran binaries and how to install with up to date libarrow [arrow]
assignUser closed issue #40286: [R] Document use of arrow 13.0.0 for cran binaries and how to install with up to date libarrow URL: https://github.com/apache/arrow/issues/40286 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] ci: pyright is failing [arrow-adbc]
lidavidm opened a new issue, #1659:
URL: https://github.com/apache/arrow-adbc/issues/1659
### What happened?
```
npm notice created a lockfile as package-lock.json. You should commit this
file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.3.3
(node_modules/pyright/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for
fsevents@2.3.3: wanted {"os":"darwin","arch":"any"} (current:
{"os":"linux","arch":"x64"})
+ pyright@1.1.355
added 1 package from 1 contributor and audited 2 packages in 1.224s
found 0 vulnerabilities
internal/modules/cjs/loader.js:626
throw err;
^
Error: Cannot find module 'node:util'
Require stack:
-
/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js
- /home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/index.js
at Function.Module._resolveFilename
(internal/modules/cjs/loader.js:623:15)
at Function.Module._load (internal/modules/cjs/loader.js:527:27)
at Module.require (internal/modules/cjs/loader.js:681:19)
at require (internal/modules/cjs/helpers.js:16:16)
at Object.9632
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js:1:610)
at o
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js:1:1243)
at Object.1264
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/vendor.js:2:794958)
at o
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js:1:1243)
at Object.1476
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright-internal.js:1:442282)
at o
(/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js:1:1243)
{
code: 'MODULE_NOT_FOUND',
requireStack: [
'/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/dist/pyright.js',
'/home/runner/.cache/pyright-python/1.1.355/node_modules/pyright/index.js'
]
}
internal/modules/cjs/loader.js:626
throw err;
^
```
### How can we reproduce the bug?
I think we need to update Node versions:
https://github.com/microsoft/pyright/issues/7365
### Environment/Setup
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[I] replace_with_mask does not properly handle chunked target array [arrow]
vyasr opened a new issue, #40780: URL: https://github.com/apache/arrow/issues/40780 ### Describe the bug, including details regarding any error messages, version, and platform. `pyarrow.compute.replace_with_mask` requires that the inputs not be chunked arrays. The default calling pattern works as expected: ``` >>> pa.compute.replace_with_mask(pa.array([False]), pa.array([True]), pa.array([True])) [ true ] ``` while specifying the replacement values as a chunked array raises a comprehensible exception: ``` >>> pa.compute.replace_with_mask(pa.array([False]), pa.array([True]), pa.chunked_array([[True]])) Traceback (most recent call last): File "/home/coder/.conda/envs/rapids/lib/python3.10/code.py", line 90, in runcode exec(code, self.locals) File "", line 1, in File "/home/coder/.conda/envs/rapids/lib/python3.10/site-packages/pyarrow/compute.py", line 246, in wrapper return func.call(args, None, memory_pool) File "pyarrow/_compute.pyx", line 385, in pyarrow._compute.Function.call File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status pyarrow.lib.ArrowInvalid: Replacements must be array or scalar, not ChunkedArray([ [ true ] ]) ``` However, specifying the values as a chunked array instead silently produces an invalid output: ``` >>> pa.compute.replace_with_mask(pa.chunked_array([[False]]), pa.array([True]), pa.array([True])) [ ] ``` The latter case should also be handled by the same error-checking logic used to validate the replacements (unless chunked arrays can in fact be supported, which would be nice but isn't too important). ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] c/driver/framework: get rid of Status/Result [arrow-adbc]
lidavidm opened a new issue, #1663: URL: https://github.com/apache/arrow-adbc/issues/1663 ### What feature or improvement would you like to see? This is more complexity than we need. We can keep using something like SetError. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [C++] Re-order loads and stores of atomics in MemoryPoolStats to mask memory latency [arrow]
felipecrv opened a new issue, #40783: URL: https://github.com/apache/arrow/issues/40783 ### Describe the enhancement requested Related to #40646 ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Go][Parquet] Enable writing of Parquet footer without closing file [arrow]
zeroshade closed issue #40630: [Go][Parquet] Enable writing of Parquet footer without closing file URL: https://github.com/apache/arrow/issues/40630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [JS] Use bigIntToNumber [arrow]
domoritz opened a new issue, #40784: URL: https://github.com/apache/arrow/issues/40784 ### Describe the enhancement requested Instead of calling Number to convert a bigint. ### Component(s) JavaScript -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R][Release] CRAN packaging checklist for 14.0.2 [arrow]
assignUser closed issue #39285: [R][Release] CRAN packaging checklist for 14.0.2 URL: https://github.com/apache/arrow/issues/39285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [JS] - Decimal conversions are broken [arrow]
domoritz closed issue #40755: [JS] - Decimal conversions are broken URL: https://github.com/apache/arrow/issues/40755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] r/adbcdrivermanager: `adbc_connection_get_objects()` for temporary tables [arrow-adbc]
lidavidm closed issue #1141: r/adbcdrivermanager: `adbc_connection_get_objects()` for temporary tables URL: https://github.com/apache/arrow-adbc/issues/1141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] c/driver/sqlite: port driver to new driver base [arrow-adbc]
lidavidm closed issue #1602: c/driver/sqlite: port driver to new driver base URL: https://github.com/apache/arrow-adbc/issues/1602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] c/driver/sqlite: port driver to new driver base [arrow-adbc]
lidavidm closed issue #1602: c/driver/sqlite: port driver to new driver base URL: https://github.com/apache/arrow-adbc/issues/1602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] ci: pyright is failing [arrow-adbc]
lidavidm closed issue #1659: ci: pyright is failing URL: https://github.com/apache/arrow-adbc/issues/1659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] Formalize CRAN only changes to win binary source [arrow]
assignUser closed issue #40285: [R] Formalize CRAN only changes to win binary source URL: https://github.com/apache/arrow/issues/40285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Packaging] Boost fallback URLs fail at sha256 verification [arrow]
danepitkin opened a new issue, #40782: URL: https://github.com/apache/arrow/issues/40782 ### Describe the bug, including details regarding any error messages, version, and platform. CMake will configure 3 URLs for Boost: https://github.com/apache/arrow/blob/3095344d68af3e4353c9ce098d73fe6768bcb626/cpp/cmake_modules/ThirdpartyToolchain.cmake#L606-L615 However, the first URL is a trimmed version of boost that is custom made by arrow. The following two URLs point to the original boost library with a different download size and sha256 hash. The CMake function `externalproject_add` does not allow for multiple hashes to be configured, so we should either add a second call to `externalproject_add` when falling back or delete the fallback URLs. ### Component(s) Packaging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] ci: fix title check again [arrow-adbc]
lidavidm closed issue #1658: ci: fix title check again URL: https://github.com/apache/arrow-adbc/issues/1658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [C++] Provide mechanism for disabling MemoryPoolStats at either compile- or run-time [arrow]
amoeba opened a new issue, #40781: URL: https://github.com/apache/arrow/issues/40781 ### Describe the enhancement requested This is a follow-on issue to https://github.com/apache/arrow/issues/40646. [MemoryPoolStats](https://github.com/apache/arrow/blob/main/cpp/src/arrow/memory_pool.h#L38) is used by Arrow's memory pools to track statistics related to memory usage by having each pool's `Allocate`, `Reallocate`, and `Free` methods call in its MemoryPoolStats's `UpdateAllocatedBytes` method. The cost of this tracking can be measured by updating a benchmark such as `AllocateDeallocate` to be multi-threaded and comparing a run with `UpdateAllocatedBytes` as-is versus a run with it commented out. With 32 threads this benchmark is 10x faster on x86 when `AllocateDeallocate` is commented out. See https://gist.github.com/amoeba/b95102829280dbe2b1f64e6c23a5f594. https://github.com/apache/arrow/pull/40647 also has much more performance information. https://github.com/apache/arrow/pull/40647 improves the performance under microbenchmarks but the cost cannot be completely avoided. I think it's reasonable to assume that real-world applications can likely detect this cost and may wish to disable `MemoryPoolStats` either at compile- or run-time. ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Artifactory repositories unavailable [arrow]
assignUser closed issue #40770: Artifactory repositories unavailable URL: https://github.com/apache/arrow/issues/40770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Go] Decimal types `GetOneForMarshal` loses precision [arrow]
zeroshade closed issue #40693: [Go] Decimal types `GetOneForMarshal` loses precision URL: https://github.com/apache/arrow/issues/40693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Packaging] `apache.jfrog.io/artifactory/arrow/ubuntu` likely down? [arrow]
assignUser closed issue #40744: [Packaging] `apache.jfrog.io/artifactory/arrow/ubuntu` likely down? URL: https://github.com/apache/arrow/issues/40744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Benchmarking] [Java] new `java.lang.OutOfMemoryError` in Java benchmarks after local build cache change [arrow]
austin3dickey opened a new issue, #40775: URL: https://github.com/apache/arrow/issues/40775 ### Describe the bug, including details regarding any error messages, version, and platform. As part of the Arrow benchmarking suite, we have been running the [Java microbenchmarks](https://github.com/voltrondata-labs/benchmarks/blob/231078e715001c32d2f4ef2ffca27abb4625fd21/benchmarks/java_micro_benchmarks.py#L50) with Archery on every Arrow commit. They are run on a bare-metal machine with the characteristics listed in the "hardware" section of [this Conbench page](https://conbench.ursa.dev/benchmark-results/065fdf61d88f74978000a56a6131eba5/). Note that the benchmarks run [this Arrow CI script](https://github.com/apache/arrow/blob/main/ci/scripts/java_build.sh) to build Java Arrow, with the environment variables in [this file](https://github.com/apache/arrow/blob/main/dev/conbench_envs/benchmarks.env) used for building and running benchmarks. Starting with this PR's commit: - https://github.com/apache/arrow/pull/39708 the Java benchmarks have started to time out after the default timeout of 6 hours. They used to take less than an hour. Before timing out, the following traceback is printed: ``` Exception in thread "CommonsExecStreamPumper-pool-9-thread-2" java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding.decode(StringCoding.java:215) at java.lang.String.(String.java:463) at java.lang.String.(String.java:515) at com.gradle.d.a.a.a(SourceFile:83) at com.gradle.d.a.a.flush(SourceFile:78) at com.gradle.d.a.a.write(SourceFile:72) at java.io.PrintStream.write(PrintStream.java:480) at com.gradle.d.a.b.write(SourceFile:203) at org.apache.commons.exec.StreamPumper.run(StreamPumper.java:112) at java.lang.Thread.run(Thread.java:750) ``` I assume that the Java memory configs need to be changed, but I don't have any experience with Java. This could be an easy fix by changing the environment variable file linked above, and using `@ursabot please benchmark lang=Java` to see if the change fixed anything. Note that the microbenchmark results will not be tracked until this is fixed. ### Component(s) Benchmarking, Java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C#] ArrowStreamReader.Schema returns null before the first record batch is read [arrow]
CurtHagenlocher closed issue #40634: [C#] ArrowStreamReader.Schema returns null before the first record batch is read URL: https://github.com/apache/arrow/issues/40634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] malloc: Incorrect checksum for freed object: probably modified after being freed in arrow-bit-util-benchmark [arrow]
pitrou closed issue #40652: [C++] malloc: Incorrect checksum for freed object: probably modified after being freed in arrow-bit-util-benchmark URL: https://github.com/apache/arrow/issues/40652 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] Building wheel for pyarrow (pyproject.toml) did not run successfully for Linux [arrow]
ManPython opened a new issue, #40776: URL: https://github.com/apache/arrow/issues/40776 ### Describe the bug, including details regarding any error messages, version, and platform. https://stackoverflow.com/questions/77318492/building-wheel-for-pyarrow-pyproject-toml-did-not-run-successfully Not working 14.0.0, also pyarrow-15.0.2.tar.gz (1.1 MB) for freebsd-13.2-RELEASE-p10-amd64-cpython-311. Working for windows. ``` pip install pygwalker Collecting pygwalker Downloading pygwalker-0.4.7-py3-none-any.whl.metadata (19 kB) Requirement already satisfied: appdirs in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (1.4.4) Requirement already satisfied: arrow in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (1.2.3) Collecting astor (from pygwalker) Using cached astor-0.8.1-py2.py3-none-any.whl.metadata (4.2 kB) Collecting cachetools (from pygwalker) Using cached cachetools-5.3.3-py3-none-any.whl.metadata (5.3 kB) Collecting duckdb==0.10.0 (from pygwalker) Downloading duckdb-0.10.0.tar.gz (11.1 MB) 11.1/11.1 MB 29.2 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting gw-dsl-parser==0.1.45a6 (from pygwalker) Downloading gw_dsl_parser-0.1.45a6-py3-none-any.whl.metadata (1.2 kB) Requirement already satisfied: ipython in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (8.18.1) Requirement already satisfied: ipywidgets in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (8.1.1) Requirement already satisfied: jinja2 in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (3.1.2) Collecting kanaries-track==0.0.4 (from pygwalker) Using cached kanaries_track-0.0.4-py3-none-any.whl.metadata (913 bytes) Requirement already satisfied: pandas in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (1.5.2) Requirement already satisfied: psutil in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (5.9.4) Collecting pyarrow (from pygwalker) Downloading pyarrow-15.0.2.tar.gz (1.1 MB) 1.1/1.1 MB 91.5 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: pydantic in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (1.10.4) Requirement already satisfied: pytz in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (2022.7) Requirement already satisfied: requests in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (2.31.0) Collecting segment-analytics-python==2.2.3 (from pygwalker) Using cached segment_analytics_python-2.2.3-py2.py3-none-any.whl.metadata (1.6 kB) Requirement already satisfied: sqlalchemy in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (1.4.46) Collecting sqlglot>=19.0.0 (from pygwalker) Downloading sqlglot-23.0.5-py3-none-any.whl.metadata (17 kB) Requirement already satisfied: typing-extensions in ./.venvs/venvpython311/lib/python3.11/site-packages (from pygwalker) (4.9.0) Collecting wasmtime==12.0.0 (from gw-dsl-parser==0.1.45a6->pygwalker) Using cached wasmtime-12.0.0-py3-none-any.whl.metadata (7.4 kB) Collecting backoff>=2.2.1 (from kanaries-track==0.0.4->pygwalker) Using cached backoff-2.2.1-py3-none-any.whl.metadata (14 kB) Collecting dateutils>=0.6.12 (from kanaries-track==0.0.4->pygwalker) Using cached dateutils-0.6.12-py2.py3-none-any.whl.metadata (1.3 kB) Requirement already satisfied: monotonic~=1.5 in ./.venvs/venvpython311/lib/python3.11/site-packages (from segment-analytics-python==2.2.3->pygwalker) (1.6) Requirement already satisfied: python-dateutil~=2.2 in ./.venvs/venvpython311/lib/python3.11/site-packages (from segment-analytics-python==2.2.3->pygwalker) (2.8.2) Requirement already satisfied: charset-normalizer<4,>=2 in ./.venvs/venvpython311/lib/python3.11/site-packages (from requests->pygwalker) (2.1.1) Requirement already satisfied: idna<4,>=2.5 in ./.venvs/venvpython311/lib/python3.11/site-packages (from requests->pygwalker) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venvs/venvpython311/lib/python3.11/site-packages (from requests->pygwalker) (1.26.13) Requirement already satisfied: certifi>=2017.4.17 in ./.venvs/venvpython311/lib/python3.11/site-packages (from requests->pygwalker) (2022.12.7) Requirement already satisfied: decorator in ./.venvs/venvpython311/lib/python3.11/site-packages (from ipython->pygwalker) (5.1.1) Requirement already satisfied: jedi>=0.16 in
Re: [I] [Python/C++] `S3FileSystem` slow to deserialize due to AWS rule engine JSON parsing [arrow]
pitrou closed issue #40279: [Python/C++] `S3FileSystem` slow to deserialize due to AWS rule engine JSON parsing URL: https://github.com/apache/arrow/issues/40279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Re-order loads and stores of atomics in MemoryPoolStats to mask memory latency [arrow]
felipecrv closed issue #40783: [C++] Re-order loads and stores of atomics in MemoryPoolStats to mask memory latency URL: https://github.com/apache/arrow/issues/40783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Packaging] Unable install Apache Arrow using Debian instructions [arrow]
kou closed issue #40759: [Packaging] Unable install Apache Arrow using Debian instructions URL: https://github.com/apache/arrow/issues/40759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [R] Use either `make sync-cpp` or bootstrap.R not both [arrow]
assignUser opened a new issue, #40787: URL: https://github.com/apache/arrow/issues/40787 ### Describe the enhancement requested After #39147 added a bootstrap.R which does the same as `make sync-cpp` we have a bit of a redundancy. It would be nice to clean that up. Maybe we want to automate more of the release steps (as I started in the PR but removed for clarity) ### Component(s) R -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C#] MapArray doesn't override Accept(IArrowArrayVisitor) [arrow]
CurtHagenlocher closed issue #40788: [C#] MapArray doesn't override Accept(IArrowArrayVisitor) URL: https://github.com/apache/arrow/issues/40788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [C#] MapArray doesn't override Accept(IArrowArrayVisitor) [arrow]
adamreeve opened a new issue, #40788: URL: https://github.com/apache/arrow/issues/40788 ### Describe the bug, including details regarding any error messages, version, and platform. The `MapArray` class doesn't override the `Accept` method, so implementing `IArrowArrayVisitor` doesn't work as expected. If the visitor also implements `IArrowArrayVisitor` then this implementation will be used, otherwise the base `IArrowArrayVisitor.Visit(IArrowArray)` method is called. ### Component(s) C# -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [C#] Slicing a StructArray doesn't slice the child field arrays [arrow]
adamreeve opened a new issue, #40790:
URL: https://github.com/apache/arrow/issues/40790
### Describe the bug, including details regarding any error messages,
version, and platform.
Code to reproduce, as an XUnit test:
```C#
[Fact]
public void TestSliceStructArray()
{
const int numRows = 10;
var fields = new List
{
new Field.Builder().Name("ints").DataType(new
Int32Type()).Nullable(true).Build(),
};
var arrays = new List
{
new Int32Array.Builder().AppendRange(Enumerable.Range(0,
numRows)).Build(),
};
var nullBitmap = new ArrowBuffer.BitmapBuilder().AppendRange(true,
numRows).Build();
var array = new StructArray(new StructType(fields), numRows, arrays,
nullBitmap, nullCount: 0);
var slicedArray = (StructArray) array.Slice(3, 4);
Assert.Equal(4, slicedArray.Length);
Assert.Single(slicedArray.Fields);
var slicedField = slicedArray.Fields[0];
Assert.Equal(4, slicedField.Length); // Fails with "Actual: 10"
}
```
Is this expected behaviour? If not, I'm happy to look into fixing this. The
existing behaviour should probably be kept for backwards compatibility though,
in case users are slicing the child fields themselves to work around this.
### Component(s)
C#
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[I] [C#] Slicing a previously sliced array computes an incorrect offset [arrow]
adamreeve opened a new issue, #40792:
URL: https://github.com/apache/arrow/issues/40792
### Describe the bug, including details regarding any error messages,
version, and platform.
Test to reproduce:
```c#
[Fact]
public void RecursiveArraySlice()
{
var initialValues = Enumerable.Range(0, 100).ToArray();
var array = new Int32Array.Builder().AppendRange(initialValues).Build();
var sliced = (Int32Array) array.Slice(20, 30);
var slicedAgain = (Int32Array) sliced.Slice(5, 10);
Assert.Equal(25, slicedAgain.Offset); // Fails with "Actual: 45"
Assert.Equal(10, slicedAgain.Length);
Assert.Equal(
initialValues.Skip(25).Take(10).Select(val => (int?) val).ToArray(),
(IReadOnlyList) slicedAgain);
}
```
This is because the current offset is accounted for twice, first at
https://github.com/apache/arrow/blob/e3b0bd1feb63d59cd6fb553af976449397b8348e/csharp/src/Apache.Arrow/Arrays/ArrowArrayFactory.cs#L113
Then again at
https://github.com/apache/arrow/blob/e3b0bd1feb63d59cd6fb553af976449397b8348e/csharp/src/Apache.Arrow/Arrays/ArrayData.cs#L112
This only works if the offset is zero.
### Component(s)
C#
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[I] [Dev][CI] pre-commit may be blocked [arrow]
kou opened a new issue, #40791: URL: https://github.com/apache/arrow/issues/40791 ### Describe the enhancement requested https://github.com/apache/arrow/actions/runs/8412061637/job/23032443383 ```text Release Audit Tool...Passed Error: The operation was canceled. ``` ### Component(s) Continuous Integration, Developer Tools -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Add TensorFromJSON helper function [arrow]
AlenkaF closed issue #40297: [C++] Add TensorFromJSON helper function URL: https://github.com/apache/arrow/issues/40297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
