Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
github-actions[bot] commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2080381656 Revision: 703137eda155222396a044c6682c82aa451c6493 Submitted crossbow builds: [ursacomputing/crossbow @ actions-a15f1ef9b4](https://github.com/ursacomputing/crossbow/branches/all?query=actions-a15f1ef9b4) |Task|Status| ||--| |preview-docs|[](https://github.com/ursacomputing/crossbow/actions/runs/8857848768/job/24325806006)| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
amoeba commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2080381018 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
github-actions[bot] commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2080367078 Revision: 0a5cbb335566d882de7fb8949796115d52e1c961 Submitted crossbow builds: [ursacomputing/crossbow @ actions-14581fd73b](https://github.com/ursacomputing/crossbow/branches/all?query=actions-14581fd73b) |Task|Status| ||--| |preview-docs|[](https://github.com/ursacomputing/crossbow/actions/runs/8857536742/job/24325148711)| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
github-actions[bot] commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2080366509 :warning: GitHub issue #41410 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
amoeba opened a new pull request, #41411: URL: https://github.com/apache/arrow/pull/41411 ### Rationale for this change See https://github.com/apache/arrow/issues/41410. ### What changes are included in this PR? Just changes to filesystem.rst. ### Are these changes tested? Yes, locally. ### Are there any user-facing changes? These are those changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]
amoeba commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2080366546 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(csharp): Implement support for transactions, isolation level and read-only flag [arrow-adbc]
CurtHagenlocher commented on PR #1784: URL: https://github.com/apache/arrow-adbc/pull/1784#issuecomment-2080354058 @davidhcoe, would appreciate a look at your leisure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] feat(csharp): Implement support for transactions, isolation level and read-only flag [arrow-adbc]
CurtHagenlocher opened a new pull request, #1784: URL: https://github.com/apache/arrow-adbc/pull/1784 Implements support for transactions, isolation level and read-only flags on imported and exported drivers. Closes #1782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41316: [CI][Python] Reduce CI time [arrow]
assignUser commented on code in PR #41378:
URL: https://github.com/apache/arrow/pull/41378#discussion_r1581724489
##
.github/workflows/python.yml:
##
@@ -171,6 +171,23 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
+ - name: Install ccache
Review Comment:
Hey! So the issue here is that `actions/cache` doesn't really work great for
caching build caches like ccache as caches are immutable and if a primary key
was hit it will not update the cache, leading to worse and worse hit rates.
In this case you also used the same key for 2 different jobs, this will lead
to a key clash and prevent one job from caching (looking at the results likely
arm64 was cached -> 0% hit rate on amd64 as ccache is os/arch specific).
A workaround is to create unique keys (e.g. `ccache-python-macos--${{
github.sha }}` on each run and use 'restore-keys: ccache-python-macos--'
to still get a hit. This is quite wasteful and leads to frequent cache
evictions as only 10GB/repo are allowed which we frequently go over (massivly)
I created an [alternative](https://github.com/assignUser/stash) for this
case but have to ask infra to allow-list it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups [arrow]
assignUser commented on code in PR #41403:
URL: https://github.com/apache/arrow/pull/41403#discussion_r1581719580
##
dev/tasks/r/github.linux.versions.yml:
##
@@ -30,7 +30,6 @@ jobs:
r_version:
# We test devel, release, and oldrel in regular CI.
# This is for older versions
- - "3.6"
- "4.0"
- "4.1"
Review Comment:
Should we add 4.2 as replacement for 3.6?
##
ci/docker/linux-apt-r.dockerfile:
##
@@ -35,7 +35,7 @@ ENV LANG=C.UTF-8
# Build R
# [1]
https://www.digitalocean.com/community/tutorials/how-to-install-r-on-ubuntu-18-04
# [2]
https://linuxize.com/post/how-to-install-r-on-ubuntu-18-04/#installing-r-packages-from-cran
-ARG r=3.6
Review Comment:
oO :D
##
r/R/dplyr-funcs-type.R:
##
@@ -140,7 +140,7 @@ register_bindings_type_cast <- function() {
fix.empty.names = TRUE,
stringsAsFactors = FALSE) {
# we need a specific value of stringsAsFactors because the default was
- # TRUE in R <= 3.6
+ # TRUE in R <= 3.6 and folks might still be cargoculting to stay in the
past.
Review Comment:
:grin:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] `snappy_jll v1.2.0` lead to Arrow_jll failed to build [arrow-julia]
visr commented on issue #505: URL: https://github.com/apache/arrow-julia/issues/505#issuecomment-2080341534 No direct wrapper yet. We built it for GDAL to be able to read geospatial data based on Arrow and Parquet; GeoArrow and GeoParquet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(python): Implement extension type and Schema metadata support [arrow-nanoarrow]
paleolimbot commented on PR #431: URL: https://github.com/apache/arrow-nanoarrow/pull/431#issuecomment-2080322076 Thank you for taking a look! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(python): Implement extension type and Schema metadata support [arrow-nanoarrow]
paleolimbot merged PR #431: URL: https://github.com/apache/arrow-nanoarrow/pull/431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(python): Implement extension type and Schema metadata support [arrow-nanoarrow]
paleolimbot commented on code in PR #431: URL: https://github.com/apache/arrow-nanoarrow/pull/431#discussion_r1581712390 ## python/src/nanoarrow/_lib.pyx: ## @@ -967,10 +989,50 @@ cdef class CSchemaBuilder: if self._ptr.release == NULL: ArrowSchemaInit(self._ptr) +@staticmethod +def copy_existing(CSchema existing_schema): +return CSchemaBuilder(existing_schema.__deepcopy__()) + @staticmethod def allocate(): return CSchemaBuilder(CSchema.allocate()) +def clear_metadata(self): Review Comment: At this level of abstraction it would be hard (since it is basically wrapping a single C API call) but I agree that it would be a better interface! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(python): Implement extension type and Schema metadata support [arrow-nanoarrow]
paleolimbot commented on code in PR #431: URL: https://github.com/apache/arrow-nanoarrow/pull/431#discussion_r1581712127 ## python/src/nanoarrow/schema.py: ## @@ -108,10 +115,73 @@ def create(obj): return TimeUnit(obj) +class ExtensionAccessor: +"""Accessor for extension type parameters""" + +def __init__(self, schema) -> None: +self._schema = schema + +@property +def name(self) -> str: +"""Extension name for this extension type""" +return self._schema._c_schema_view.extension_name + +@property +def metadata(self) -> Union[bytes, None]: +"""Extension metadata for this extension type if present""" +extension_metadata = self._schema._c_schema_view.extension_metadata +return extension_metadata if extension_metadata else None + +@property +def storage(self): +"""Storage type for this extension type""" +metadata = dict(self._schema.metadata.items()) +del metadata[b"ARROW:extension:name"] +if b"ARROW:extension:metadata" in metadata: +del metadata[b"ARROW:extension:metadata"] Review Comment: I added a comment! (Removing the metadata keys is required, or else the schema will still be treated as an extension). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41307: [Java] Use org.apache:apache parent pom version 31 [arrow]
laurentgo commented on PR #41309: URL: https://github.com/apache/arrow/pull/41309#issuecomment-2080316679 I believe I found the issue. Recent version of `maven-jar-plugin` uses an updated version of `plexus-archiver` which will rewrite the jar content to fix modification time if the jar contains a `module-info.class` file. But as Java 11 `jar` command does not support `--date` option, `plexus-archiver` implements a [fallback](https://github.com/codehaus-plexus/plexus-archiver/blob/8458ffacecb543c60dae8e618accbba96a6ff689/src/main/java/org/codehaus/plexus/archiver/jar/JarToolModularJarArchiver.java#L201) which relies on `Files#createTemp()` method. Unfortunately, the file created by this method has only user read/write permission and no group/others permissions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
conbench-apache-arrow[bot] commented on PR #41372: URL: https://github.com/apache/arrow/pull/41372#issuecomment-2080315675 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit ea314a3f8d9d4446836aa999b66659c07421f7a4. There were no benchmark performance regressions. 🎉 The [full Conbench report](https://github.com/apache/arrow/runs/24322775693) has more details. It also includes information about 15 possible false positives for unstable benchmarks that are known to sometimes produce them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Change default branch name from master to main [arrow-rs]
Jefffrey closed issue #3569: Change default branch name from master to main URL: https://github.com/apache/arrow-rs/issues/3569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Change default branch name from master to main [arrow-rs]
Jefffrey commented on issue #3569: URL: https://github.com/apache/arrow-rs/issues/3569#issuecomment-2080315554 Closed as duplicate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] GH-41397: n [R][CI] Windows job failing after R 4.4 release [arrow]
assignUser opened a new pull request, #41409: URL: https://github.com/apache/arrow/pull/41409 ### Rationale for this change We can't throw warnings on cran. ### What changes are included in this PR? Update function to match changes in libarrow ### Are these changes tested? CI ### Are there any user-facing changes? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41397: n [R][CI] Windows job failing after R 4.4 release [arrow]
github-actions[bot] commented on PR #41409: URL: https://github.com/apache/arrow/pull/41409#issuecomment-2080307237 :warning: GitHub issue #41397 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R][CI] Windows job failing after R 4.4 release [arrow]
assignUser commented on issue #41398: URL: https://github.com/apache/arrow/issues/41398#issuecomment-2080305862 I'll give it a try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Add support for mold [arrow]
amoeba commented on issue #40394: URL: https://github.com/apache/arrow/issues/40394#issuecomment-2080272148 Done, see https://github.com/apache/arrow/issues/41408. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Add support for mold [arrow]
amoeba commented on issue #40394: URL: https://github.com/apache/arrow/issues/40394#issuecomment-2080271217 Yep, will do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41367: [C++] Replace [[maybe_unused]] with Arrow macro [arrow]
github-actions[bot] commented on PR #41359: URL: https://github.com/apache/arrow/pull/41359#issuecomment-2080269396 Revision: bdf0bfff1332e4446005d485462e2c9a0088fa9c Submitted crossbow builds: [ursacomputing/crossbow @ actions-eb949ffd4d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-eb949ffd4d) |Task|Status| ||--| |preview-docs|[](https://github.com/ursacomputing/crossbow/actions/runs/8855874267/job/24321419381)| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41367: [C++] Replace [[maybe_unused]] with Arrow macro [arrow]
kou commented on PR #41359: URL: https://github.com/apache/arrow/pull/41359#issuecomment-2080268577 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Add support for mold [arrow]
kou commented on issue #40394: URL: https://github.com/apache/arrow/issues/40394#issuecomment-2080268067 You're right. We should document this and LLD's one. We just forgot them... Could you open an issue for them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] `snappy_jll v1.2.0` lead to Arrow_jll failed to build [arrow-julia]
kongdd commented on issue #505: URL: https://github.com/apache/arrow-julia/issues/505#issuecomment-2080265483 > Arrow.jl is a Julia implementation of the Arrow format, and does not use Arrow_jll, so this issue can be closed. closed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Rename `Function::is_impure()` to `is_pure()` [arrow]
amoeba commented on issue #40607: URL: https://github.com/apache/arrow/issues/40607#issuecomment-2080244437 Hi @felipecrv, this looks like a breaking change to me. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix(csharp/src/Apache.Arrow.Adbc): correct StandardSchemas.ColumnSchema data types [arrow-adbc]
birschick-bq commented on PR #1731: URL: https://github.com/apache/arrow-adbc/pull/1731#issuecomment-2080244619 > On the whole, I'm not excited about exposing two public and non-ADBC-specific functions from this assembly -- particularly without a stronger specification for them. Can we make them internal and then use InternalsVisibleTo? This is already being used in the C# Arrow implementation, so the build processes should support it. > > It would also be nice to eliminate the large number of nullability warnings by either removing the nullability annotations or by putting the code into a `#nullable` block. RE: nullability annotations I believe we need a separate ticket for this. It will require hundreds of small changes to mark current non-nullable types as nullable. I'll create a new issue for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix(csharp/src/Apache.Arrow.Adbc): correct StandardSchemas.ColumnSchema data types [arrow-adbc]
birschick-bq commented on code in PR #1731:
URL: https://github.com/apache/arrow-adbc/pull/1731#discussion_r1581606785
##
csharp/src/Apache.Arrow.Adbc/Extensions/ListArrayExtensions.cs:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using Apache.Arrow.Types;
+
+namespace Apache.Arrow.Adbc.Extensions
+{
+public static class ListArrayExtensions
+{
+///
+/// Creates a from a list of data for the given datatype .
+///
+/// The list of data.
+/// The data type of the contained data.
+/// A of the data.
+public static ListArray CreateNestedListArray(this
IReadOnlyList arrayList, IArrowType dataType)
Review Comment:
Yes. I believe this should likely be included in the Arrow library. Not sure
of it proper placement in the namespace, yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [C++] Add support for mold [arrow]
amoeba commented on issue #40394: URL: https://github.com/apache/arrow/issues/40394#issuecomment-2080236538 Hi @kou, @assignUser: It seems like we should document this in the developer docs. Any reason not to? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix(csharp/src/Apache.Arrow.Adbc): correct StandardSchemas.ColumnSchema data types [arrow-adbc]
birschick-bq commented on code in PR #1731:
URL: https://github.com/apache/arrow-adbc/pull/1731#discussion_r1581606335
##
csharp/src/Apache.Arrow.Adbc/Extensions/StandardSchemaExtensions.cs:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using Apache.Arrow.Types;
+
+namespace Apache.Arrow.Adbc.Extensions
+{
+public static class StandardSchemaExtensions
+{
+
+///
+/// Validates a data array that its column number and types match a
given schema.
+///
+/// The schema to validate against.
+/// The data array to validate.
+/// Throws an exception if the
number of columns or type data types in the data array do not match the schema
fields.
+public static IReadOnlyList Validate(this Schema schema,
IReadOnlyList data)
Review Comment:
I'd be happy to use the existing class, if it were exposed. I could also
make this better and more rigorous, if you believe it might be used outside of
its current context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] fix(csharp/src/Apache.Arrow.Adbc): correct StandardSchemas.ColumnSchema data types [arrow-adbc]
birschick-bq commented on code in PR #1731:
URL: https://github.com/apache/arrow-adbc/pull/1731#discussion_r1581605494
##
csharp/src/Apache.Arrow.Adbc/Extensions/ListArrayExtensions.cs:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using Apache.Arrow.Types;
+
+namespace Apache.Arrow.Adbc.Extensions
+{
+internal static class ListArrayExtensions
Review Comment:
This extensions is marked as `internal`.
##
csharp/src/Apache.Arrow.Adbc/Properties/AssemblyInfo.cs:
##
@@ -0,0 +1,20 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+using System.Runtime.CompilerServices;
+
+[assembly: InternalsVisibleTo("Apache.Arrow.Adbc.Drivers.Apache,
PublicKey=0024048094000602002452534131000401000100e504183f6d470d6b67b6d19212be3e1f598f70c246a120194bc38130101d0c1853e4a0f2232cb12e37a7a90e707aabd38511dac4f25fcb0d691b2aa265900bf42de7f70468fc997551a40e1e0679b605aa2088a4a69e07c117e988f5b1738c570ee66997fba02485e7856a49eca5fd0706d09899b8312577cbb9034599fc92d4")]
Review Comment:
Expose internal interfaces to three libraries, assuming the current signing
key.
##
csharp/src/Apache.Arrow.Adbc/Extensions/StandardSchemaExtensions.cs:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using Apache.Arrow.Types;
+
+namespace Apache.Arrow.Adbc.Extensions
+{
+internal static class StandardSchemaExtensions
Review Comment:
This extension is marked as internal.
##
csharp/src/Apache.Arrow.Adbc/Extensions/ListArrayExtensions.cs:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using Apache.Arrow.Types;
+
+namespace Apache.Arrow.Adbc.Extensions
+{
+public static class ListArrayExten
Re: [I] `snappy_jll v1.2.0` lead to Arrow_jll failed to build [arrow-julia]
Moelf commented on issue #505: URL: https://github.com/apache/arrow-julia/issues/505#issuecomment-2080229853 What is using Arrow_jll -- do we have a wrapper ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] chore: wrap .env values in quotes to prevent parsing errors [arrow-adbc]
lidavidm commented on PR #1779: URL: https://github.com/apache/arrow-adbc/pull/1779#issuecomment-2080202473 The integration test pipeline sets everything by hand, I suppose it could source .env instead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix(go/adbc/driver/flightsql): should use `ctx.Err().Error()` [arrow-adbc]
lidavidm commented on PR #1769: URL: https://github.com/apache/arrow-adbc/pull/1769#issuecomment-2080201521 @zeroshade there's something in the Flight SQL client that is treating io.EOF as an actual error, I think -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] Support join "na_matches" argument [arrow]
nealrichardson commented on issue #41358: URL: https://github.com/apache/arrow/issues/41358#issuecomment-2080196949 Fixed by https://github.com/apache/arrow/pull/41372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
nealrichardson merged PR #41372: URL: https://github.com/apache/arrow/pull/41372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R][CI] Windows job failing after R 4.4 release [arrow]
nealrichardson commented on issue #41398: URL: https://github.com/apache/arrow/issues/41398#issuecomment-2080195502 The line in the C++ header in question was introduced here: https://github.com/apache/arrow/pull/39888/files#diff-dd3ca7b08e6c9982fcc1be7ea36c93a2d7998ed3ff6bab2ae1e0343ada0358d3R53 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] CRAN packaging checklist for version 16.0.0 [arrow]
nealrichardson commented on issue #41404: URL: https://github.com/apache/arrow/issues/41404#issuecomment-2080195010 Re: #41398, I suspect we do need to fix it before releasing, unless we're not updating the libarrow binaries. It's raising on a line that was changed since 15.0: https://github.com/apache/arrow/pull/39888/files#diff-dd3ca7b08e6c9982fcc1be7ea36c93a2d7998ed3ff6bab2ae1e0343ada0358d3R53 The reason we have `-Werror` on that CI job is because CRAN will cause problems if there are compiler warnings thrown, so we want to be alerted earlier. Re: the so-called non-API functions, I wonder if those are related to ALTREP, and if they'll go away once newer R-devel builds cycle through after the policy was clarified? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
jonkeane commented on code in PR #41372:
URL: https://github.com/apache/arrow/pull/41372#discussion_r1581582262
##
r/src/compute-exec.cpp:
##
@@ -411,10 +411,17 @@ std::shared_ptr ExecNode_Join(
const std::shared_ptr& right_data,
std::vector left_keys, std::vector right_keys,
std::vector left_output, std::vector
right_output,
-std::string output_suffix_for_left, std::string output_suffix_for_right) {
+std::string output_suffix_for_left, std::string output_suffix_for_right,
+bool na_matches) {
std::vector left_refs, right_refs, left_out_refs,
right_out_refs;
+ std::vector key_cmps;
for (auto&& name : left_keys) {
left_refs.emplace_back(std::move(name));
+// Populate key_cmps in this loop, one for each key
+// Note that Acero supports having different values for each key, but dplyr
+// only supports one value for all keys, so we're only going to support
that
+// for now.
+key_cmps.emplace_back(na_matches ? acero::JoinKeyCmp::IS :
acero::JoinKeyCmp::EQ);
Review Comment:
So many ways of swapping and passing and evaluating booleans 😂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [C++][R] R Sanitizer nightly error [arrow]
jonkeane commented on issue #41407: URL: https://github.com/apache/arrow/issues/41407#issuecomment-2080182585 I suspect this is also what is responsible for the test-fedore-r-clang-sanitizer as well: https://github.com/ursacomputing/crossbow/actions/runs/8809030613/job/24179194857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41179: [Docs] Documentation for Dissociated IPC Protocol [arrow]
kou commented on code in PR #41180:
URL: https://github.com/apache/arrow/pull/41180#discussion_r1581577699
##
ci/docker/linux-apt-docs.dockerfile:
##
@@ -55,7 +57,10 @@ RUN apt-get update -y && \
r-base=${r}* \
rsync \
ruby-dev \
-wget && \
+wget \
+libasound2-dev && \
+wget -q
https://freeshell.de/phd/chromium/jammy/pool/latest/chromium_124.0.6367.60~linuxmint1+virginia_amd64.deb
&& \
+apt-get install -y ./chromium_124.0.6367.60~linuxmint1+virginia_amd64.deb
&& \
Review Comment:
https://freeshell.de/phd/chromium/jammy/pool/latest/chromium_124.0.6367.60~linuxmint1+virginia_amd64.deb
will be invalid eventually. Our CI will be failed after that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] csharp/adbc/capi: CAdbcDriverImporter leaks memory [arrow-adbc]
CurtHagenlocher commented on issue #1303: URL: https://github.com/apache/arrow-adbc/issues/1303#issuecomment-2080177247 Retroactive duplicate of #1774 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] CRAN packaging checklist for version 16.0.0 [arrow]
jonkeane commented on issue #41404: URL: https://github.com/apache/arrow/issues/41404#issuecomment-2080168636 # Status of [current check results](https://cran.r-project.org/web/checks/check_results_arrow.html) - [x] There are two errors on macos — both are libtool. Which are resolved by https://github.com/apache/arrow/pull/40259 which was not included in the last submission. - [ ] We have a number of notes about using `SETLENGTH` and `SET_TRUELENGTH` IIUC, this is CRAN attempting to cut down on people using so called non-API functions. I believe these are coming from vctrs or cpp11 (oddly cpp11 checks are clean, even [though they definitely have those](https://github.com/r-lib/cpp11/blob/51f4cd5ad9425a491dedf951a3679346d416e51c/inst/include/cpp11/r_vector.hpp#L900-L901) Note: we will need to figure out if we need to resolve #41398 before we can submit to CRAN. The current windows check _is_ on 4.4.0 — that doesn't compile from source there anyway so so long as #41398 is truly a warning -> error situation, we should ok to ship without it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
nealrichardson commented on code in PR #41372:
URL: https://github.com/apache/arrow/pull/41372#discussion_r1581571725
##
r/src/compute-exec.cpp:
##
@@ -411,10 +411,17 @@ std::shared_ptr ExecNode_Join(
const std::shared_ptr& right_data,
std::vector left_keys, std::vector right_keys,
std::vector left_output, std::vector
right_output,
-std::string output_suffix_for_left, std::string output_suffix_for_right) {
+std::string output_suffix_for_left, std::string output_suffix_for_right,
+bool na_matches) {
std::vector left_refs, right_refs, left_out_refs,
right_out_refs;
+ std::vector key_cmps;
for (auto&& name : left_keys) {
left_refs.emplace_back(std::move(name));
+// Populate key_cmps in this loop, one for each key
+// Note that Acero supports having different values for each key, but dplyr
+// only supports one value for all keys, so we're only going to support
that
+// for now.
+key_cmps.emplace_back(na_matches ? acero::JoinKeyCmp::IS :
acero::JoinKeyCmp::EQ);
Review Comment:
Correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] GH-41405: [Release][Docs][GLib] Use Sphinx based GLib front page [arrow]
github-actions[bot] commented on PR #41406: URL: https://github.com/apache/arrow/pull/41406#issuecomment-2080158797 :warning: GitHub issue #41405 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] GH-41405: [Release][Docs][GLib] Use Sphinx based GLib front page [arrow]
kou opened a new pull request, #41406: URL: https://github.com/apache/arrow/pull/41406 ### Rationale for this change We should use the GLib front page generated by Sphinx. ### What changes are included in this PR? Stop reverting the GLib front page change in release script. ### Are these changes tested? No. ### Are there any user-facing changes? Yes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [R] CRAN packaging checklist for version 16.0.0 [arrow]
jonkeane commented on issue #41404: URL: https://github.com/apache/arrow/issues/41404#issuecomment-2080158134 This release will be slightly different since it will include the change in maintainer. IIUC We'll need to do: > * Explain any change in the maintainer’s email address and if possible send confirmation from the previous address (by a separate email to cran-submissi...@r-project.org) or explain why it is not possible. > If the package needs special treatment (for example if vignettes can only be run or re-built on the maintainer’s machine or take a very long time), say so on the submission form. from https://cran.r-project.org/web/packages/policies.html#Submission-1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Release][Docs][GLib] Use Sphinx based GLib front page [arrow]
kou commented on issue #41405: URL: https://github.com/apache/arrow/issues/41405#issuecomment-2080157124 The GLib front page: https://arrow.apache.org/docs/c_glib/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(csharp/test): implement DuckDb test fixture [arrow-adbc]
CurtHagenlocher merged PR #1781: URL: https://github.com/apache/arrow-adbc/pull/1781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41390: [CI] Use setup-python GitHub action on csharp macOS job [arrow]
conbench-apache-arrow[bot] commented on PR #41392: URL: https://github.com/apache/arrow/pull/41392#issuecomment-2080148925 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 15986ae5ffef2f274c04cf0d5eec2155fe6523a6. There were no benchmark performance regressions. 🎉 The [full Conbench report](https://github.com/apache/arrow/runs/24317778328) has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
jonkeane commented on code in PR #41372:
URL: https://github.com/apache/arrow/pull/41372#discussion_r1581558982
##
r/src/compute-exec.cpp:
##
@@ -411,10 +411,17 @@ std::shared_ptr ExecNode_Join(
const std::shared_ptr& right_data,
std::vector left_keys, std::vector right_keys,
std::vector left_output, std::vector
right_output,
-std::string output_suffix_for_left, std::string output_suffix_for_right) {
+std::string output_suffix_for_left, std::string output_suffix_for_right,
+bool na_matches) {
std::vector left_refs, right_refs, left_out_refs,
right_out_refs;
+ std::vector key_cmps;
for (auto&& name : left_keys) {
left_refs.emplace_back(std::move(name));
+// Populate key_cmps in this loop, one for each key
+// Note that Acero supports having different values for each key, but dplyr
+// only supports one value for all keys, so we're only going to support
that
+// for now.
+key_cmps.emplace_back(na_matches ? acero::JoinKeyCmp::IS :
acero::JoinKeyCmp::EQ);
Review Comment:
Out of my curiosity to make sure I'm following this all:
`acero::JoinKeyCmp::IS` ends up being used when `na_match == "na"`, and
therefore `NA`s will match. `acero::JoinKeyCmp::EQ` ends up in all other cases
(i.e. `na_match == "never"`) and that uses equality where `NA`s _do not_ match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] GH-40339: [Java] StringView Initial Implementation [arrow]
kou commented on PR #40340: URL: https://github.com/apache/arrow/pull/40340#issuecomment-2080135497 https://github.com/apache/arrow/pull/41379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41402: [CI][R] Update our backwards compatibility CI any other R 4.4 cleanups [arrow]
github-actions[bot] commented on PR #41403: URL: https://github.com/apache/arrow/pull/41403#issuecomment-2080133567 Revision: f31f859ed8e74b02b8b57e39c2107e5f0ff16197 Submitted crossbow builds: [ursacomputing/crossbow @ actions-802b06fdc8](https://github.com/ursacomputing/crossbow/branches/all?query=actions-802b06fdc8) |Task|Status| ||--| |r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419894/job/24317420516)| |test-fedora-r-clang-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419771/job/24317405716)| |test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419991/job/24317406573)| |test-r-depsource-bundled|[](https://github.com/ursacomputing/crossbow/runs/24317407694)| |test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420303/job/24317407415)| |test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420092/job/24317407012)| |test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420026/job/24317407011)| |test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419869/job/24317405980)| |test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420094/job/24317406777)| |test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420213/job/24317407542)| |test-r-install-local-minsizerel|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419963/job/24317407000)| |test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions/runs/8854420379/job/24317408618)| |test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419657/job/24317405210)| |test-r-linux-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419701/job/24317405402)| |test-r-minimal-build|[](https://github.com/ursacomputing/crossbow/runs/24317407597)| |test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions/runs/8854419814/job/24317405717)| |test-r-offline-minimal|[](https://github.com/ursacomputing/crossbow/runs/24317407676)| |test-r-rhub-debian-gcc-devel-lto-latest|[ |Task|Status| ||--| |test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions/runs/8854226651/job/24316838627)| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-40735: [Packaging][CentOS] Drop support for CentOS 7 [arrow]
kou commented on PR #41395: URL: https://github.com/apache/arrow/pull/41395#issuecomment-2080114806 FYI: manylinux uses AlmaLinux 8 for manylinux_2_28 (CentOS 7 for manylinux2014): https://github.com/pypa/manylinux/blob/main/README.rst#docker-images -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-40735: [Packaging][CentOS] Drop support for CentOS 7 [arrow]
kou commented on PR #41395: URL: https://github.com/apache/arrow/pull/41395#issuecomment-2080113123 > I am happy to migrate the `centos-cpp-static` from CentOS 7 to a newer CentOS on a different PR Let's work on this on a different PR. We may use AlmaLinux 8 for it. > but seem to be unused at the moment. They are used here: https://github.com/apache/arrow/blob/15986ae5ffef2f274c04cf0d5eec2155fe6523a6/dev/tasks/r/github.packages.yml#L120-L147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41402: [CI] [R] Update our backwards compatibility CI any other R 4.4 cleanups [arrow]
jonkeane commented on PR #41403: URL: https://github.com/apache/arrow/pull/41403#issuecomment-2080112825 @github-actions crossbow submit test-r-arrow-backwards-compatibility -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] GH-41402: [CI] [R] Update our backwards compatibility CI any other R 4.4 cleanups [arrow]
jonkeane opened a new pull request, #41403: URL: https://github.com/apache/arrow/pull/41403 ### Rationale for this change Keep up with the state of the world, ensure we are maintaining backwards compatibility. ### What changes are included in this PR? Add missed versions to our backwards compatibility, ensure we're running on R 4.4 elsewhere ### Are these changes tested? By definition. ### Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41402: [CI] [R] Update our backwards compatibility CI any other R 4.4 cleanups [arrow]
github-actions[bot] commented on PR #41403: URL: https://github.com/apache/arrow/pull/41403#issuecomment-2080112385 :warning: GitHub issue #41402 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-40735: [Packaging][CentOS] Drop support for CentOS 7 [arrow]
kou commented on PR #41395:
URL: https://github.com/apache/arrow/pull/41395#issuecomment-2080101740
```diff
diff --git a/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in
b/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in
index 3ede1814b8..258759a1ec 100644
--- a/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in
+++ b/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in
@@ -25,19 +25,8 @@
%define _rhel %{?rhel:%{rhel}}%{!?rhel:0}
%define is_rhel (%{_rhel} != 0)
-%define is_centos_7 (%{_rhel} == 7 && !%{is_amazon_linux})
-
%define major_version %(echo @VERSION@ | grep -o '^[0-9]*')
-%define boost_version %( \
- if [ %{_rhel} -eq 7 ]; then \
-echo 169; \
- fi)
-%define cmake_version %( \
- if [ %{_rhel} -eq 7 ]; then \
-echo 3; \
- fi)
-
%define lz4_requirement %( \
if [ %{_amzn} -eq 0 ]; then \
echo ">= 1.8.0"; \
@@ -55,31 +44,17 @@
%define arrow_cmake_install DESTDIR="%{buildroot}" make -C
%{arrow_cmake_builddir} install
%endif
-%if %{is_centos_7}
-%define gcc_package devtoolset-11-gcc
-%else
-%define gcc_package gcc
-%endif
-
-%define use_flight (%{_rhel} >= 8 || %{_amzn} >= 2023)
-%define use_gandiva (%{_rhel} >= 8 || %{_amzn} >= 2023)
-%define use_gcs (%{_rhel} >= 8)
%define use_gflags (!%{is_amazon_linux})
## TODO: Enable this when glog stopped depending on gflags-devel.
# %%define use_glog (%%{_rhel} <= 8)
%define use_glog 0
-%define use_mimalloc (%{_rhel} >= 8)
# TODO: Enable this. This works on local but is fragile on GitHub Actions
and
# Travis CI.
# %%define use_s3 (%%{_rhel} >= 8)
%define use_s3 0
-%define use_vala (%{_rhel} >= 8 || %{is_amazon_linux})
%define have_grpc (%{_amzn} >= 2023)
-%define have_lz4_libs (%{_rhel} >= 8 || %{_amzn} >= 2023)
%define have_rapidjson (%{_rhel} != 8)
-%define have_re2 (%{_rhel} >= 8 || %{_amzn} >= 2023)
-%define have_thrift (%{_rhel} >= 8)
%define have_utf8proc (%{_rhel} >= 9 || %{_amzn} >= 2023)
%define enable_glib_doc (%{_rhel} >= 9 || %{is_amazon_linux})
@@ -94,16 +69,14 @@ URL: https://arrow.apache.org/
Source0:
https://dist.apache.org/repos/dist/release/@PACKAGE@/@PACKAGE@-%{version}/apache-@PACKAGE@-%{version}.tar.gz
BuildRequires: bison
-BuildRequires: boost%{boost_version}-devel
+BuildRequires: boost-devel
BuildRequires: brotli-devel
BuildRequires: bzip2-devel
-%if %{use_flight}
BuildRequires: c-ares-devel
-%endif
-BuildRequires: cmake%{cmake_version}
+BuildRequires: cmake
BuildRequires: curl-devel
BuildRequires: flex
-BuildRequires: %{gcc_package}-c++
+BuildRequires: gcc-c++
%if %{use_gflags}
BuildRequires: gflags-devel
%endif
@@ -115,38 +88,27 @@ BuildRequires: glog-devel
BuildRequires: grpc-devel
BuildRequires: grpc-plugins
%endif
-%if %{use_gcs}
BuildRequires: json-devel
-%endif
BuildRequires: libzstd-devel
+BuildRequires: llvm-devel
BuildRequires: lz4-devel %{lz4_requirement}
+BuildRequires: ncurses-devel
BuildRequires: ninja-build
BuildRequires: openssl-devel
BuildRequires: pkgconfig
%if %{have_rapidjson}
BuildRequires: rapidjson-devel
%endif
-%if %{have_re2}
BuildRequires: re2-devel
-%endif
BuildRequires: snappy-devel
-%if %{have_thrift}
BuildRequires: thrift-devel
-%endif
%if %{have_utf8proc}
BuildRequires: utf8proc-devel
%endif
BuildRequires: zlib-devel
-%if %{use_gandiva}
-BuildRequires: llvm-devel
-BuildRequires: ncurses-devel
-%endif
-
BuildRequires: gobject-introspection-devel
-%if %{use_vala}
BuildRequires: vala
-%endif
%description
Apache Arrow is a data processing library for analysis.
@@ -161,21 +123,13 @@ cd cpp
-DARROW_BUILD_UTILITIES=ON \
-DARROW_CSV=ON \
-DARROW_DATASET=ON \
-%if %{use_flight}
-DARROW_FLIGHT=ON \
-DARROW_FLIGHT_SQL=ON \
-%endif
-%if %{use_gandiva}
-DARROW_GANDIVA=ON \
-%endif
-%if %{use_gcs}
-DARROW_GCS=ON \
-%endif
-DARROW_HDFS=ON \
-DARROW_JSON=ON \
-%if %{use_mimalloc}
-DARROW_MIMALLOC=ON \
-%endif
-DARROW_ORC=ON \
-DARROW_PACKAGE_KIND=rpm \
-DARROW_PARQUET=ON \
@@ -200,12 +154,7 @@ cd c_glib
%if %{_amzn} >= 2023
# Do nothing
%else
- %if %{is_centos_7}
-# Meson 0.62.0 or later requires Python 3.7 or later.
-pip3 install 'meson<0.62.0'
- %else
-pip3 install meson
- %endif
+ pip3 install meson
%endif
%if %{enable_glib_doc}
pip3 install gi-docgen
@@ -219,9 +168,7 @@ meson setup build \
%if %{enable_gli
Re: [PR] GH-40735: [Packaging][CentOS] Drop support for CentOS 7 [arrow]
kou commented on code in PR #41395:
URL: https://github.com/apache/arrow/pull/41395#discussion_r1581510792
##
dev/tasks/tasks.yml:
##
@@ -554,12 +554,9 @@ tasks:
"almalinux-8",
"amazon-linux-2023",
"centos-9-stream",
- "centos-8-stream",
- "centos-7"] %}
- {% set is_rhel7_based = (target in ["amazon-linux-2", "centos-7"]) %}
- {% for architecture
- in ["amd64", "arm64"]
- if not (target == "centos-7" and architecture == "arm64") %}
+ "centos-8-stream"] %}
+ {% set is_rhel7_based = (target in ["amazon-linux-2"]) %}
Review Comment:
Could you also remove `is_rhel7_based` entirely?
`amazon-linux-2` isn't supported too.
##
docs/source/developers/continuous_integration/crossbow.rst:
##
@@ -222,7 +222,7 @@ Run only ``conda`` package builds and a Linux one:
.. code::
- $ archery crossbow submit --group conda centos-7
+ $ archery crossbow submit --group conda centos-9
Review Comment:
`centos-9` doesn't exist...
```suggestion
$ archery crossbow submit --group conda almalinux-9
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] feat(csharp/test): implement DuckDb test fixture [arrow-adbc]
CurtHagenlocher commented on code in PR #1781:
URL: https://github.com/apache/arrow-adbc/pull/1781#discussion_r1581508617
##
csharp/test/Apache.Arrow.Adbc.Tests/ImportedDuckDbTests.cs:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using Apache.Arrow.Types;
+using Xunit;
+
+namespace Apache.Arrow.Adbc.Tests
+{
+public class ImportedDuckDbTests : IClassFixture
+{
+readonly DuckDbFixture _duckDb;
+
+public ImportedDuckDbTests(DuckDbFixture duckDb)
+{
+_duckDb = duckDb;
+}
+
+[Fact]
+public void SimpleEndToEndTest()
+{
+using var database = _duckDb.OpenDatabase("test.db");
+using var connection = database.Connect(null);
+using var statement = connection.CreateStatement();
+
+statement.SqlQuery = "CREATE TABLE integers(foo INTEGER, bar
INTEGER);";
+statement.ExecuteUpdate();
+
+statement.SqlQuery = "INSERT INTO integers VALUES (3, 4), (5, 6),
(7, 8);";
+statement.ExecuteUpdate();
+
+statement.SqlQuery = "SELECT * from integers";
+var results = statement.ExecuteQuery();
+
+var schema = results.Stream.Schema;
+Assert.Equal(2, schema.FieldsList.Count);
+Assert.Equal(ArrowTypeId.Int32,
schema.FieldsList[0].DataType.TypeId);
+Assert.Equal(ArrowTypeId.Int32,
schema.FieldsList[1].DataType.TypeId);
+
+var firstBatch = results.Stream.ReadNextRecordBatchAsync().Result;
+Assert.Equal(3, firstBatch.Length);
+Assert.Equal(3, (firstBatch.Column(0) as Int32Array).Values[0]);
+Assert.Equal(5, (firstBatch.Column(0) as Int32Array).Values[1]);
+Assert.Equal(7, (firstBatch.Column(0) as Int32Array).Values[2]);
+
+
+var secondBatch = results.Stream.ReadNextRecordBatchAsync().Result;
Review Comment:
oh, hey, I guess I didn't finish this :/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [CI][Packaging] Nightly test for centos-8-stream-arm64 times out installing arrow-libs [arrow]
kou commented on issue #33260: URL: https://github.com/apache/arrow/issues/33260#issuecomment-2080083142 We don't need to do anything for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] feat(csharp/test): implement DuckDb test fixture [arrow-adbc]
CurtHagenlocher opened a new pull request, #1781: URL: https://github.com/apache/arrow-adbc/pull/1781 Adds support for running tests against DuckDb and implements a simple smoke test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] feat(python): Implement extension type and Schema metadata support [arrow-nanoarrow]
danepitkin commented on code in PR #431: URL: https://github.com/apache/arrow-nanoarrow/pull/431#discussion_r1581480452 ## python/src/nanoarrow/_lib.pyx: ## @@ -967,10 +989,50 @@ cdef class CSchemaBuilder: if self._ptr.release == NULL: ArrowSchemaInit(self._ptr) +@staticmethod +def copy_existing(CSchema existing_schema): +return CSchemaBuilder(existing_schema.__deepcopy__()) Review Comment: nit: I'd prefer a shorter name. WDYT? ```suggestion def copy(CSchema schema): return CSchemaBuilder(schema.__deepcopy__()) ``` ## python/src/nanoarrow/iterator.py: ## @@ -160,6 +164,17 @@ def get_iterator(cls, obj, schema=None): yield from iterator._iter1(0, array.length) def _iter1(self, offset, length): +# Check for and extension type first since this isn't reflected by Review Comment: ```suggestion # Check for an extension type first since this isn't reflected by ``` ## python/src/nanoarrow/schema.py: ## @@ -108,10 +115,73 @@ def create(obj): return TimeUnit(obj) +class ExtensionAccessor: +"""Accessor for extension type parameters""" + +def __init__(self, schema) -> None: +self._schema = schema + +@property +def name(self) -> str: +"""Extension name for this extension type""" +return self._schema._c_schema_view.extension_name + +@property +def metadata(self) -> Union[bytes, None]: +"""Extension metadata for this extension type if present""" +extension_metadata = self._schema._c_schema_view.extension_metadata +return extension_metadata if extension_metadata else None + +@property +def storage(self): +"""Storage type for this extension type""" +metadata = dict(self._schema.metadata.items()) +del metadata[b"ARROW:extension:name"] +if b"ARROW:extension:metadata" in metadata: +del metadata[b"ARROW:extension:metadata"] Review Comment: why do we delete the objects here? ## python/src/nanoarrow/schema.py: ## @@ -108,10 +115,73 @@ def create(obj): return TimeUnit(obj) +class ExtensionAccessor: +"""Accessor for extension type parameters""" + +def __init__(self, schema) -> None: +self._schema = schema + +@property +def name(self) -> str: +"""Extension name for this extension type""" +return self._schema._c_schema_view.extension_name + +@property +def metadata(self) -> Union[bytes, None]: +"""Extension metadata for this extension type if present""" +extension_metadata = self._schema._c_schema_view.extension_metadata +return extension_metadata if extension_metadata else None + +@property +def storage(self): +"""Storage type for this extension type""" +metadata = dict(self._schema.metadata.items()) +del metadata[b"ARROW:extension:name"] +if b"ARROW:extension:metadata" in metadata: +del metadata[b"ARROW:extension:metadata"] + +return Schema(self._schema, metadata=metadata) + + class Schema: -"""The Schema is nanoarrow's high-level data type representation whose scope maps to -that of the ArrowSchema in the Arrow C Data interface. See :func:`schema` for class -details. +"""Create a nanoarrow Schema + +The Schema is nanoarrow's high-level data type representation, encompasing Review Comment: ```suggestion The Schema is nanoarrow's high-level data type representation, encompassing ``` ## python/src/nanoarrow/_lib.pyx: ## @@ -967,10 +989,50 @@ cdef class CSchemaBuilder: if self._ptr.release == NULL: ArrowSchemaInit(self._ptr) +@staticmethod +def copy_existing(CSchema existing_schema): +return CSchemaBuilder(existing_schema.__deepcopy__()) + @staticmethod def allocate(): return CSchemaBuilder(CSchema.allocate()) +def clear_metadata(self): Review Comment: Optional: What if we made this more object oriented? e.g. `CSchemaBuilder.clear_metadata()` vs `CSchemaBuilder.metadata.clear()`. Same with `append_metadata`. Does it make sense to implement this way? If not (or its a lot of work), ignore this comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-40078: [C++] Import/Export ArrowDeviceArrayStream [arrow]
zeroshade commented on PR #40807: URL: https://github.com/apache/arrow/pull/40807#issuecomment-2080046370 Anyone have any further comments here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] ci: Add pipeline support to bundle Go binaries in NuGet packages [arrow-adbc]
CurtHagenlocher merged PR #1730: URL: https://github.com/apache/arrow-adbc/pull/1730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] ci: Add pipeline support to bundle Go binaries in NuGet packages [arrow-adbc]
CurtHagenlocher commented on PR #1730: URL: https://github.com/apache/arrow-adbc/pull/1730#issuecomment-2080040894 Given what @davidhcoe says about switching to v4, I'm going to commit as-is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
zeroshade commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2080039850 We are already using the most recent version of gosnowflake. ADBC v0.11.0 went out using gosnowflake v1.8.0, and the main branch is using v1.9.0 The snowflake ADBC driver hasn't used gosnowflake v1.6.22 since ADBC v0.9.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2080019213 I believe the combination mentioned was just a workaround of the issue that is caused by the old version 1.6.22 of gosnowflake. Can we consider updating the gosnowflake dependency to a more recent version that has already fixed the issue? Appreciate suggesting the workaround but I believe having a more recent version of gosnowflake that already fixed the issue under the hood would be more efficient and beneficial. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41386: [Docs] Document Arrow Go Avro read support [arrow]
conbench-apache-arrow[bot] commented on PR #41393: URL: https://github.com/apache/arrow/pull/41393#issuecomment-2079998546 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 0f56339ee803858b597418aefdabb993def19f48. There were no benchmark performance regressions. 🎉 The [full Conbench report](https://github.com/apache/arrow/runs/24313803524) has more details. It also includes information about 30 possible false positives for unstable benchmarks that are known to sometimes produce them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
zeroshade commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079993072 Since we don't control the Azure ADF UI, you would have to talk to them about being able to try the fix that @davlee1972 mentioned if the UI doesn't provide the ability to try that combination. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-15058: [C++][Python] Native support for UUID [arrow]
rok commented on code in PR #37298: URL: https://github.com/apache/arrow/pull/37298#discussion_r1581448620 ## cpp/src/arrow/extension/CMakeLists.txt: ## @@ -15,10 +15,16 @@ # specific language governing permissions and limitations # under the License. +set(CANONICAL_EXTENSION_TESTS uuid_test.cc) + +if(ARROW_JSON) Review Comment: See my reply below. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-15058: [C++][Python] Native support for UUID [arrow]
rok commented on code in PR #37298:
URL: https://github.com/apache/arrow/pull/37298#discussion_r1581445564
##
cpp/src/arrow/extension_type.cc:
##
@@ -145,12 +146,17 @@ static void CreateGlobalRegistry() {
g_registry = std::make_shared();
#ifdef ARROW_JSON
Review Comment:
`fixed_shape_tensor` uses rapidjson to serialize and deserialize metadata
(see `FixedShapeTensorType::Serialize` and `FixedShapeTensorType::Deserialize`)
of the type so we need `ARROW_JSON` to build it. UUID has no metadata to
serialize so we don't require JSON. It's annoying that for some builds we get
all extensions and for some only a subset, but I don't have a better idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] GH-41358: [R] Support join "na_matches" argument [arrow]
nealrichardson commented on PR #41372: URL: https://github.com/apache/arrow/pull/41372#issuecomment-2079983661 CI failure appears unrelated: https://github.com/apache/arrow/issues/41398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079977533 Was referring to Azure ADF UI as @ruowan shared above, and it relies on the driver to construct the host URL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]
mapleFU commented on code in PR #41335:
URL: https://github.com/apache/arrow/pull/41335#discussion_r1581424142
##
cpp/src/arrow/acero/query_context.cc:
##
@@ -23,6 +23,36 @@ namespace arrow {
using arrow::internal::CpuInfo;
namespace acero {
+namespace internal {
+
+int64_t GetTempStackSizeFromEnvVar() {
+ auto maybe_env_value = arrow::internal::GetEnvVar(kTempStackSizeEnvVar);
+ if (!maybe_env_value.ok()) {
+return kDefaultTempStackSize;
+ }
+ auto env_value = *std::move(maybe_env_value);
+ if (env_value.empty()) {
+return kDefaultTempStackSize;
+ }
+
+ int64_t temp_stack_size = 0;
+ size_t length = 0;
+ bool exception = false;
+ try {
+temp_stack_size = std::stoll(env_value.c_str(), &length);
+ } catch (const std::exception&) {
+exception = true;
+ }
+ if (length != env_value.length() || exception || temp_stack_size <= 0) {
+ARROW_LOG(WARNING) << "Invalid temp stack size provided in " <<
kTempStackSizeEnvVar
+ << ". Using default temp stack size: " <<
kDefaultTempStackSize;
+return kDefaultTempStackSize;
+ }
+ return temp_stack_size;
Review Comment:
Should that fill better only in debug mode?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
zeroshade commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079963796 Does our C# binding provide a UI? or is that something external to the ADBC project? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079961188 @zeroshade Thanks for checking but from the UI, we don't have control to try the combination mentioned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
zeroshade commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079950427 @kiyoungoh were you able to try using the approach that @davlee1972 mentioned in https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2078612172 which he said was able to work? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079928027 The team owning the driver was not able to repro the issue with their latest version and closed the ticket. https://github.com/snowflakedb/gosnowflake/issues/1033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079919293 Wasn't the issue fixed with 1.6.25? https://docs.snowflake.com/en/release-notes/clients-drivers/golang-2023#:~:text=Fixed%20an%20issue%20with%20OCSP%20fallback%20requests%20in%20PrivateLink%20environments.  https://github.com/apache/arrow-adbc/issues/1422#:~:text=Returns%20with%20these%20errors%3A  Can we test with 1.6.25 or any version above and see if that helps? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support casting `StringArray`/`BinaryArray` --> `StringView` / `BinaryView` [arrow-rs]
alamb commented on PR #5686: URL: https://github.com/apache/arrow-rs/pull/5686#issuecomment-2079858373 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support casting `StringArray`/`BinaryArray` --> `StringView` / `BinaryView` [arrow-rs]
alamb merged PR #5686: URL: https://github.com/apache/arrow-rs/pull/5686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41367: [C++] Replace [[maybe_unused]] with Arrow macro [arrow]
WillAyd commented on code in PR #41359:
URL: https://github.com/apache/arrow/pull/41359#discussion_r1581355489
##
cpp/src/arrow/device.h:
##
@@ -150,9 +150,10 @@ class ARROW_EXPORT Device : public
std::enable_shared_from_this,
/// @param release_fn a function to call during destruction, `nullptr` or
///a no-op function can be passed to indicate ownership is maintained
///externally
- virtual Result> WrapStream(
- [[maybe_unused]] void* device_stream,
- [[maybe_unused]] Stream::release_fn_t release_fn) {
+ virtual Result> WrapStream(void* device_stream,
+ Stream::release_fn_t
release_fn) {
+ARROW_UNUSED(device_stream);
+ARROW_UNUSED(release_fn);
Review Comment:
Just pushed up this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] Snowflake driver report invalid JWT token for the keypair auth with privateLink accountIdentifer [arrow-adbc]
kiyoungoh commented on issue #1777: URL: https://github.com/apache/arrow-adbc/issues/1777#issuecomment-2079817423 Hi Team, do we have any ETA to fix the issue? Truly appreciate your time and consideration in advance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [C++] Enable using the GCS+GRPC plugin with Arrow [arrow]
benibus commented on issue #40254: URL: https://github.com/apache/arrow/issues/40254#issuecomment-2079807518 This is tangential, but upgrading gRPC to v1.61.0 or later would also allow us to use the new [OpenTelemetry plugin](https://github.com/grpc/grpc/blob/b361574a0b647b81129ed33749aaa74585acd779/include/grpcpp/ext/otel_plugin.h). I think we could handle the version bump(s) in a standalone PR first - assuming we wouldn't need to target a lower version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] csharp: imported drivers can leak transient objects on errors [arrow-adbc]
CurtHagenlocher commented on issue #1774: URL: https://github.com/apache/arrow-adbc/issues/1774#issuecomment-2079785707 Closed by #1776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41375: [C#] Move to .NET 8.0 [arrow]
conbench-apache-arrow[bot] commented on PR #41376: URL: https://github.com/apache/arrow/pull/41376#issuecomment-2079785458 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 2710626b234d5e387a3c63988ca5899c70547dcf. There were no benchmark performance regressions. 🎉 The [full Conbench report](https://github.com/apache/arrow/runs/24309514782) has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix(csharp): imported drivers have the potential for a lot of memory leaks [arrow-adbc]
CurtHagenlocher merged PR #1776: URL: https://github.com/apache/arrow-adbc/pull/1776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] GH-41282: [Dev] Always prompt next major version on merge script if it exists [arrow]
conbench-apache-arrow[bot] commented on PR #41305: URL: https://github.com/apache/arrow/pull/41305#issuecomment-2079780742 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 6cbdec55b6bbaee3f00a42541bf89998452a3718. There were no benchmark performance regressions. 🎉 The [full Conbench report](https://github.com/apache/arrow/runs/24309396491) has more details. It also includes information about 19 possible false positives for unstable benchmarks that are known to sometimes produce them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [CI] macOS 11 C# job fails to install python dependencies [arrow]
raulcd commented on issue #41390: URL: https://github.com/apache/arrow/issues/41390#issuecomment-2079777844 Issue resolved by pull request 41392 https://github.com/apache/arrow/pull/41392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
