[jira] [Created] (ARROW-18385) [Java]
Jacob Wujciak-Jens created ARROW-18385: -- Summary: [Java] Key: ARROW-18385 URL: https://issues.apache.org/jira/browse/ARROW-18385 Project: Apache Arrow Issue Type: Wish Components: Java Reporter: Jacob Wujciak-Jens Fix For: 11.0.0 Attachments: image.png While verifying 10.0.1 I came across this java test error that is caused by a mismatch in the ordering of the JSON metadata description (see attached image) ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.177 s <<< FAILURE! - in org.apache.arrow.adapter.jdbc.JdbcToArrowCommentMetadataTest [ERROR] org.apache.arrow.adapter.jdbc.JdbcToArrowCommentMetadataTest.schemaCommentWithDatabaseMetadata Time elapsed: 0.141 s <<< FAILURE! org.opentest4j.AssertionFailedError: cc [~lidavidm] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18188) [CI] CUDA nightly docker upload fails due to wrong tag
Jacob Wujciak-Jens created ARROW-18188: -- Summary: [CI] CUDA nightly docker upload fails due to wrong tag Key: ARROW-18188 URL: https://issues.apache.org/jira/browse/ARROW-18188 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 11.0.0 Due to the different CUDA version required for this job vs the version in .env the CUDA envvar needs to be set in the push step too to generate the correct tag. https://github.com/ursacomputing/crossbow/actions/runs/3341920350/jobs/5533611188#step:7:9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18016) [CI] Add sccache to more jobs
Jacob Wujciak-Jens created ARROW-18016: -- Summary: [CI] Add sccache to more jobs Key: ARROW-18016 URL: https://issues.apache.org/jira/browse/ARROW-18016 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 Building on the work in [ARROW-17021] we can now activate sccache on more builds and save more time! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17990) [CI][R][C++] R macOS 10.13 build due to bmi2 changes
Jacob Wujciak-Jens created ARROW-17990:
--
Summary: [CI][R][C++] R macOS 10.13 build due to bmi2 changes
Key: ARROW-17990
URL: https://issues.apache.org/jira/browse/ARROW-17990
Project: Apache Arrow
Issue Type: Bug
Components: C++, Continuous Integration, R
Reporter: Jacob Wujciak-Jens
Fix For: 10.0.0
After the changes in [ARROW-15768] the macos 10.13 build is failing:
{code}
/Users/voltrondata/tmp/hbtmp/apache-arrow-20221011-20438-19rxs8w/cpp/src/parquet/level_conversion_inc.h:278:10:
error: always_inline function '_pext_u64' requires target feature 'bmi2', but
would be inlined into function 'ExtractBits' that is compiled without support
for 'bmi2'
{code}
https://github.com/ursacomputing/crossbow/actions/runs/3225397138/jobs/5278904370#step:13:7016
cc: ~kou
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17972) [CI] Update cuda docker jobs
Jacob Wujciak-Jens created ARROW-17972: -- Summary: [CI] Update cuda docker jobs Key: ARROW-17972 URL: https://issues.apache.org/jira/browse/ARROW-17972 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 The cuda job config in docker-compose.yml are outdated and do not work anymore. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17876) [R][CI] Remove ubuntu-18.04 from nixlibs & prebuilt binaries
Jacob Wujciak-Jens created ARROW-17876: -- Summary: [R][CI] Remove ubuntu-18.04 from nixlibs & prebuilt binaries Key: ARROW-17876 URL: https://issues.apache.org/jira/browse/ARROW-17876 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 The new dts compiled centos-7 binaries ([ARROW-17594]) should be able to replace the ubuntu-18.04 binaries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17854) [CI][Developer] Hoste preview docs on S3
Jacob Wujciak-Jens created ARROW-17854: -- Summary: [CI][Developer] Hoste preview docs on S3 Key: ARROW-17854 URL: https://issues.apache.org/jira/browse/ARROW-17854 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Developer Tools Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 Hosting on Github Pages as implemented in [ARROW-12958] is unsustainable due to the size of the arrow docs (~ 200mb). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17849) [R][Docs] Document changes due to C++17 for centos-7 users
Jacob Wujciak-Jens created ARROW-17849: -- Summary: [R][Docs] Document changes due to C++17 for centos-7 users Key: ARROW-17849 URL: https://issues.apache.org/jira/browse/ARROW-17849 Project: Apache Arrow Issue Type: Improvement Components: Documentation, R Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 With the switch to C++17 centos 7 users need to install and enable devtoolset (and possibly change makevars) to be able to compile the R package, even when using the libarrow binary. This should be documented and noted in the release notes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17795) [C++][R] Using ARROW_ZSTD_USE_SHARED fails
Jacob Wujciak-Jens created ARROW-17795: -- Summary: [C++][R] Using ARROW_ZSTD_USE_SHARED fails Key: ARROW-17795 URL: https://issues.apache.org/jira/browse/ARROW-17795 Project: Apache Arrow Issue Type: Improvement Components: C++, R Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 See zulip discussion [here|https://ursalabs.zulipchat.com/#narrow/stream/180245-dev/topic/zstd.20cmake.20changes] Changes to the find zstd module cause failure when ARROW_ZSTD_USE_SHARED is used -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17782) [C++][R] R package not building on macos 10.13 with C++17 std lib
Jacob Wujciak-Jens created ARROW-17782: -- Summary: [C++][R] R package not building on macos 10.13 with C++17 std lib Key: ARROW-17782 URL: https://issues.apache.org/jira/browse/ARROW-17782 Project: Apache Arrow Issue Type: Improvement Components: C++, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 The R package also needs `-D_LIBCPP_DISABLE_AVAILABILITY` to be able to be compiled on macos 10.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17726) [CI] Enable sccache on more builds
Jacob Wujciak-Jens created ARROW-17726: -- Summary: [CI] Enable sccache on more builds Key: ARROW-17726 URL: https://issues.apache.org/jira/browse/ARROW-17726 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 As a follow up to [ARROW-17021]. Enabling sccache should be as easy as adding the install script to the relevant docker image and the sccache env to the docker-compose service. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17621) [CI] Audit workflows
Jacob Wujciak-Jens created ARROW-17621: -- Summary: [CI] Audit workflows Key: ARROW-17621 URL: https://issues.apache.org/jira/browse/ARROW-17621 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 Set minimal permissions for token, check for out-dated actions, pin shas etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17616) [CI][Java] Java nightly upload job fails after introduction of pruning
Jacob Wujciak-Jens created ARROW-17616: -- Summary: [CI][Java] Java nightly upload job fails after introduction of pruning Key: ARROW-17616 URL: https://issues.apache.org/jira/browse/ARROW-17616 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration, Java Reporter: Jacob Wujciak-Jens The nightly java upload job has been failing ever since [ARROW-17293]. https://github.com/apache/arrow/actions/workflows/java_nightly.yml It looks like the "Build Repository" step clashes with the synced repo? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17477) [CI][Docs] Document Docs PR Preview
Jacob Wujciak-Jens created ARROW-17477: -- Summary: [CI][Docs] Document Docs PR Preview Key: ARROW-17477 URL: https://issues.apache.org/jira/browse/ARROW-17477 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Documentation Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 Document the changes from [ARROW-12958] here: https://arrow.apache.org/docs/developers/documentation.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17422) [C++][CI] Travis builds are missing dependencies
Jacob Wujciak-Jens created ARROW-17422: -- Summary: [C++][CI] Travis builds are missing dependencies Key: ARROW-17422 URL: https://issues.apache.org/jira/browse/ARROW-17422 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 [ARROW-17394] added new system dependencies that are missing in the linux builds. E.g. https://github.com/ursacomputing/crossbow/runs/7834744542 https://github.com/ursacomputing/crossbow/runs/7834709298 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17421) [C++] CUDA on Windows fails to buil
Jacob Wujciak-Jens created ARROW-17421: -- Summary: [C++] CUDA on Windows fails to buil Key: ARROW-17421 URL: https://issues.apache.org/jira/browse/ARROW-17421 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 9.0.0 Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 While working on the vcpkg port for 9.0.0 I noticed that Arrow with ARROW_CUDA does not build on Windows due to an issue with unique_ptr in CudaDevice. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17353) [Release] R libarrow binaries have the wrong version number
Jacob Wujciak-Jens created ARROW-17353: -- Summary: [Release] R libarrow binaries have the wrong version number Key: ARROW-17353 URL: https://issues.apache.org/jira/browse/ARROW-17353 Project: Apache Arrow Issue Type: Bug Components: Developer Tools Affects Versions: 9.0.0 Reporter: Jacob Wujciak-Jens Fix For: 10.0.0 The libarrow binaries that are uploaded during the release process have the wrong version number. This is an issue with the submit binaries script/r-binary-packages job. The arrow version should be picked up by the job even if not passed explicitly as a custom param. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17293) [Java][CI] Prune java nightly builds
Jacob Wujciak-Jens created ARROW-17293: -- Summary: [Java][CI] Prune java nightly builds Key: ARROW-17293 URL: https://issues.apache.org/jira/browse/ARROW-17293 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Java Reporter: Jacob Wujciak-Jens Currently we are accumulating a huge number of nightly java jars. We should prune them to keep max. 14 versions around. (see r_nightly.yml) It might also be nice to always rename/copy the most recent jars to something fixed so there is no need to update your local config to always have the newest version? (but up to the java users if this is necessary/worth it). cc [~dsusanibara] [~ljw1001] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17282) [Python] flake8 update fails linter CI
Jacob Wujciak-Jens created ARROW-17282: -- Summary: [Python] flake8 update fails linter CI Key: ARROW-17282 URL: https://issues.apache.org/jira/browse/ARROW-17282 Project: Apache Arrow Issue Type: Improvement Components: Python Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 flake8 > 5.0.0 uses pycodestyle 2.9.0 which added a few new thing including "space after keyword" which causes the linting to fail in CI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17281) [C++] Windows reports aggregated L-Cache sizes
Jacob Wujciak-Jens created ARROW-17281:
--
Summary: [C++] Windows reports aggregated L-Cache sizes
Key: ARROW-17281
URL: https://issues.apache.org/jira/browse/ARROW-17281
Project: Apache Arrow
Issue Type: Bug
Components: C++
Affects Versions: 9.0.0
Reporter: Jacob Wujciak-Jens
On Windows 10 Version 10.0.19044 Build 19044 {{CpuInfo::CacheLevel}} over
reports the L-cache size:
640K for L1
10M for L2
It seems that windows is reporting the total size of cache for each core
aggregated as one value. Possibly due to "intel smart cache" cache sharing?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17236) [Release][R] Add verification for R binaries to rc script
Jacob Wujciak-Jens created ARROW-17236: -- Summary: [Release][R] Add verification for R binaries to rc script Key: ARROW-17236 URL: https://issues.apache.org/jira/browse/ARROW-17236 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 As the R binaries are now part of the release process they need to be added to verify-release-candidate.sh See https://github.com/apache/arrow/pull/13622#issuecomment-1191006292 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17234) [Release][R] Add r-binary-packages to packaging group
Jacob Wujciak-Jens created ARROW-17234: -- Summary: [Release][R] Add r-binary-packages to packaging group Key: ARROW-17234 URL: https://issues.apache.org/jira/browse/ARROW-17234 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 10.0.0 r-binary-packages is only in nightly-packaging and missing from the release relevant packaging group. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17211) [Java] Fix java-jar nightly on gh & self-hosted runners
Jacob Wujciak-Jens created ARROW-17211:
--
Summary: [Java] Fix java-jar nightly on gh & self-hosted runners
Key: ARROW-17211
URL: https://issues.apache.org/jira/browse/ARROW-17211
Project: Apache Arrow
Issue Type: Bug
Components: Continuous Integration, Java
Reporter: Jacob Wujciak-Jens
Assignee: Jacob Wujciak-Jens
Fix For: 9.0.0
[ARROW-16943] added clean up to {{java_full_build.sh}} to fix issues with
multiple jars when the job was run on a self-hosted (aka non-ephemeral) runner.
This does fails when {{.~/m2}} does not exists.
I marked this as a blocker because this prevents us from building the release
Jars.
cc: [~kszucs] [~raulcd]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17206) [R] Skip test to fix snappy sanitizer issue
Jacob Wujciak-Jens created ARROW-17206: -- Summary: [R] Skip test to fix snappy sanitizer issue Key: ARROW-17206 URL: https://issues.apache.org/jira/browse/ARROW-17206 Project: Apache Arrow Issue Type: Bug Components: R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Known bug with snappy in a new test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17034) [C++] Enable compiler caching for ThirdpartyToolchain.cmake
Jacob Wujciak-Jens created ARROW-17034:
--
Summary: [C++] Enable compiler caching for
ThirdpartyToolchain.cmake
Key: ARROW-17034
URL: https://issues.apache.org/jira/browse/ARROW-17034
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Jacob Wujciak-Jens
Assignee: Jacob Wujciak-Jens
Fix For: 9.0.0
The vars {{CMAKE_C_COMPILER_LAUNCHER}} and {{CMAKE_CXX_COMPILERLAUNCHER}} are
available in ThirdpartyToolchain.cmake but not used. They should be passed on
as cmake args when set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-17021) [C++][R][CI] Enable use of sccache in crossbow
Jacob Wujciak-Jens created ARROW-17021: -- Summary: [C++][R][CI] Enable use of sccache in crossbow Key: ARROW-17021 URL: https://issues.apache.org/jira/browse/ARROW-17021 Project: Apache Arrow Issue Type: Task Components: C++, Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 sccache is ccache with a cloud storage back-end and allows us to circumvent the issue of cross branch caching in crossbow (not possible by design). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-17001) [Release][R] Use apache artifactory for libarrow binaries.
Jacob Wujciak-Jens created ARROW-17001: -- Summary: [Release][R] Use apache artifactory for libarrow binaries. Key: ARROW-17001 URL: https://issues.apache.org/jira/browse/ARROW-17001 Project: Apache Arrow Issue Type: Sub-task Components: R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Move the pre-built libarrow binaries for the release version from nightlies to the apache artifactory and add this to the release process/scripts. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-16901) [R][CI] Prune R nightly builds
Jacob Wujciak-Jens created ARROW-16901: -- Summary: [R][CI] Prune R nightly builds Key: ARROW-16901 URL: https://issues.apache.org/jira/browse/ARROW-16901 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 We only want to keep 14days/versions and remove everything else -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16899) [R][CI] R nightly builds used old libarrow
Jacob Wujciak-Jens created ARROW-16899:
--
Summary: [R][CI] R nightly builds used old libarrow
Key: ARROW-16899
URL: https://issues.apache.org/jira/browse/ARROW-16899
Project: Apache Arrow
Issue Type: Bug
Components: Continuous Integration, R
Reporter: Jacob Wujciak-Jens
Assignee: Jacob Wujciak-Jens
Fix For: 9.0.0
We had not set {{ARROW_R_DEV}} so there was no way to see that the builds where
using an old libarrow version from rwinlib.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (ARROW-16840) [CI] replace actions/setup-ruby with ruby/setup-ruby
Jacob Wujciak-Jens created ARROW-16840: -- Summary: [CI] replace actions/setup-ruby with ruby/setup-ruby Key: ARROW-16840 URL: https://issues.apache.org/jira/browse/ARROW-16840 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 [actions/setup-ruby|https://github.com/actions/setup-ruby] is deprecated and should no longer be used. [rub/setup-ruby|https://github.com/ruby/setup-ruby] is the maintained version -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16803) [R][CI] Fix caching for R mingw build
Jacob Wujciak-Jens created ARROW-16803: -- Summary: [R][CI] Fix caching for R mingw build Key: ARROW-16803 URL: https://issues.apache.org/jira/browse/ARROW-16803 Project: Apache Arrow Issue Type: Bug Components: Continuous Integration, R Affects Versions: 8.0.0 Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Caching is not working for the libarrow mingw builds due to overlapping cache keys: https://github.com/apache/arrow/runs/6819961123?check_suite_focus=true#step:21:2 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16708) [Dev] Replace basic auth with token auth for JIRA
Jacob Wujciak-Jens created ARROW-16708: -- Summary: [Dev] Replace basic auth with token auth for JIRA Key: ARROW-16708 URL: https://issues.apache.org/jira/browse/ARROW-16708 Project: Apache Arrow Issue Type: Sub-task Components: Developer Tools Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Jira allows for the creation of [PATs|https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html] to use in scripts and basic auth is [deprecated|https://developer.atlassian.com/cloud/jira/platform/deprecation-notice-basic-auth-and-cookie-based-auth/] in Jira Cloud. The Jira python package already supports token authentication with minimal changes required. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16689) [CI] Improve R Nightly Workflow
Jacob Wujciak-Jens created ARROW-16689: -- Summary: [CI] Improve R Nightly Workflow Key: ARROW-16689 URL: https://issues.apache.org/jira/browse/ARROW-16689 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Only upload if all tests succeed, improve overall polish, and improve documentation. Add ubuntu 22.04 binary (see ARROW-16678). -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16605) [CI][R] Fix revdep Crossbow job
Jacob Wujciak-Jens created ARROW-16605: -- Summary: [CI][R] Fix revdep Crossbow job Key: ARROW-16605 URL: https://issues.apache.org/jira/browse/ARROW-16605 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 The revdep Crossbow job is currently not functioning correctly. This led to changed behaviour affecting a revdep with the 8.0.0 release, requiring a patch after initial submission. cc: [~jonkeane] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16594) [R] Consistently use "getOption" to set nightly repo
Jacob Wujciak-Jens created ARROW-16594:
--
Summary: [R] Consistently use "getOption" to set nightly repo
Key: ARROW-16594
URL: https://issues.apache.org/jira/browse/ARROW-16594
Project: Apache Arrow
Issue Type: Wish
Components: R
Reporter: Jacob Wujciak-Jens
Fix For: 9.0.0
In {{install_arrow}} the nightly repo url is set via
{{getOption("arrow.dev_repo", "https;//...")}}
this should also be the case in {{winlibs.R}} and {{nixlibs.R}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (ARROW-16507) [CI][C++] Use system gtest with numba/conda
Jacob Wujciak-Jens created ARROW-16507: -- Summary: [CI][C++] Use system gtest with numba/conda Key: ARROW-16507 URL: https://issues.apache.org/jira/browse/ARROW-16507 Project: Apache Arrow Issue Type: Bug Components: C++, Continuous Integration Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 With the change in ARROW-1490 removal of gtest is not needed and breaks the build. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16490) [C++] Fix gtest source
Jacob Wujciak-Jens created ARROW-16490: -- Summary: [C++] Fix gtest source Key: ARROW-16490 URL: https://issues.apache.org/jira/browse/ARROW-16490 Project: Apache Arrow Issue Type: Bug Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 There is an issue where the vcpkg gtest version and the thridparty toolchain version differ and arrow is build with one and linked against the other. This is caused by logic introduced a long time ago to fix issues with conda. See also [here|https://github.com/apache/arrow/pull/13063] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16489) [R] wrong encoding causes parsing error
Jacob Wujciak-Jens created ARROW-16489:
--
Summary: [R] wrong encoding causes parsing error
Key: ARROW-16489
URL: https://issues.apache.org/jira/browse/ARROW-16489
Project: Apache Arrow
Issue Type: Bug
Components: R
Reporter: Jacob Wujciak-Jens
Assignee: Jacob Wujciak-Jens
Fix For: 9.0.0
The encoding of {{latin1.R}} was changed from utf8 and causes parsing errors on
linux:
https://github.com/ursacomputing/crossbow/runs/6316672224?check_suite_focus=true#step:5:3416
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (ARROW-16433) [Release][C++] parquet-arrow-test test fails on windows
Jacob Wujciak-Jens created ARROW-16433:
--
Summary: [Release][C++] parquet-arrow-test test fails on windows
Key: ARROW-16433
URL: https://issues.apache.org/jira/browse/ARROW-16433
Project: Apache Arrow
Issue Type: Bug
Components: C++
Reporter: Jacob Wujciak-Jens
Fix For: 8.0.0
While verifying RC1 on windows this error happened reproducible using
"verify-release-candidate.bat"
{code:java}
The following tests FAILED:
59 - parquet-arrow-test (Failed)
C:/tmp/arrow-verify-release/apache-arrow-8.0.0/cpp/src/parquet/arrow/arrow_reader_writer_test.cc(1236):
error: Expected: this->ReadAndCheckSingleColumnFile(*values) doesn't generate
new fatal failures in the current thread.
Actual: it does.
[ FAILED ] TestInt96ParquetIO.ReadIntoTimestamp (0 ms)
{code}
cc [~apitrou]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (ARROW-16432) [Docs] Update verify RC instructions - JDK8
Jacob Wujciak-Jens created ARROW-16432: -- Summary: [Docs] Update verify RC instructions - JDK8 Key: ARROW-16432 URL: https://issues.apache.org/jira/browse/ARROW-16432 Project: Apache Arrow Issue Type: Improvement Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 The PPA for Oracle JDK is discontinued due to the license change https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16405) [R][CI] Implement alternative nightly hosting
Jacob Wujciak-Jens created ARROW-16405: -- Summary: [R][CI] Implement alternative nightly hosting Key: ARROW-16405 URL: https://issues.apache.org/jira/browse/ARROW-16405 Project: Apache Arrow Issue Type: Sub-task Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16406) [Docs][R] Update documentation with new nightly location
Jacob Wujciak-Jens created ARROW-16406: -- Summary: [Docs][R] Update documentation with new nightly location Key: ARROW-16406 URL: https://issues.apache.org/jira/browse/ARROW-16406 Project: Apache Arrow Issue Type: Sub-task Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 See ARROW-16405 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16404) [R][CI] Research alternative binary hosting
Jacob Wujciak-Jens created ARROW-16404: -- Summary: [R][CI] Research alternative binary hosting Key: ARROW-16404 URL: https://issues.apache.org/jira/browse/ARROW-16404 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Find a way to host the nightly binaries in some form of artifactory under ASF umbrella. Python wheels are hosted fury.io Maybe possible to create repo via ghcr.io? See [pak|https://github.com/r-lib/pak], if so is there possibility to do this for other components? -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16403) [R] Test Changes and integrate changes into nightly process
Jacob Wujciak-Jens created ARROW-16403: -- Summary: [R] Test Changes and integrate changes into nightly process Key: ARROW-16403 URL: https://issues.apache.org/jira/browse/ARROW-16403 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16402) [R] Modify Crossbow tasks
Jacob Wujciak-Jens created ARROW-16402: -- Summary: [R] Modify Crossbow tasks Key: ARROW-16402 URL: https://issues.apache.org/jira/browse/ARROW-16402 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Add/modify Crossbow tasks as analyzed in ARROW-16401 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16401) [R] Dissect arrow-r-nightly workflow into Crossbow tasks
Jacob Wujciak-Jens created ARROW-16401: -- Summary: [R] Dissect arrow-r-nightly workflow into Crossbow tasks Key: ARROW-16401 URL: https://issues.apache.org/jira/browse/ARROW-16401 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Analyze the different task in the [existing workflow|https://github.com/ursa-labs/arrow-r-nightly/blob/master/.github/workflows/build-and-test-all.yml] and look for redundancies and missing functionality in Crossbow and PR jobs. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16400) [R][CI] Integrate arrow-r nightly/release builds into Crossbow
Jacob Wujciak-Jens created ARROW-16400: -- Summary: [R][CI] Integrate arrow-r nightly/release builds into Crossbow Key: ARROW-16400 URL: https://issues.apache.org/jira/browse/ARROW-16400 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 9.0.0 Move the build jobs from [ursa-labs/arrow-r-nightly|[https://github.com/ursa-labs/arrow-r-nightly|https://gtihub.com/ursa-labs/arrow-r-nightly]] into Crossbow and integrate them into the nightly and release process. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16370) [Docs] Document Crossbow builds outside of code
Jacob Wujciak-Jens created ARROW-16370: -- Summary: [Docs] Document Crossbow builds outside of code Key: ARROW-16370 URL: https://issues.apache.org/jira/browse/ARROW-16370 Project: Apache Arrow Issue Type: Sub-task Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 Find a way to document the different builds that are available on-demand and run regularly (nightly) outside of docker-compose.yml & tasks.yml. Something like a "build matrix", ideally we would also have a "supported-config matrix" to compare against each other. CC [~raulcd] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16368) [Docs] How to update PR CI
Jacob Wujciak-Jens created ARROW-16368: -- Summary: [Docs] How to update PR CI Key: ARROW-16368 URL: https://issues.apache.org/jira/browse/ARROW-16368 Project: Apache Arrow Issue Type: Sub-task Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16369) [Docs] How to update & test crossbow jobs
Jacob Wujciak-Jens created ARROW-16369: -- Summary: [Docs] How to update & test crossbow jobs Key: ARROW-16369 URL: https://issues.apache.org/jira/browse/ARROW-16369 Project: Apache Arrow Issue Type: Sub-task Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16367) [Docs] CI FAQ Dev
Jacob Wujciak-Jens created ARROW-16367: -- Summary: [Docs] CI FAQ Dev Key: ARROW-16367 URL: https://issues.apache.org/jira/browse/ARROW-16367 Project: Apache Arrow Issue Type: Sub-task Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 General info, conda feedstocks etc. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16366) [Docs] CI FAQ User Section
Jacob Wujciak-Jens created ARROW-16366: -- Summary: [Docs] CI FAQ User Section Key: ARROW-16366 URL: https://issues.apache.org/jira/browse/ARROW-16366 Project: Apache Arrow Issue Type: Sub-task Components: Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 - where can I find the results of running one of these tasks locally? - why might I want to use archery docker run vs. docker run? dev - ... -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16365) [Docs][CI] How to run CI builds locally - MacOS
Jacob Wujciak-Jens created ARROW-16365: -- Summary: [Docs][CI] How to run CI builds locally - MacOS Key: ARROW-16365 URL: https://issues.apache.org/jira/browse/ARROW-16365 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16364) [Docs][CI] How to run CI builds locally - Linux
Jacob Wujciak-Jens created ARROW-16364: -- Summary: [Docs][CI] How to run CI builds locally - Linux Key: ARROW-16364 URL: https://issues.apache.org/jira/browse/ARROW-16364 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16363) [Docs][CI] How to run CI builds locally - Windows
Jacob Wujciak-Jens created ARROW-16363: -- Summary: [Docs][CI] How to run CI builds locally - Windows Key: ARROW-16363 URL: https://issues.apache.org/jira/browse/ARROW-16363 Project: Apache Arrow Issue Type: Sub-task Components: Continuous Integration, Documentation Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16338) [CI] Update azure windows image as vs2017-win2016 is retired
Jacob Wujciak-Jens created ARROW-16338: -- Summary: [CI] Update azure windows image as vs2017-win2016 is retired Key: ARROW-16338 URL: https://issues.apache.org/jira/browse/ARROW-16338 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Developer Tools Reporter: Jacob Wujciak-Jens See [https://devblogs.microsoft.com/devops/hosted-pipelines-image-deprecation/#windows] We should replace it with windows-2019 I am not sure if this will require other changes in the setup, will open PR to test it. cc [~raulcd] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16308) [CI] Upgrade windows runner version as windows-2016 is deprecated.
Jacob Wujciak-Jens created ARROW-16308: -- Summary: [CI] Upgrade windows runner version as windows-2016 is deprecated. Key: ARROW-16308 URL: https://issues.apache.org/jira/browse/ARROW-16308 Project: Apache Arrow Issue Type: Task Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 8.0.0 "The windows-2016 environment is deprecated and will be removed on April 1st, 2022" So we need to upgrade all runners to at least windows-2019 (or 2022/latest ?) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16285) [CI][Python} Enable skipped kartothek integration tests
Jacob Wujciak-Jens created ARROW-16285: -- Summary: [CI][Python} Enable skipped kartothek integration tests Key: ARROW-16285 URL: https://issues.apache.org/jira/browse/ARROW-16285 Project: Apache Arrow Issue Type: Task Components: Continuous Integration, Python Reporter: Jacob Wujciak-Jens -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (ARROW-16265) [C++][Docs]
Jacob Wujciak-Jens created ARROW-16265:
--
Summary: [C++][Docs]
Key: ARROW-16265
URL: https://issues.apache.org/jira/browse/ARROW-16265
Project: Apache Arrow
Issue Type: Improvement
Components: C++, Documentation
Reporter: Jacob Wujciak-Jens
Fix For: 9.0.0
Add a note
[here|https://arrow.apache.org/docs/developers/cpp/windows.html#windows-dependency-resolution-issues]
that {{ZSTD_MSVC_STATIC_LIB_SUFFIX}} will be automatically set to {{_static
}}if MCVS is used (see [PR #7388|https://github.com/apache/arrow/pull/7388])
and the option needs to be passed as{{ -DZSTD_MSVC_STATIC_LIB_SUFFIX=. }}Maybe
a Log message in case of set suffix would also make sense.
I ran into this issue due to the current VCPKG version deviating from the
zstd_static.lib naming scheme in a recent commit,
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (ARROW-16198) [CI][Packaging][Python] Update VCPKG version
Jacob Wujciak-Jens created ARROW-16198: -- Summary: [CI][Packaging][Python] Update VCPKG version Key: ARROW-16198 URL: https://issues.apache.org/jira/browse/ARROW-16198 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, Packaging, Python Reporter: Jacob Wujciak-Jens Assignee: Jacob Wujciak-Jens Fix For: 8.0.0 While testing out the windows docker images I cam across this issue: [https://github.com/microsoft/vcpkg/issues/2518] This probably has not affected nightlies because of cached builds (?) Some of the port patches (addressing arm compatibility) are no longer needed either. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16190) [CI][R] Implement CI on Apple M1 for R
Jacob Wujciak-Jens created ARROW-16190: -- Summary: [CI][R] Implement CI on Apple M1 for R Key: ARROW-16190 URL: https://issues.apache.org/jira/browse/ARROW-16190 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration, R Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ARROW-16189) [CI][C++] Implement CI on Apple M1 for C++
Jacob Wujciak-Jens created ARROW-16189: -- Summary: [CI][C++] Implement CI on Apple M1 for C++ Key: ARROW-16189 URL: https://issues.apache.org/jira/browse/ARROW-16189 Project: Apache Arrow Issue Type: Improvement Components: C++, Continuous Integration Reporter: Jacob Wujciak-Jens Fix For: 9.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
