colinmarc opened a new pull request, #1199:
URL: https://github.com/apache/datafusion-python/pull/1199
This fixes a very niche build problem: depending on datafusion-python as an
rlib, and then attempting to build on macOS.
Because of the presence of an 'include' configuration in Cargo.toml,
everything else is excluded - as well as `build.rs`, which means the necessary
pyo3 linker flags aren't set when building.
I verified this fixes my build issue using a git dependency on
datafusion-python:
```toml
datafusion-python = { git = "https://github.com/apache/datafusion-python";,
tag = "48.0.0" }
```
Whereas just using the crates.io version doesn't build:
```
error: linking with `cc` failed: exit status: 1
|
= note: "cc" "-Wl,-exported_symbols_list"
"-Wl,/var/folders/b5/5n9p78x14yn8z4ky6fxtdp280000gn/T/rustc7RNcRS/list"
"/var/folders/b5/5n9p78x14yn8z4ky6fxtdp280000gn/T/rustc7RNcRS/symbols.o" "<17
object files omitted>"
"/Users/colinmarc/dev/bpln/all-events/target/release/deps/{libpyo3_log-bd50c2e7d1a9be99.rlib,libarc_swap-ad5cf4ebc23fa86d.rlib,libpyo3_async_runtimes-eba16526ca814ba0.rlib,libdatafusion_ffi-93bcb82d1826af3c.rlib,libsemver-69712b2f6934f52b.rlib,libasync_ffi-109cfda0a79f6f46.rlib,libdatafusion_proto-98671f6b4c156261.rlib,libdatafusion_proto_common-c828cce4df9cba68.rlib,libpbjson-e8c3fb6b43fe6a5d.rlib,libbase64-aabdebdc40eac9d2.rlib,libprost-492503f16444bcb4.rlib,libabi_stable-7f2a8da44cc2ad72.rlib,libconst_panic-04abeecc901c5835.rlib,librepr_offset-aec98a1518dda497.rlib,libtstr-e05c375dcbbdf609.rlib,libgenerational_arena-b27652eb370a1900.rlib,liblibloading-c9cfdbf7935ce071.rlib,libcrossbeam_channel-b3c1336cf04f651d.rlib,libabi_stable_shared-4a32241e1181b964.rlib,libcore_extensions-6be478146b530258.rlib,libdatafusion-68a5090fb1c58149.rlib,libdatafusion_functions_table-a785450daa220237.rlib,libdatafusion_functions_window-5c8ad3b057064cc3.rlib,libdatafusion_functions_nested-67c01
8d9521f14b2.rlib,libdatafusion_optimizer-52cbf43a83dc754a.rlib,libdatafusion_catalog_listing-7240cba6b8bb5026.rlib,libdatafusion_datasource_parquet-1d0a14c6642ff3d7.rlib,libdatafusion_physical_optimizer-07e0639703d8adc2.rlib,libdatafusion_datasource_avro-4b527ad125549f8f.rlib,libdatafusion_datasource_json-819f00f9c2f06d50.rlib,libdatafusion_datasource_csv-65d7c49418d40a12.rlib,libdatafusion_sql-c3b0e476dda0b9bd.rlib,libdatafusion_catalog-2958831fff8d6b8e.rlib,libdatafusion_datasource-a3b673b920baa280.rlib,libglob-5ed07e1ef250c612.rlib,libdatafusion_session-2b1ac2b9b5f38399.rlib,libasync_compression-8b5063c331ce847f.rlib,libbzip2-4f06fb5ed9de7d54.rlib,libdatafusion_physical_plan-2c5fe747d192870e.rlib,libdatafusion_common_runtime-5f702cc8d21973d0.rlib,libdatafusion_functions_aggregate-6b7b6f62fc4cf9c4.rlib,libdatafusion_physical_expr-1224453ebb9347b0.rlib,libpetgraph-d52f52b88facb45f.rlib,libfixedbitset-178cc05d890be25d.rlib,libdatafusion_functions-41d93652653ef031.rlib,libhex-b6c5adf
57041980f.rlib,libsha2-c9b63e0c039043f3.rlib,libblake3-b47f6144315c4d77.rlib,libconstant_time_eq-42fd34e10c578200.rlib,libarrayvec-f9543f3f5ae864ab.rlib,libarrayref-d31fc091e2d6ee0e.rlib,libblake2-f4bf0046e980e172.rlib,libdatafusion_execution-c55886d381b3d34d.rlib,libtempfile-a1b2a5a4c563dc3b.rlib,libfastrand-36f87d3ad77c6fd5.rlib,librustix-aa2aaf9c3cec78aa.rlib,liberrno-79ccaecae0769ee1.rlib,libdashmap-6146eda158b025d0.rlib,libcrossbeam_utils-93015497c3e214cc.rlib,libdatafusion_expr-387d959328307058.rlib,libdatafusion_functions_aggregate_common-d90b999257ee962c.rlib,libdatafusion_functions_window_common-57329e93b7b40877.rlib,libdatafusion_doc-4d6559444419a12e.rlib,libdatafusion_physical_expr_common-d485d08f09369c57.rlib,libdatafusion_expr_common-5ef60fe07d8ee1b6.rlib,libdatafusion_common-934a95f38743bfaf.rlib,libhashbrown-80c15c43a5999630.rlib,libparquet-6c7cbfd00f9488c6.rlib,libbrotli-7ab64b7a0bc1c6e8.rlib,libbrotli_decompressor-ee61c12d1794f7c3.rlib,liballoc_stdlib-0497de1978d9f0
33.rlib,liballoc_no_stdlib-19d7b7eef48a3ff3.rlib,libflate2-9cafbdaee45ba358.rlib,libminiz_oxide-5a7d86077f7b9bd8.rlib,libadler2-5fc4e824e4457afb.rlib,liblibz_rs_sys-ed17e8a28580c3a2.rlib,libzlib_rs-d96186838fec8b4a.rlib,libobject_store-6edbb4329d27bd6c.rlib,libhumantime-5e0e4d17e02e6915.rlib,libthiserror-effe5e3ac7db3b8f.rlib,libwalkdir-9576fe72c80a1302.rlib,libsame_file-1fb48a6cc3f84c12.rlib,libparking_lot-1c1bc5ffa45d3ab6.rlib,libparking_lot_core-29fe49a404d7f533.rlib,liblock_api-a8dec283b0b7515c.rlib,libscopeguard-c1c5590a3fd72fb5.rlib,librustls_pemfile-432c5a98bc793fc7.rlib,librand-9516c305f7708ff0.rlib,librand_chacha-de4fd04099850197.rlib,librand_core-c5592f420480d246.rlib,libquick_xml-489731081c9f4e25.rlib,libmd5-ecdd0a59f23205fe.rlib,libitertools-13b9d0655b2429f3.rlib,libeither-20f8714b75d621d9.rlib,libreqwest-5e3084d5cec20c6e.rlib,libserde_urlencoded-5042827945b6a31b.rlib,libhyper_rustls-224c761b353653e2.rlib,libwebpki_roots-7026cf1d449d14b4.rlib,librustls_native_certs-d906b
1ed148a3439.rlib,libsecurity_framework-2087cf7c1037269c.rlib,libcore_foundation-6b0653818f6f8a46.rlib,libsecurity_framework_sys-3d967eaa6d5db32a.rlib,libtokio_rustls-283618c8ca1d0dba.rlib,librustls-f7941ad045182529.rlib,libwebpki-94a5de63cdded17c.rlib,libring-e7703330386703aa.rlib,libuntrusted-c90860e50999b2a9.rlib,librustls_pki_types-0041ed224af372d3.rlib,libzeroize-9022ecac0a5e4acb.rlib,libtower_http-30f42cd6330d5ccd.rlib,libiri_string-0347e74622b08586.rlib,libtower-9503d8b3450d06d1.rlib,libtower_layer-f2e5f5dfcab61a2e.rlib,libhyper_util-1528424faa1a1748.rlib,libipnet-59b76d180a7ed2c3.rlib,libtower_service-2b92735bb08efeaa.rlib,libhyper-6e0744af25ac9bb9.rlib,libwant-79f9ecf12d0e2f65.rlib,libtry_lock-d5b41966a8a8def6.rlib,libhttparse-ef4be4395f90b1ab.rlib,libh2-dfc8aeaf325014af.rlib,libtracing-edd79e603cfb2c6e.rlib,libtracing_core-adf72d8e51cfce6a.rlib,libatomic_waker-834bc70eb5a5ee4c.rlib,libtokio_util-072198e6abe236af.rlib,libhttp_body_util-b3bc1a1aaae7b453.rlib,libhttp_body-7597
5b120e974f59.rlib,liburl-e285a33fec72a732.rlib,libidna-14cae32c7750eb8b.rlib,libutf8_iter-86aeeea8b517fd05.rlib,libidna_adapter-01f5c27feba1aa00.rlib,libicu_properties-de77c15a0df5c0e0.rlib,libicu_properties_data-5ca75e326589e0ed.rlib,libicu_normalizer-26c421f9ad902b44.rlib,libsmallvec-249d3f4ca8754cd8.rlib,libicu_normalizer_data-832c02018fb20fdc.rlib,libicu_collections-d9edb28e1775c80c.rlib,libpotential_utf-caba7d3c0f82f7bd.rlib,libicu_provider-677e08df1f065d0d.rlib,libicu_locale_core-c93769ac58b78da0.rlib,libtinystr-b4c0d3e234080f71.rlib,liblitemap-7434bcab13789da1.rlib,libwriteable-3595c6a4727da580.rlib,libzerovec-4d93715f0ed3cf04.rlib,libzerotrie-20ffe62c782ed7db.rlib,libyoke-2397f33fa66e22e1.rlib,libstable_deref_trait-1ad5d0b738b52bb9.rlib,libzerofrom-68088c4d8a6d3565.rlib,libform_urlencoded-8511452e00f9e778.rlib,libpercent_encoding-30f7b203e5794502.rlib,libhttp-45aadb77363e617b.rlib,libfnv-e07de106c7b4c98c.rlib,libsync_wrapper-068989de3d3e9fb3.rlib,libtokio-fa259681e99ba4f4.rl
ib,libsocket2-aae2154fbabf005f.rlib,libmio-261f0c46b28be8f2.rlib,libfutures-c5d96c7fcbbc14e1.rlib,libfutures_executor-67bf974f0b56d822.rlib,libfutures_util-1c3fe680cafecb70.rlib,libfutures_io-f4e11ec4606227af.rlib,libslab-4f719ac61e3db7c6.rlib,libfutures_channel-0cd37d2768807004.rlib,libpin_project_lite-d6b4c3abaa6ba590.rlib,libfutures_sink-0c37eddd73a396c5.rlib,libfutures_task-227a24be170cf210.rlib,libpin_utils-7565c1c1b0d23345.rlib,libfutures_core-73cf6b1d709bf5f9.rlib,libthrift-3009cb079a023204.rlib,libordered_float-40d7d306ca8c64ca.rlib,libthreadpool-07469e391275dec2.rlib,libnum_cpus-eb9b30df06252e68.rlib,libinteger_encoding-aaa7155d90d30c4f.rlib,libbyteorder-99912f0ff4e26454.rlib,libapache_avro-3204a2ccbb43cc6c.rlib,librand-c85d250afe322256.rlib,librand_chacha-f0f4cae7e338eae2.rlib,libppv_lite86-97de4abe00feffa7.rlib,librand_core-81fd3c91bb54801c.rlib,libgetrandom-d93181f1ee66e022.rlib,libzstd-fafe94de6cb95623.rlib,libzstd_safe-82265c9d926be9f7.rlib,libzstd_sys-6576136844b57447
.rlib,libsnap-8b39be63489ddb57.rlib,libstrum-4fa9c7db083a41ed.rlib,libserde_bytes-a1c48fa0a9358c6f.rlib,libtyped_builder-8773f0b700ae2c31.rlib,libthiserror-b114ffb56e089b69.rlib,libregex_lite-1cd5459a9075a5b9.rlib,libdigest-7d57b25a08b94040.rlib,libsubtle-fba2f0087773609b.rlib,libconst_oid-6fc9dbda5666a297.rlib,libblock_buffer-21f68a0774c20711.rlib,libcrypto_common-9d5b7d24ca0d1cc9.rlib,libgeneric_array-c52cfd15b4c81270.rlib,libtypenum-dbc94d9ac6ecc909.rlib,libuuid-f7ee5ffe9fc05ec2.rlib,libxz2-fc787732f1ebc2b1.rlib,liblzma_sys-bb85373e4479f82a.rlib,libbzip2-d914a72e469559f1.rlib,libbzip2_sys-d2d139929083af2d.rlib,liblibflate-8b21986f86eb0661.rlib,libdary_heap-f4a7bbd38e9aefc5.rlib,libadler32-7c8e6a0369083f50.rlib,liblibflate_lz77-13c7bc3f6a43dba7.rlib,librle_decode_fast-ccfd9a82bdfba950.rlib,libcore2-d3ea4253b3ae26d0.rlib,libbigdecimal-30e928dee9fc5700.rlib,liblibm-82228d2d19a02ab8.rlib,libcrc32fast-f8aca6d2b09ed99b.rlib,libarrow-8a1f7ce8c9dd14f7.rlib,libarrow_row-84eb08d4cd961ffd.r
lib,libarrow_pyarrow-5622f52f998232d6.rlib,libarrow_json-4b0a30102174a6de.rlib,libsimdutf8-46b6d81e66194e68.rlib,libindexmap-21d88474d1f1486d.rlib,libarrow_ipc-85d999ef949f9f5a.rlib,liblz4_flex-b12f51b3c2452d9a.rlib,libtwox_hash-1d2f8dbc720fa8f7.rlib,libflatbuffers-16321d00244e270f.rlib,libarrow_csv-05870c3b279a0398.rlib,libcsv-86a9275ba06878ec.rlib,libcsv_core-7faf0ea49beed3df.rlib,libarrow_string-c8f4d5c67fe11223.rlib,libregex-a26b2741ad471a02.rlib,libregex_automata-63be80d6e35fd497.rlib,libaho_corasick-ebe662c7d81718e6.rlib,libregex_syntax-2fd5791d4664d0bb.rlib,libarrow_cast-3ef77d16a9989f53.rlib,libatoi-898e3c475248cd16.rlib,libbase64-93cc24d04722c6b2.rlib,libcomfy_table-b31f844dae7a2715.rlib,libunicode_segmentation-08caeb8119b6a428.rlib,libunicode_width-819d17f485789e82.rlib,liblexical_core-6b29bf7482d4ac1a.rlib,liblexical_write_float-95a6204f3a06f52c.rlib,liblexical_write_integer-d38178122b23ea98.rlib,liblexical_parse_float-b2b676c7cae99aec.rlib,liblexical_parse_integer-bf0edb
bc6c845c6c.rlib,liblexical_util-dff9ce4f975beea9.rlib,libstatic_assertions-b5177fa3f151f88a.rlib,libarrow_arith-8f71231d05b60d21.rlib,libarrow_ord-b8bbf587e071d183.rlib,libarrow_select-149bab3643fafe47.rlib,libarrow_array-cac0bf1b95860c27.rlib,libchrono_tz-7976adf5e08a5faf.rlib,libphf-e4bcc5492e1638c4.rlib,libphf_shared-29089568c7a98d73.rlib,libsiphasher-5e073432ba80b0ec.rlib,libahash-76010dfde000bb47.rlib,libgetrandom-a8a2fba13b7c3926.rlib,libzerocopy-93ff1ffbd9c920f9.rlib,libhashbrown-c09e8c50d4e4de7e.rlib,libfoldhash-8add571487c208a9.rlib,libequivalent-eaa0685fbf36c459.rlib,liballocator_api2-bf6b28bb29149b15.rlib,libchrono-16a492c8d6535c05.rlib,libiana_time_zone-9838eca935254022.rlib,libcore_foundation_sys-c7bdeb0be8f778de.rlib,libarrow_data-ad1393f0813db00b.rlib,libarrow_schema-b2763afed1d75869.rlib,libserde_json-d8d7def69aaeab54.rlib,libmemchr-010e50dec96889d4.rlib,libitoa-9897d6f33baa8efe.rlib,libryu-89145247fa865d8b.rlib,libbitflags-0b752ee354c1714f.rlib,libarrow_buffer-df745
097521deb5d.rlib,libbytes-e9a5beea09e177f1.rlib,libhalf-1d5a96a0d129616c.rlib,libnum-1191270fef722447.rlib,libnum_iter-864a6772fe1c3728.rlib,libnum_rational-42bf3d6e0977e09b.rlib,libnum_complex-04cb518d1a34e126.rlib,libnum_bigint-f614862af97b5053.rlib,libserde-379e177ca80553ea.rlib,libnum_integer-9ea6283701072edd.rlib,libnum_traits-73dfdd8565e95ecb.rlib,libsqlparser-dae7132ab45c41eb.rlib,librecursive-ebdee60bb2e2bbbb.rlib,libstacker-b6c1c942456734b9.rlib,libpsm-d45da3fd3c987ab9.rlib,liblog-f848965e9d671f27.rlib,libpyo3-90b98f762f1a9d3b.rlib,libcfg_if-34bf03306b1dc143.rlib,libmemoffset-b977bfb2f9030c4a.rlib,libonce_cell-068f7909549c15e0.rlib,libpyo3_ffi-75cd3589555f4be7.rlib,libunindent-47c92a8720c117c0.rlib,libmimalloc-72a66481084f7fdc.rlib,liblibmimalloc_sys-ad4f99036bc921e8.rlib,liblibc-0caa3c72f476e2c6.rlib}.rlib"
"<sysroot>/lib/rustlib/aarch64-apple-darwin/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,libcfg_if-*,liblibc-*,liballoc-*,librustc_std_workspace_core-*,libcore-*,libcompiler_builtins-*}.rlib"
"-framework" "Security" "-framework" "CoreFoundation" "-liconv"
"-lSystem" "-lc" "-lm" "-arch" "arm64" "-mmacosx-version-min=11.0.0" "-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/bzip2-sys-ce2535ad003224bd/out/lib"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/lzma-sys-e3b54090af14827e/out"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/zstd-sys-89467c99fe98418d/out"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/ring-6c070c10dae4d85c/out"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/psm-478f69e03f9c498e/out"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/blake3-b9ff590313ad109f/out"
"-L"
"/Users/colinmarc/dev/bpln/all-events/target/release/build/libmimalloc-sys-b64c9c1ed3774578/out"
"-o"
"/Users/colinmarc/dev/bpln/all-events/target/release/deps/libdatafusion_python-62a9be6a9f738ad3.dylib"
"-Wl,-dead_strip" "-dynamiclib" "-nodefaultlibs"
= note: some arguments are omitted. use `--verbose` to show all
linker arguments
= note: ld: warning: object file
(/Users/colinmarc/dev/bpln/all-events/target/release/deps/libblake3-b47f6144315c4d77.rlib[5](a1edd97dd51cd48d-blake3_neon.o))
was built for newer 'macOS' version (15.5)
than being linked (11.0)
Undefined symbols for architecture arm64:
"_PyBaseObject_Type", referenced from:
<snip>
...
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v
to see invocation)
error: could not compile `datafusion-python` (lib) due to 1 previous
error
💥 maturin failed
Caused by: Failed to build a native library through cargo
Caused by: Cargo build finished with "exit status: 101": `env -u
CARGO MACOSX_DEPLOYMENT_TARGET="11.0"
PYO3_ENVIRONMENT_SIGNATURE="cpython-3.12-64bit"
PYO3_PYTHON="/Users/colinmarc/.cache/uv/builds-v0/.tmppBVRox/bin/python"
PYTHON_SYS_EXECUTABLE="/Users/colinmarc/.cache/uv/builds-v0/.tmppBVRox/bin/python"
"cargo" "rustc" "--features"
"pyo3/extension-module" "--features" "python" "--message-format"
"json-render-diagnostics" "--manifest-path"
"/Users/colinmarc/dev/bpln/all-events/crates/bpln-df-extensions/Cargo.toml"
"--release" "--lib"
"--crate-type" "cdylib" "--" "-C" "link-arg=-undefined" "-C"
"link-arg=dynamic_lookup" "-C"
"link-args=-Wl,-install_name,@rpath/bauplan_datafusion_extensions.abi3.so"`
Error: command ['maturin', 'pep517', 'build-wheel', '-i',
'/Users/colinmarc/.cache/uv/builds-v0/.tmppBVRox/bin/python',
'--compatibility', 'off'] returned non-zero exit status 1
```
Note that you can verify build.rs is missing this way:
```
curl -fsSL https://crates.io/api/v1/crates/datafusion-python/48.0.0/download
-o - | tar tf - | grep build.rs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
