andygrove commented on code in PR #7356: URL: https://github.com/apache/datafusion-comet/pull/7356#discussion_r4565944410
########## docs/source/user-guide/latest/source.md: ########## @@ -78,3 +78,32 @@ To build Comet from the source distribution on an isolated environment without a ```console make release-nogit PROFILES="-Pspark-4.1" ``` + +## Building with the jemalloc allocator + +The native library allocates its memory with the system allocator, which is glibc malloc on Linux. +In a long-running executor, glibc can keep several GiB of freed native memory resident (see +[#7108](https://github.com/apache/datafusion-comet/issues/7108)). The published jars use the system +allocator. To build Comet with [jemalloc](https://jemalloc.net/) instead, add the `jemalloc` profile: + +```console +make release PROFILES="-Pspark-4.1 -Pjemalloc" +``` + +`make release-nogit` accepts it too. The profile builds the native library with its `jemalloc` Cargo Review Comment: Could this say that `make` is what adds the `jemalloc` Cargo feature when `PROFILES` names the profile? The `jemalloc` profile itself has no settings, so someone running `./mvnw install -Pjemalloc` after their own `cargo build` would get the system allocator without any warning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
