This is an automated email from the ASF dual-hosted git repository.
houqp pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/master by this push:
new 50cce1a Updated user-guide library docs with optimized config (#976)
50cce1a is described below
commit 50cce1afd2fb67df3845ffb986f7f07d3445313d
Author: Matthew Turner <[email protected]>
AuthorDate: Tue Sep 7 16:46:19 2021 -0400
Updated user-guide library docs with optimized config (#976)
* Updated user-guide library docs with optimized config
* clarify description and remove datafusion git reference
---
docs/user-guide/src/library.md | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/docs/user-guide/src/library.md b/docs/user-guide/src/library.md
index d255d78..1a1bbfb 100644
--- a/docs/user-guide/src/library.md
+++ b/docs/user-guide/src/library.md
@@ -19,6 +19,8 @@
# Using DataFusion as a library
+## Default Configuration
+
DataFusion is [published on crates.io](https://crates.io/crates/datafusion),
and is [well documented on docs.rs](https://docs.rs/datafusion/).
To get started, add the following to your `Cargo.toml` file:
@@ -27,3 +29,33 @@ To get started, add the following to your `Cargo.toml` file:
[dependencies]
datafusion = "5.1.0"
```
+
+## Optimized Configuration
+
+For an optimized build several steps are required. First, use the below in
your `Cargo.toml`. It is
+worth noting that using the settings in the `[profile.release]` section will
significantly increase the build time.
+
+```toml
+[dependencies]
+datafusion = { version = "5.0" , features = ["simd"]}
+tokio = { version = "^1.0", features = ["macros", "rt", "rt-multi-thread"] }
+snmalloc-rs = {version = "0.2", features= ["cache-friendly"]}
+num_cpus = "1.0"
+
+[profile.release]
+lto = true
+codegen-units = 1
+```
+
+Then, in `main.rs.` update the memory allocator with the below after your
imports:
+
+```rust
+#[global_allocator]
+static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
+```
+
+Finally, in order to build with the `simd` optimization `cargo nightly` is
required. Based on the instruction
+set architecture you are building on you will want to configure the
`target-cpu` as well, ideally
+with `native` or at least `avx2`.
+
+`RUSTFLAGS='-C target-cpu=native' cargo +nightly run --release`
