This is an automated email from the ASF dual-hosted git repository.
mbutrovich pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new d9ea22b1d docs: Add common pitfalls and improve PR checklist in
development guide (#3231)
d9ea22b1d is described below
commit d9ea22b1d755b4f817724f3fc685c018df013e43
Author: Andy Grove <[email protected]>
AuthorDate: Wed Jan 21 09:22:17 2026 -0700
docs: Add common pitfalls and improve PR checklist in development guide
(#3231)
Add a new "Common Build and Test Pitfalls" section documenting:
- Native code must be built before running JVM tests
- Debug vs release mode (release only needed for benchmarking)
- LD_LIBRARY_PATH requirement for libjvm.so when running Rust tests
- Avoid using -pl to select modules (stale common module issue)
- Use wildcardSuites for flexible test matching
Restructure "Submitting a Pull Request" section as a numbered checklist:
1. Format code with `make format`
2. Build and verify with `make`
3. Run Clippy (recommended)
4. Run tests
Add a quick Pre-PR Summary command reference.
Co-authored-by: Claude Opus 4.5 <[email protected]>
---
docs/source/contributor-guide/development.md | 116 ++++++++++++++++++++++++++-
1 file changed, 114 insertions(+), 2 deletions(-)
diff --git a/docs/source/contributor-guide/development.md
b/docs/source/contributor-guide/development.md
index 953ac31f7..4d6bbe9f2 100644
--- a/docs/source/contributor-guide/development.md
+++ b/docs/source/contributor-guide/development.md
@@ -48,6 +48,74 @@ A few common commands are specified in project's `Makefile`:
such as Spark.
- `make clean`: clean up the workspace
+## Common Build and Test Pitfalls
+
+### Native Code Must Be Built First
+
+The native Rust code must be compiled before running JVM tests. If you skip
this step, tests will
+fail because they cannot find the native library. Always run `make core` (or
`cd native && cargo build`)
+before running Maven tests.
+
+```sh
+# Correct order
+make core # Build native code first
+./mvnw test -Dsuites="..." # Then run JVM tests
+```
+
+### Debug vs Release Mode
+
+There is no need to use release mode (`make release`) during normal
development. Debug builds
+are faster to compile and provide better error messages. Only use release mode
when:
+
+- Running benchmarks
+- Testing performance
+- Creating a release build for deployment
+
+For regular development and testing, use `make` or `make core` which build in
debug mode.
+
+### Running Rust Tests
+
+When running Rust tests directly with `cargo test`, the JVM library
(`libjvm.so`) must be on
+your library path. Set the `LD_LIBRARY_PATH` environment variable to include
your JDK's `lib/server`
+directory:
+
+```sh
+# Find your libjvm.so location (example for typical JDK installation)
+export LD_LIBRARY_PATH=$JAVA_HOME/lib/server:$LD_LIBRARY_PATH
+
+# Now you can run Rust tests
+cd native && cargo test
+```
+
+Alternatively, use `make test-rust` which handles the JVM compilation
dependency automatically.
+
+### Avoid Using `-pl` to Select Modules
+
+When running Maven tests, avoid using `-pl spark` to select only the spark
module. This can cause
+Maven to pick up the `common` module from your local Maven repository instead
of using the current
+codebase, leading to inconsistent test results:
+
+```sh
+# Avoid this - may use stale common module from local repo
+./mvnw test -pl spark -Dsuites="..."
+
+# Do this instead - builds and tests with current code
+./mvnw test -Dsuites="..."
+```
+
+### Use `wildcardSuites` for Running Tests
+
+When running specific test suites, use `wildcardSuites` instead of `suites`
for more flexible
+matching. The `wildcardSuites` parameter allows partial matching of suite
names:
+
+```sh
+# Run all suites containing "CometCast"
+./mvnw test -DwildcardSuites="CometCast"
+
+# Run specific suite with filter
+./mvnw test -Dsuites="org.apache.comet.CometCastSuite valid"
+```
+
## Development Environment
Comet is a multi-language project with native code written in Rust and JVM
code written in Java and Scala.
@@ -161,10 +229,31 @@ It is possible to debug both native and JVM code
concurrently as described in th
## Submitting a Pull Request
+Before submitting a pull request, follow this checklist to ensure your changes
are ready:
+
+### 1. Format Your Code
+
Comet uses `cargo fmt`, [Scalafix](https://github.com/scalacenter/scalafix)
and [Spotless](https://github.com/diffplug/spotless/tree/main/plugin-maven) to
-automatically format the code. Before submitting a pull request, you can
simply run `make format` to format the code.
+automatically format the code. Run the following command to format all code:
+
+```sh
+make format
+```
+
+### 2. Build and Verify
+
+After formatting, run a full build to ensure everything compiles correctly and
generated
+documentation is up to date:
+
+```sh
+make
+```
-Additionally, it's strongly recommended to run Clippy locally to catch
potential issues before the CI/CD pipeline does. You can run the same Clippy
checks used in CI/CD with:
+This builds both native and JVM code. Fix any compilation errors before
proceeding.
+
+### 3. Run Clippy (Recommended)
+
+It's strongly recommended to run Clippy locally to catch potential issues
before the CI/CD pipeline does. You can run the same Clippy checks used in
CI/CD with:
```bash
cd native
@@ -173,6 +262,29 @@ cargo clippy --color=never --all-targets --workspace -- -D
warnings
Make sure to resolve any Clippy warnings before submitting your pull request,
as the CI/CD pipeline will fail if warnings are present.
+### 4. Run Tests
+
+Run the relevant tests for your changes:
+
+```sh
+# Run all tests
+make test
+
+# Or run only Rust tests
+make test-rust
+
+# Or run only JVM tests (native must be built first)
+make test-jvm
+```
+
+### Pre-PR Summary
+
+```sh
+make format # Format code
+make # Build everything and update generated docs
+make test # Run tests (optional but recommended)
+```
+
## How to format `.md` document
We are using `prettier` to format `.md` files.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
