ericm-db opened a new pull request, #57907: URL: https://github.com/apache/spark/pull/57907
### What changes were proposed in this pull request? This is layer 3 of the local Connect pool stack: merged #57684 -> #57685 -> this PR. Until the lower layer merges, GitHub shows its cumulative diff; the new review unit here is commit `06648f81705`. This layer adds member compatibility and claiming on top of the filesystem state model: - fingerprints for the master, startup conf, working directory, and Python environment; - validated member records with defensive handling of malformed fields; - Spark-version, process-liveness, and socket-reachability checks; and - deterministic FIFO claiming through an atomic state-file rename. Pool sizing now lives with its first consumer in acquisition layer #57687. Process lifecycle, acquisition, SparkSession integration, and JIT warmup remain in later PRs. ### Why are the changes needed? The filesystem layer defines safe state storage, but a client also needs to distinguish compatible servers and claim exactly one member without racing other processes. Isolating that contract keeps record validation and the ready-to-claimed transition independently reviewable before lifecycle and launch orchestration are added. ### Does this PR introduce _any_ user-facing change? No. Claiming is internal and is not wired into SparkSession in this layer. ### How was this patch tested? Added six focused tests at this layer (nine cumulative in the suite) covering Linux zombie detection, fingerprint identity and Python-interpreter precedence, fingerprint-aware claiming, FIFO ordering, unreachable members, and malformed, dead, or version-incompatible records. ```bash python/run-tests --testnames pyspark.sql.tests.connect.test_connect_local_server_pool ``` These cases passed on Python 3.11 before the history-only split from #57685. The rebuilt commit also passed `git diff --check`, Python AST parsing, and changed-line ASCII and 100-column checks. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Fable 5) and OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
