smengcl opened a new pull request, #11954: URL: https://github.com/apache/ozone/pull/11954
Generated-by: Claude Code (Opus 5.5) ## What changes were proposed in this pull request? Since HDDS-17235, the inter-SCM gRPC server accepts only SCM certificates when `hdds.grpc.tls.enabled` is true. Recon downloads the SCM DB through that server with its own leaf certificate, so the handshake fails and Recon reports `GET_SCM_DB_SNAPSHOT_FAILED` whenever it needs a full SCM DB sync. Accepting Recon on the gRPC server is not safe: nothing in a Recon certificate tells it apart from a Datanode or OM certificate (subject and extensions come from the CSR, and the role checked at issuance is not recorded). So this PR moves the download to an authenticated HTTP path instead: - Recon downloads the SCM DB from the leader SCM's `/dbCheckpoint` over HTTP(S), with SPNEGO when `hdds.scm.http.auth.type` is `kerberos`. It uses the leader's hostname (needed for SPNEGO and HTTPS hostname checks) and the HTTP(S) address configured for that SCM node, falling back to the common address. It rejects a checkpoint without the completion marker, like the OM DB download. If gRPC TLS is on but `ozone.http.policy` does not enable HTTPS, it logs a warning that the transfer is not encrypted, the same situation as the OM DB download in that configuration. - `SCMDBCheckpointServlet` allows the Recon principal (`ozone.recon.kerberos.principal`), the same way `OMDBCheckpointServlet` does. Like the gRPC download, it now flushes the SCM transaction buffer and uses the current DB store before taking the checkpoint. Without the flush, Recon missed recently allocated containers. - The inter-SCM gRPC server and client, and SCM Ratis, keep accepting SCM certificates only. Notes for reviewers: - Recon now needs to reach the leader SCM's HTTP(S) port instead of `ozone.scm.grpc.port`. - Upgrade SCM before Recon: an older SCM rejects Recon on `/dbCheckpoint` when SPNEGO and admin authorization are enabled. - Recon is admitted by the short name of its principal, as in OM. Datanode and OM principals are still rejected. - When `hdds.scm.http.auth.type` is `simple`, `/dbCheckpoint` does no authorization. This is unchanged by this PR. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17721 ## How was this patch tested? - `TestStorageContainerServiceProviderImpl`: leader selection with IPv6, SCM HA per-node HTTP address with SPNEGO, HTTPS port, the unencrypted HTTP warning, and rejection of an incomplete checkpoint. The download tests fail without the fix. - `TestSCMDbCheckpointServlet#testSpnegoEnabled`: the Recon principal is allowed, a Datanode principal is still rejected. - `TestReconScmSnapshot`: Recon downloads the SCM DB end to end over HTTP in a mini cluster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
