OIiveirra opened a new pull request, #69480:
URL: https://github.com/apache/doris/pull/69480
### What problem does this PR solve?
Issue Number: N/A
Related PR: #69478 (backport to branch-4.1)
Problem Summary: When a Hudi JNI scan uses explicit S3A credentials,
enabling scanner DEBUG logging prints every parameter value, including access
keys, secret keys and session tokens. Log parameter names without values and
record the query ID separately so scans remain traceable. The Hadoop
configuration used to read data is unchanged.
Add a regression test that captures the real scanner's DEBUG output, checks
that parameter names and the query ID remain visible, rejects ordinary and
bucket-specific credentials and custom tokens, and verifies that Hadoop
properties retain their original values.
### Release note
Hudi JNI scanner debug logs no longer include storage credential values and
continue to include query IDs for diagnostics.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
Validation with JDK 17:
- `EXTRA_FE_MODULES=hudi=be-java-extensions/hadoop-hudi-scanner
./run-fe-ut.sh --run
'org.apache.doris.hudi.HadoopHudiJniScannerLogTest,org.apache.doris.hudi.HadoopHudiColumnValueTest'`:
5 tests passed.
- The log regression test verifies query ID retention, credential-free
DEBUG output and unchanged Hadoop properties.
- The test captures this branch's Log4j/reload4j backends.
- Maven `validate` / Checkstyle for
`be-java-extensions/hadoop-hudi-scanner` and its dependencies, plus `git diff
--check`: passed.
- The logging regression is covered directly by JUnit. FE/BE + S3
end-to-end regression was not run locally; SQL planning, credential propagation
and data-reading logic are unchanged.
- Behavior changed:
- [ ] No.
- [x] Yes. DEBUG logs retain parameter names and query IDs without
printing other parameter values.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
