Gabriel39 commented on code in PR #70697:
URL: https://github.com/apache/doris/pull/70697#discussion_r4610840523


##########
fe/fe-core/src/main/java/org/apache/doris/common/OssThriftClientTransportProvider.java:
##########
@@ -18,16 +18,39 @@
 package org.apache.doris.common;
 
 import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.tls.PrivateRpcTls;
+import org.apache.doris.tls.TlsThriftSocket;
+import org.apache.doris.tls.server.TlsProtocolSet.Protocol;
 
 import org.apache.thrift.transport.TSocket;
 import org.apache.thrift.transport.TTransport;
 import org.apache.thrift.transport.layered.TFramedTransport;
 
 public class OssThriftClientTransportProvider implements 
ThriftClientTransportProvider {
+    @Override
+    public boolean supportsPrivateFileKeys() {
+        // Legacy subclasses may override transport creation without using the 
OSS TLS implementation.
+        return getClass() == OssThriftClientTransportProvider.class
+                && PrivateRpcTls.supportsPrivateFileKeys(Protocol.THRIFT);
+    }
+
     @Override
     public TTransport createTransport(TNetworkAddress address, int timeoutMs, 
boolean nonBlocking) throws Exception {
-        return nonBlocking
-                ? new TFramedTransport(new TSocket(address.hostname, 
address.port, timeoutMs))
+        TSocket socket = PrivateRpcTls.enabled(Protocol.THRIFT)
+                ? new TlsThriftSocket(address.hostname, address.port, 
timeoutMs)

Review Comment:
   Integrated the TLS prerequisite fixes in e3a7782e1553. Heartbeat pools 
retain the registered DNS hostname separately from the resolved connection/pool 
address, including authenticated reopen and address turnover. Both Java and 
native clients verify the logical peer identity. The selected FE tests include 
the real-listener trust/identity, reopen and DNS turnover coverage; all 145 
selected FE tests pass.



##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergEncryptionRead.java:
##########
@@ -60,10 +91,18 @@ static void check(Table table, IcebergCatalogProperties 
catalog, ConnectorContex
             throw failure("UNSUPPORTED_ENCRYPTION_MANAGER", "Encrypted Iceberg 
reads require the SDK standard "
                     + "encryption scheme with native Parquet AES_GCM_V1 
files");
         }
-        // A PME-enabled Arrow build does not prove key-transport, 
delete-reader or mixed-version scheduling
-        // support. Until explicit BE negotiation is wired, even an enabled FE 
must never emit encrypted tasks.
-        throw failure("BACKEND_CAPABILITY_UNAVAILABLE", "Encrypted Iceberg 
read capability is not verified "
-                + "by the engine; native encrypted scan execution is not 
available");
+        // Do not initialize a KMS before the engine verifies both protected 
transports and native readers.
+        if (context == null || 
context.getStorageContext().getNativeFileEncryptionVersion() < 1) {
+            throw failure("BACKEND_CAPABILITY_UNAVAILABLE", "Encrypted Iceberg 
read capability is not verified "
+                    + "by the engine; native encrypted scan execution is not 
available");
+        }
+        if (manager == null) {
+            manager = table.encryption();
+        }
+        if (!(manager instanceof StandardEncryptionManager)) {

Review Comment:
   Fixed in e3a7782e1553. Read admission no longer determines write 
eligibility. Native write planning, transaction begin, SDK refresh and final 
metadata CAS independently reject encrypted targets with WRITE_UNAVAILABLE. The 
existing validating operations retain schema/identity/snapshot checks, 
authentication and OCC behavior without adding an unconditional metadata-file 
reread for ordinary writes. Real SDK tests cover every WriteOperation, 
concurrent encryption adoption at refresh/final CAS and a successful plaintext 
commit. The SQL regression also proves encrypted INSERT/OVERWRITE refusal, 
unchanged encrypted rows and successful plaintext INSERT.



##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java:
##########
@@ -274,16 +278,20 @@ static final class TableOwner {
         private final String snapshotMetadataJson;
         private final String snapshotMetadataLocation;
         final MetaCacheSizeEstimate sizeEstimate;
+        private final boolean cacheable;
         // A newly loaded value starts with a cache reference and a temporary 
loader reference. The temporary
         // reference bridges publication/discard to the first borrow, 
including invalidation-before-publication.
         private final AtomicInteger references;
 
         private TableOwner(Table table, Runnable cleanup, boolean loading, 
boolean estimateWeight) {
             this.table = table;
             this.cleanup = cleanup;
+            // SDK managers cache unwrapped keys. Even an unweighted cache 
must release this table
+            // with its statement, so a later statement performs its own 
catalog/KMS authorization.
+            this.cacheable = !IcebergEncryptionRead.isEncrypted(table);

Review Comment:
   Fixed in e3a7782e1553. Encrypted tables explicitly throw 
SQL_CACHE_UNAVAILABLE from the connector version provider, so both admission 
and replay use the existing refusal path rather than the legacy fallback for an 
empty token. Tests cover rejection before key unwrap and eviction of a 
previously warm entry. The enabled SQL regression sets both cache switches, 
proves a plaintext PhysicalSqlCache hit, warms encrypted COUNT/SUM, revokes KMS 
permission at the same endpoint, verifies both queries fail repeatedly, then 
verifies recovery. The plaintext cache-hit control remains valid during 
revocation. Expected SQL output was generated through the repository runner and 
replayed successfully.



##########
fe/fe-core/src/main/java/org/apache/doris/system/Backend.java:
##########
@@ -886,7 +899,14 @@ public String toString() {
      * return true if any port changed, or alive state is changed.
      */
     public boolean handleHbResponse(BackendHbResponse hbResponse, boolean 
isReplay) {
-        boolean isChanged = false;
+        FileEncryptionCapability previousCapability = fileEncryptionCapability;
+        fileEncryptionCapability = hbResponse.getStatus() == HbStatus.OK
+                && hbResponse.getNativeFileEncryptionVersion() > 0
+                ? new 
FileEncryptionCapability(hbResponse.getNativeFileEncryptionVersion(), 
hbResponse.getHbTime())
+                : null;
+        // Positive leases must reach followers every round. Missing/failed 
heartbeats must revoke them
+        // immediately, even while the usual liveness grace still considers 
the backend alive.
+        boolean isChanged = previousCapability != null || 
fileEncryptionCapability != null;

Review Comment:
   Fixed in e3a7782e1553. The leader processes and retains the original BAD 
response for failure counting, eviction and diagnostics. When capability 
revocation changes state before the liveness threshold, the journal instead 
carries an OK snapshot of the resulting cached backend state with capability 
zero; older followers can interpret it without prematurely marking the backend 
dead. The serialized HbPackage test uses tolerance three and checks immediate 
capability loss, matching leader/follower liveness through each failure, 
preserved journal fields and successful recovery. It fails on the original 
premature-death behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to