This is an automated email from the ASF dual-hosted git repository.

okumin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git


The following commit(s) were added to refs/heads/master by this push:
     new af44ec916b2 HIVE-29970: Document mask_hash and aes_encrypt limitations 
(#6896)
af44ec916b2 is described below

commit af44ec916b2a11bab27be487248f6cdf232dc428
Author: Ruben Quesada Lopez <[email protected]>
AuthorDate: Thu Oct 22 15:08:43 2026 +0100

    HIVE-29970: Document mask_hash and aes_encrypt limitations (#6896)
    
    * HIVE-29970: Document mask_hash and aes_encrypt limitations
    
    * Update THREAT_MODEL.md
    
    Co-authored-by: Shohei Okumiya <[email protected]>
    
    * Update THREAT_MODEL.md
    
    Co-authored-by: Shohei Okumiya <[email protected]>
    
    ---------
    
    Co-authored-by: Shohei Okumiya <[email protected]>
---
 THREAT_MODEL.md                                             | 13 +++++++++++++
 .../hadoop/hive/ql/udf/generic/GenericUDFAesBase.java       |  4 +++-
 .../hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java    |  2 ++
 .../hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java    |  4 ++++
 .../hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java      |  7 ++++++-
 .../test/results/clientpositive/llap/udf_aes_decrypt.q.out  |  1 +
 .../test/results/clientpositive/llap/udf_aes_encrypt.q.out  |  1 +
 ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out |  5 +++++
 8 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/THREAT_MODEL.md b/THREAT_MODEL.md
index a6350cd24bc..30c9502292b 100644
--- a/THREAT_MODEL.md
+++ b/THREAT_MODEL.md
@@ -212,6 +212,15 @@ controls. *(maintainer — okumin)*
   (e.g. `hive.query.max.length`) and YARN resource pools, and — where stronger
   isolation than HS2 + YARN can provide is required — separate HS2 instances or
   separate Hadoop/YARN clusters. *(maintainer — okumin, §14 Q11)*
+- **Cryptographic strength of built-in masking / encryption UDFs.**
+  `mask_hash` is a deterministic, unsalted, unkeyed SHA-256/512 digest (so
+  masked values remain joinable); `aes_encrypt`/`aes_decrypt` currently use
+  AES/ECB/PKCS5Padding (no IV, no authentication) by default, for
+  compatibility with existing ciphertexts and MySQL-style `AES_ENCRYPT`. These
+  functions do not provide semantic security or resistance to
+  dictionary/enumeration attacks, and Hive does not claim they do. Choosing a
+  masking type appropriate to the data (e.g. nullify/redact for low-entropy
+  columns) is the policy author's responsibility.  *(documented: 
GenericUDFMaskHash, GenericUDFAesEncrypt, GenericUDFAesDecrypt)*.
 
 ## §8 — Key configuration levers (load-bearing)
 
@@ -303,6 +312,10 @@ These are the dispositions the PMC has recorded; confirm 
and extend
   the operator has not configured the blacklist → `OUT-OF-MODEL:
   operator responsibility`, unless the report shows a
   *bypass* of a configured blacklist. *(maintainer — okumin.)*
+- **"`mask_hash` is invertible / `aes_encrypt` uses ECB / a built-in hash UDF
+  is weak."** By design — see §7. Valid only if the report shows a bypass of a
+  configured masking policy (e.g. reading the unmasked column), not the
+  inherent properties of the function. `BY-DESIGN`. *(documented: 
GenericUDFMaskHash, GenericUDFAesEncrypt, GenericUDFAesDecrypt))*.
 - **Dependency-tail CVEs** (Hadoop, Log4j, a transitive JAR) surfaced by an
   SCA scanner against Hive's build — triage upstream unless Hive's own code
   reaches the vulnerable path with untrusted input.
diff --git 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
index 41d5c8c5b6f..d3d63f5d028 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesBase.java
@@ -106,7 +106,9 @@ public ObjectInspector initialize(ObjectInspector[] 
arguments) throws UDFArgumen
     }
 
     try {
-      cipher = Cipher.getInstance("AES");
+      // Explicit form of the provider default for "AES". ECB is kept for 
compatibility with existing
+      // ciphertexts and MySQL-style AES_ENCRYPT; see the 
aes_encrypt/aes_decrypt descriptions.
+      cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");
     } catch (NoSuchPaddingException | NoSuchAlgorithmException e) {
       throw new RuntimeException(e);
     }
diff --git 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
index d5c3d456e02..028279d725e 100644
--- 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
+++ 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesDecrypt.java
@@ -31,6 +31,8 @@
     + "Key lengths of 128, 192 or 256 bits can be used. 192 and 256 bits keys 
can be used if "
     + "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction 
Policy Files are installed. "
     + "If either argument is NULL or the key length is not one of the 
permitted values, the return value is NULL.\n"
+    + "Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, 
MySQL-style) with no IV and no "
+    + "authentication; see the aes_encrypt description for its limitations.\n"
     + "Example: > SELECT _FUNC_(unbase64('y6Ss+zCYObpCbgfWfyNWTw=='), 
'1234567890123456');\n 'ABC'")
 public class GenericUDFAesDecrypt extends GenericUDFAesBase {
 
diff --git 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
index b5be296b548..eab265941dd 100644
--- 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
+++ 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFAesEncrypt.java
@@ -31,6 +31,10 @@
     + "Key lengths of 128, 192 or 256 bits can be used. 192 and 256 bits keys 
can be used if "
     + "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction 
Policy Files are installed. "
     + "If either argument is NULL or the key length is not one of the 
permitted values, the return value is NULL.\n"
+    + "Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, 
MySQL-style) with no IV and no "
+    + "authentication. Encryption is deterministic: equal inputs encrypted 
with the same key produce equal "
+    + "outputs, so anyone who can read the ciphertext can detect duplicates 
and infer value frequencies, and "
+    + "ciphertexts are malleable. It is not suitable for protecting columns 
with repetitive or low-entropy values.\n"
     + "Example: > SELECT base64(_FUNC_('ABC', '1234567890123456'));\n 
'y6Ss+zCYObpCbgfWfyNWTw=='")
 public class GenericUDFAesEncrypt extends GenericUDFAesBase {
 
diff --git 
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java 
b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
index 0f216ea11c1..1900420779c 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFMaskHash.java
@@ -34,7 +34,12 @@
              extended = "Examples:\n "
                       + "  mask_hash(value)\n "
                       + "Arguments:\n "
-                      + "  value - value to mask. Supported types: STRING, 
VARCHAR, CHAR"
+                      + "  value - value to mask. Supported types: STRING, 
VARCHAR, CHAR\n "
+                      + "Note:\n "
+                      + "  The result is an unsalted, unkeyed SHA-256 digest 
(SHA-512 if hive.masking.algo=sha512).\n "
+                      + "  It is deterministic, so values from small or 
predictable domains (e.g. phone numbers,\n "
+                      + "  national IDs, dates) can be recovered by hashing 
every candidate value. For such\n "
+                      + "  columns prefer a nullifying or redacting mask."
             )
 public class GenericUDFMaskHash extends BaseMaskUDF {
   public static final String UDF_NAME = "mask_hash";
diff --git a/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out 
b/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
index c4d983c82eb..c6f7c367129 100644
--- a/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_aes_decrypt.q.out
@@ -9,6 +9,7 @@ POSTHOOK: query: DESC FUNCTION EXTENDED aes_decrypt
 POSTHOOK: type: DESCFUNCTION
 aes_decrypt(input binary, key string/binary) - Decrypt input using AES.
 AES (Advanced Encryption Standard) algorithm. Key lengths of 128, 192 or 256 
bits can be used. 192 and 256 bits keys can be used if Java Cryptography 
Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If 
either argument is NULL or the key length is not one of the permitted values, 
the return value is NULL.
+Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, MySQL-style) 
with no IV and no authentication; see the aes_encrypt description for its 
limitations.
 Example: > SELECT aes_decrypt(unbase64('y6Ss+zCYObpCbgfWfyNWTw=='), 
'1234567890123456');
  'ABC'
 Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesDecrypt
diff --git a/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out 
b/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
index 4879bf7009b..1afd2ee4755 100644
--- a/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_aes_encrypt.q.out
@@ -9,6 +9,7 @@ POSTHOOK: query: DESC FUNCTION EXTENDED aes_encrypt
 POSTHOOK: type: DESCFUNCTION
 aes_encrypt(input string/binary, key string/binary) - Encrypt input using AES.
 AES (Advanced Encryption Standard) algorithm. Key lengths of 128, 192 or 256 
bits can be used. 192 and 256 bits keys can be used if Java Cryptography 
Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If 
either argument is NULL or the key length is not one of the permitted values, 
the return value is NULL.
+Note: this function uses AES in ECB mode (AES/ECB/PKCS5Padding, MySQL-style) 
with no IV and no authentication. Encryption is deterministic: equal inputs 
encrypted with the same key produce equal outputs, so anyone who can read the 
ciphertext can detect duplicates and infer value frequencies, and ciphertexts 
are malleable. It is not suitable for protecting columns with repetitive or 
low-entropy values.
 Example: > SELECT base64(aes_encrypt('ABC', '1234567890123456'));
  'y6Ss+zCYObpCbgfWfyNWTw=='
 Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFAesEncrypt
diff --git a/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out 
b/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
index e823d829a8f..893336058cf 100644
--- a/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
+++ b/ql/src/test/results/clientpositive/llap/udf_mask_hash.q.out
@@ -12,6 +12,11 @@ Examples:
    mask_hash(value)
  Arguments:
    value - value to mask. Supported types: STRING, VARCHAR, CHAR
+ Note:
+   The result is an unsalted, unkeyed SHA-256 digest (SHA-512 if 
hive.masking.algo=sha512).
+   It is deterministic, so values from small or predictable domains (e.g. 
phone numbers,
+   national IDs, dates) can be recovered by hashing every candidate value. For 
such
+   columns prefer a nullifying or redacting mask.
 Function class:org.apache.hadoop.hive.ql.udf.generic.GenericUDFMaskHash
 Function type:BUILTIN
 PREHOOK: query: explain select mask_hash('TestString-123')

Reply via email to