Darryl Dutton created HIVE-20628: ------------------------------------ Summary: Parsing error when using a complex map data type under dynamic column masking Key: HIVE-20628 URL: https://issues.apache.org/jira/browse/HIVE-20628 Project: Hive Issue Type: Bug Components: Hive, HiveServer2, Parser, Security Affects Versions: 2.1.0 Environment: The error can be simulated using HDP 2.6.4 sandbox Reporter: Darryl Dutton
When trying to use the map complex data type as part of dynamic column mask, Hive throws a parsing error as it is expecting a primitive type (see trace pasted below). The use case is trying to apply masking to elements within a map type by applying a custom hive UDF (to apply the mask) using Ranger. Expect Hive to support complex data types for masking in addition to the primitive types. The expectation occurs when Hive need to evaluate the UDF or apply a standard mask (pass-through works as expected). You can recreate the problem by creating a simple table with a map data type column, then applying the masking to that column through a Ranger resource based policy and a custom function (you can use a standard Hive UDF str_to_map('F4','XXXX') to simulate returning a map). CREATE TABLE `mask_test`( `key` string, `value` map<string,string>) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' INSERT INTO TABLE mask_test SELECT 'AAA' as key, map('F1','2022','F2','2222','F3','3333333') as value FROM (select 1 ) as temp; Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.SemanticException:org.apache.hadoop.hive.ql.parse.ParseException: line 1:57 cannot recognize input near 'map' '<' 'string' in primitive type specification at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10370) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10486) at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:219) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:465) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:321) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1224) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1218) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:146) ... 15 more Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.ParseException:line 1:57 cannot recognize input near 'map' '<' 'string' in primitive type specification at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:214) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:171) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10368) -- This message was sent by Atlassian JIRA (v7.6.3#76005)