[GitHub] [hive] Aggarwal-Raghav opened a new pull request, #4664: HIVE-27662: Incorrect parsing of nested complex types containing map during vectorized text processing

[via GitHub]

Aggarwal-Raghav opened a new pull request, #4664:
URL: https://github.com/apache/hive/pull/4664

   ### What changes were proposed in this pull request?
   When reading a text table with vectorization on and 
hive.fetch.task.conversion as none, wrong parsing of delimiter is happening in 
nested complex types containing map. For example, if a columns schema is like: 
map<string,struct<id:string,name:string>> then \u0004 char is coming in the 
output. I have attached the q files along the patch.
   
   Create table command:
   `create EXTERNAL table `test` as
   select
     'bob' as name,
     map(
         "Map_Key1",
           named_struct(
               'Id',
               'Id_Value1',
               'Name',
               'Name_Value1'
           ),
         "Map_Key2",
           named_struct(
               'Id',
               'Id_Value2',
               'Name',
               'Name_Value2'
           )
     ) as testmarks;
   `
   
   Ouput Before Patch:
   `bob·    
{"Map_Key1":{"id":"Id_Value1\u0004Name_Value1","name":null},"Map_Key2":{"id":"Id_Value2\u0004Name_Value2","name":null}}`
   
   Expected Output:
   `bob·    
{"Map_Key1":{"id":"Id_Value1","name":"Name_Value1"},"Map_Key2":{"id":"Id_Value2","name":"Name_Value2"}}`
   
   
   ### Why are the changes needed?
   Parsing for MAP Complex type is not working which will result in 
incorrect/wrong/corrupt data.
   
   ### Does this PR introduce _any_ user-facing change?
   NO
   
   
   
   ### How was this patch tested?
   Wrote q files for this patch.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org