[ https://issues.apache.org/jira/browse/HIVE-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425468#comment-13425468 ]
Travis Crawford commented on HIVE-3253: --------------------------------------- I ran the full test suite ([test report|https://travis.ci.cloudbees.com/job/HIVE-3318_moar_nesting/1/testReport/]) increasing the number of separators to 16 and there were two failures: * org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_nulls * org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 Neither look related to this change, and they still failed after reverting the change & rerunning just those tests. AFAICT increasing this constant is a safe thing to do. > ArrayIndexOutOfBounds exception for deeply nested structs > --------------------------------------------------------- > > Key: HIVE-3253 > URL: https://issues.apache.org/jira/browse/HIVE-3253 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers > Affects Versions: 0.9.0 > Reporter: Swarnim Kulkarni > Assignee: Travis Crawford > > It was observed that while creating table with deeply nested structs might > throw this exception: > {code} > java.lang.ArrayIndexOutOfBoundsException: 9 > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:281) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:263) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObjectInspector(LazyFactory.java:276) > at > org.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyStructInspector(LazyFactory.java:354) > {code} > The reason being that currently the separators array has been hardcoded to be > of size 8 in the LazySimpleSerde. > {code} > // Read the separators: We use 8 levels of separators by default, but we > // should change this when we allow users to specify more than 10 levels > // of separators through DDL. > serdeParams.separators = new byte[8]; > {code} > If possible, we should increase this size or at least make it configurable to > properly handle deeply nested structs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira