[jira] [Created] (HIVE-22161) UDF: FunctionRegistry should cache statefulness of UDF
Gopal V created HIVE-22161: -- Summary: UDF: FunctionRegistry should cache statefulness of UDF Key: HIVE-22161 URL: https://issues.apache.org/jira/browse/HIVE-22161 Project: Hive Issue Type: Bug Components: UDF Reporter: Gopal V There's a hidden synchronization across threads when looking up isStateful and isDeterministic. https://github.com/apache/hive/blob/master/common/src/java/org/apache/hive/common/util/AnnotationUtils.java#L27 {code} // to avoid https://bugs.openjdk.java.net/browse/JDK-7122142 public static T getAnnotation(Class clazz, Class annotationClass) { synchronized (annotationClass) { return clazz.getAnnotation(annotationClass); } } {code} This is serializing multiple threads initializing UDFs (or checking them during compilation) & also being locked across threads for each instance of GenericUDFOpEqual in the specific scenario. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HIVE-22160) Job Configuration Properties Unable to Override hive-site.xml Properties in Master Node
Haotian Zhang created HIVE-22160: Summary: Job Configuration Properties Unable to Override hive-site.xml Properties in Master Node Key: HIVE-22160 URL: https://issues.apache.org/jira/browse/HIVE-22160 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 3.1.2 Reporter: Haotian Zhang Assignee: Haotian Zhang When running on the master node, these two ways of creating HiveConf can result in different Hive configurations: {code:java} HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration()); {code} the above one referred as Style1, and {code:java} HiveConf hiveConf = new HiveConf(); hiveConf.addResource(job.getConfiguration()); {code} the above one referred as Style2. If there are properties exist both in hiveSite and jobConf. For example, in hive-site.xml, we have property {noformat} hive.metastore.uris = AAA{noformat} And if at the same time in jobConf, such as action configurations in Oozie's workflow.xml, we have {noformat} hive.metastore.uris = BBB{noformat} Then in Style1, the "hive.metastore.uris" property in hiveConf is "AAA" and in Style2, the property in hiveConf is "BBB". Here is another example to illustrate this. Suppose in hiveSite, "hive.metastore.uris" is AAA and in jobConf it is BBB, when executing the following code: {code:java} HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration()); log.info("the hive metastore uri in hiveconf first is: " + hiveConf.get("hive.metastore.uris")); hiveConf = HCatUtil.getHiveConf(job.getConfiguration()); log.info("the hive metastore uri in hiveconf second is: " + hiveConf.get("hive.metastore.uris")); {code} And the log output is: {noformat} the hive metastore uri in hiveconf first is: AAA the hive metastore uri in hiveconf first is: BBB{noformat} In all, the jobConf does not got set to the configuration as expected when there's already such a property existed in hive site. This happen only on the master node but not on task nodes. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HIVE-22158) HMS Translation layer - Disallow non-ACID MANAGED tables.
Naveen Gangam created HIVE-22158: Summary: HMS Translation layer - Disallow non-ACID MANAGED tables. Key: HIVE-22158 URL: https://issues.apache.org/jira/browse/HIVE-22158 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam In the recent commits, we have allowed non-ACID MANAGED tables to be created by clients that have some form of ACID WRITE capabilities. I think it would make sense to disallow this entirely. MANAGED tables should be ACID tables only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HIVE-22159) HMS Translation layer - Turn off HMS Translation by default.
Naveen Gangam created HIVE-22159: Summary: HMS Translation layer - Turn off HMS Translation by default. Key: HIVE-22159 URL: https://issues.apache.org/jira/browse/HIVE-22159 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam Because of certain backward incompatibilities in terms of behavior, I think it makes sense to turn off this translation in the Apache Hive codebase. Consumers can selectively enable it and even plugin their own set of translation rules as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)