[jira] [Created] (HIVE-22161) UDF: FunctionRegistry should cache statefulness of UDF
Gopal V created HIVE-22161:
--
Summary: UDF: FunctionRegistry should cache statefulness of UDF
Key: HIVE-22161
URL: https://issues.apache.org/jira/browse/HIVE-22161
Project: Hive
Issue Type: Bug
Components: UDF
Reporter: Gopal V
There's a hidden synchronization across threads when looking up isStateful and
isDeterministic.
https://github.com/apache/hive/blob/master/common/src/java/org/apache/hive/common/util/AnnotationUtils.java#L27
{code}
// to avoid https://bugs.openjdk.java.net/browse/JDK-7122142
public static T getAnnotation(Class clazz, Class
annotationClass) {
synchronized (annotationClass) {
return clazz.getAnnotation(annotationClass);
}
}
{code}
This is serializing multiple threads initializing UDFs (or checking them during
compilation) & also being locked across threads for each instance of
GenericUDFOpEqual in the specific scenario.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HIVE-22160) Job Configuration Properties Unable to Override hive-site.xml Properties in Master Node
Haotian Zhang created HIVE-22160:
Summary: Job Configuration Properties Unable to Override
hive-site.xml Properties in Master Node
Key: HIVE-22160
URL: https://issues.apache.org/jira/browse/HIVE-22160
Project: Hive
Issue Type: Bug
Components: Configuration
Affects Versions: 3.1.2
Reporter: Haotian Zhang
Assignee: Haotian Zhang
When running on the master node, these two ways of creating HiveConf can result
in different Hive configurations:
{code:java}
HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
{code}
the above one referred as Style1, and
{code:java}
HiveConf hiveConf = new HiveConf();
hiveConf.addResource(job.getConfiguration());
{code}
the above one referred as Style2. If there are properties exist both in
hiveSite and jobConf. For example, in hive-site.xml, we have property
{noformat}
hive.metastore.uris = AAA{noformat}
And if at the same time in jobConf, such as action configurations in Oozie's
workflow.xml, we have
{noformat}
hive.metastore.uris = BBB{noformat}
Then in Style1, the "hive.metastore.uris" property in hiveConf is "AAA" and in
Style2, the property in hiveConf is "BBB".
Here is another example to illustrate this. Suppose in hiveSite,
"hive.metastore.uris" is AAA and in jobConf it is BBB, when executing the
following code:
{code:java}
HiveConf hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
log.info("the hive metastore uri in hiveconf first is: " +
hiveConf.get("hive.metastore.uris"));
hiveConf = HCatUtil.getHiveConf(job.getConfiguration());
log.info("the hive metastore uri in hiveconf second is: " +
hiveConf.get("hive.metastore.uris"));
{code}
And the log output is:
{noformat}
the hive metastore uri in hiveconf first is: AAA
the hive metastore uri in hiveconf first is: BBB{noformat}
In all, the jobConf does not got set to the configuration as expected when
there's already such a property existed in hive site. This happen only on the
master node but not on task nodes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HIVE-22158) HMS Translation layer - Disallow non-ACID MANAGED tables.
Naveen Gangam created HIVE-22158: Summary: HMS Translation layer - Disallow non-ACID MANAGED tables. Key: HIVE-22158 URL: https://issues.apache.org/jira/browse/HIVE-22158 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam In the recent commits, we have allowed non-ACID MANAGED tables to be created by clients that have some form of ACID WRITE capabilities. I think it would make sense to disallow this entirely. MANAGED tables should be ACID tables only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HIVE-22159) HMS Translation layer - Turn off HMS Translation by default.
Naveen Gangam created HIVE-22159: Summary: HMS Translation layer - Turn off HMS Translation by default. Key: HIVE-22159 URL: https://issues.apache.org/jira/browse/HIVE-22159 Project: Hive Issue Type: Sub-task Components: Hive Affects Versions: 4.0.0 Reporter: Naveen Gangam Assignee: Naveen Gangam Because of certain backward incompatibilities in terms of behavior, I think it makes sense to turn off this translation in the Apache Hive codebase. Consumers can selectively enable it and even plugin their own set of translation rules as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
