[
https://issues.apache.org/jira/browse/HIVE-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683340#comment-17683340
]
László Bodor edited comment on HIVE-26984 at 2/2/23 11:00 AM:
--------------------------------------------------------------
hey [~kgyrtkirk], thanks for your input, 2 different aspects here:
1. after thinking about the discussion with [~zabetak] on the PR of HIVE-26984,
I decided to postpone the hiding of constructors to another patch (which should
be -1ed if we have evidence that 3rd parties rely on HiveConf constructors
heavily), so HIVE-26984 will only contain the new *create* methods, this way
I'll still have the chance to finish this one
2. regarding agents (so related to HIVE-26985): okay, I agree that agents are a
good thing (especially if it saves us from reinventing the steel), but let's
please consider the installation burden too: the agent approach already
involves some messing with an agent (custom JVM args), why wouldn't we go for a
simple setting like:
{code}
set hive.conf.property.tracking=true;
{code}
I haven't had to implement huge stuff to achieve this (look at this [single
commit|https://github.com/apache/hive/pull/4002/commits/ade1a24716dd9db03822fa7e630f9dcca33d2adf]),
so going for the agent approach looks simply ineffective: we're doing horrible
things to HiveConf, everywhere (so: locally, on cluster, on dev, on customer
side, etc.), I want to make them discoverable as easy as possible, e.g. a
scenario when tez settings don't make their way to the application master (fun
fact: this is happening nowadays), what do you feel like doing:
1. ask the customer to set the above option and rerun the query
2. introduce this agent stuff to hiveserver2 + tez, restart the world, and then
rerun the query
was (Author: abstractdog):
hey [~kgyrtkirk], thanks for your input, 2 different aspects here:
1. after thinking about the discussion with [~zabetak] on the PR of HIVE-26984,
I decided to postpone the hiding of constructors to another patch (which should
be -1ed if we have evidence that 3rd parties rely on HiveConf constructors
heavily), so HIVE-26984 will only contain the new *create* methods, this way
I'll still have the chance to finish this one
2. regarding agents (so related to HIVE-26985): okay, I agree that agents are a
good thing (especially if it saves us from reinventing the steel), but let's
please consider the installation burden too: the agent approach already
involves some messing with an agent (custom JVM args), why wouldn't we go for a
simple setting like:
{code}
set hive.conf.property.tracking=true;
{code}
I haven't had to implement huge stuff to achieve this (look at this [single
commit|https://github.com/apache/hive/pull/4002/commits/ade1a24716dd9db03822fa7e630f9dcca33d2adf],
so going for the agent approach looks simply ineffective: we're doing horrible
things to HiveConf, everywhere (so: locally, on cluster, on dev, on customer
side, etc.), I want to make them discoverable as easy as possible, e.g. a
scenario when tez settings don't make their way to the application master (fun
fact: this is happening nowadays), what do you feel like doing:
1. ask the customer to set the above option and rerun the query
2. introduce this agent stuff to hiveserver2 + tez, restart the world, and then
rerun the query
> Deprecate public HiveConf constructors
> --------------------------------------
>
> Key: HIVE-26984
> URL: https://issues.apache.org/jira/browse/HIVE-26984
> Project: Hive
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> From time to time we investigate configuration object problems that are hard
> to investigate. We can improve this area, e.g. with HIVE-26985, but first, we
> need to introduce a public static factory method to hook into the creation
> process. I can see this pattern in another projects as well, like:
> HBaseConfiguration.
> Creating custom HiveConf subclasses can be useful because putting optional
> (say: if else branches or whatever) stuff into the original HiveConf object's
> hot codepaths can turn it less performant instantly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
