[jira] [Commented] (HIVE-26985) Create a trackable hive configuration object
[ https://issues.apache.org/jira/browse/HIVE-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17683330#comment-17683330 ] Zoltan Haindrich commented on HIVE-26985: - I think you could probably achieve something similar by using AspectJ or Byteman or other java agent stuff; or write your own agent: https://stackify.com/what-are-java-agents-and-how-to-profile-with-them/ I will -1 the current patch because it makes a significant API change by making the HiveConf constructor protected - which will break all 3rd party extensions which may use `new HiveConf()` > Create a trackable hive configuration object > > > Key: HIVE-26985 > URL: https://issues.apache.org/jira/browse/HIVE-26985 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: hive.log > > Time Spent: 10m > Remaining Estimate: 0h > > During configuration-related investigations, I want to be able to easily find > out when and how a certain configuration is changed. I'm looking for an > improvement that simply logs if "hive.a.b.c" is changed from "hello" to > "asdf" or even null and on which thread/codepath. > Not sure if there is already a trackable configuration object in hadoop that > we can reuse, or we need to implement it in hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26985) Create a trackable hive configuration object
[ https://issues.apache.org/jira/browse/HIVE-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682604#comment-17682604 ] László Bodor commented on HIVE-26985: - attached [^hive.log] for the initial implementation, look for TrackedHiveConf > Create a trackable hive configuration object > > > Key: HIVE-26985 > URL: https://issues.apache.org/jira/browse/HIVE-26985 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: hive.log > > > During configuration-related investigations, I want to be able to easily find > out when and how a certain configuration is changed. I'm looking for an > improvement that simply logs if "hive.a.b.c" is changed from "hello" to > "asdf" or even null and on which thread/codepath. > Not sure if there is already a trackable configuration object in hadoop that > we can reuse, or we need to implement it in hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26985) Create a trackable hive configuration object
[ https://issues.apache.org/jira/browse/HIVE-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680576#comment-17680576 ] Ayush Saxena commented on HIVE-26985: - I haven't heard anything similar in Hadoop, It doesn't exist or may be just my lack of awareness, but I feel it is a good thing to have. Can help a lot in debugging those crazy race conditions and corner cases. Have a suggestion as well here: May be we can have a separate config like: tracable-config which can take some csv's and if anyone of the configs in the value are modified we can log it. Doing for all or always might have some adverse performance impacts, and secondly can be explored in Hadoop itself also. > Create a trackable hive configuration object > > > Key: HIVE-26985 > URL: https://issues.apache.org/jira/browse/HIVE-26985 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Priority: Major > > During configuration-related investigations, I want to be able to easily find > out when and how a certain configuration is changed. I'm looking for an > improvement that simply logs if "hive.a.b.c" is changed from "hello" to > "asdf" or even null and on which thread/codepath. > Not sure if there is already a trackable configuration object in hadoop that > we can reuse, or we need to implement it in hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26985) Create a trackable hive configuration object
[ https://issues.apache.org/jira/browse/HIVE-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680531#comment-17680531 ] Stamatis Zampetakis commented on HIVE-26985: Very interesting idea, that will be very useful. Possibly after this change we will not need to have snippets like this in the code: https://github.com/apache/hive/blob/e6e2f41e6f4c422281eb5803828ec7f544a01a9a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java#L169 > Create a trackable hive configuration object > > > Key: HIVE-26985 > URL: https://issues.apache.org/jira/browse/HIVE-26985 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Priority: Major > > During configuration-related investigations, I want to be able to easily find > out when and how a certain configuration is changed. I'm looking for an > improvement that simply logs if "hive.a.b.c" is changed from "hello" to > "asdf" or even null and on which thread/codepath. > Not sure if there is already a trackable configuration object in hadoop that > we can reuse, or we need to implement it in hive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
