[jira] [Commented] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681343#comment-16681343
]
Bohdan Chupika commented on HIVE-17477:
---
Reattached the patch for jenkins rebuild
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch, HIVE-17477.4.patch, HIVE-17477.5.patch, HIVE-17477.6.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.6.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch, HIVE-17477.4.patch, HIVE-17477.5.patch, HIVE-17477.6.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.5.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch, HIVE-17477.4.patch, HIVE-17477.5.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-18902:
--
Attachment: HIVE-18902.04.patch
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch,
> HIVE-18902.03.patch, HIVE-18902.04.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-18902:
--
Attachment: HIVE-18902.03.patch
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch,
> HIVE-18902.03.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447902#comment-16447902
]
Bohdan Chupika commented on HIVE-17477:
---
[~ashutoshc] could you review, please?
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch, HIVE-17477.4.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.4.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch, HIVE-17477.4.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447603#comment-16447603
]
Bohdan Chupika commented on HIVE-18902:
---
[~belugabehr] could you review, please?
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443609#comment-16443609
]
Bohdan Chupika commented on HIVE-18902:
---
What's wrong with ASF license in hive/ql/exec/spark/TestSparkPlan.java?)
I haven't changing this file :)
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441148#comment-16441148
]
Bohdan Chupika commented on HIVE-18902:
---
[~belugabehr] Thank you, for your reply! I see your point, fixed it in the new
patch HIVE-18902.02.patch
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-18902:
--
Attachment: HIVE-18902.02.patch
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch, HIVE-18902.02.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.3.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch,
> HIVE-17477.3.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437182#comment-16437182
]
Bohdan Chupika commented on HIVE-17477:
---
HIVE-17477.2.patch fixed checkstyle
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.2.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch, HIVE-17477.2.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437170#comment-16437170
]
Bohdan Chupika commented on HIVE-18902:
---
[~belugabehr] Could you review please?
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-18902:
--
Status: Patch Available (was: Open)
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-18902:
--
Attachment: HIVE-18902.01.patch
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
> Attachments: HIVE-18902.01.patch
>
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18902) Lower Logging Level for Cleaning Up "local RawStore"
[
https://issues.apache.org/jira/browse/HIVE-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika reassigned HIVE-18902:
-
Assignee: Bohdan Chupika
> Lower Logging Level for Cleaning Up "local RawStore"
>
>
> Key: HIVE-18902
> URL: https://issues.apache.org/jira/browse/HIVE-18902
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Bohdan Chupika
>Priority: Trivial
> Labels: noob
>
> [https://github.com/apache/hive/blob/7c22d74c8d0eb0650adf6e84e0536127c103e46c/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L7756-L7768]
>
> {code:java}
> private static void cleanupRawStore() {
> try {
> RawStore rs = HMSHandler.getRawStore();
> if (rs != null) {
> HMSHandler.logInfo("Cleaning up thread local RawStore...");
> rs.shutdown();
> }
> } finally {
> HMSHandler handler = HMSHandler.threadLocalHMSHandler.get();
> if (handler != null) {
> handler.notifyMetaListenersOnShutDown();
> }
> HMSHandler.threadLocalHMSHandler.remove();
> HMSHandler.threadLocalConf.remove();
> HMSHandler.threadLocalModifiedConfig.remove();
> HMSHandler.removeRawStore();
> HMSHandler.logInfo("Done cleaning up thread local RawStore");
> }
> }
> {code}
> {code}
> 2018-03-03 17:21:49,832 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19:
> Cleaning up thread local RawStore...
> 2018-03-03 17:21:49,834 INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-4-thread-21]: 19: Done
> cleaning up thread local RawStore
> {code}
> Not very helpful logging. Please change logging levels to _debug_ or even
> _trace_
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Attachment: HIVE-17477.1.patch
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika updated HIVE-17477:
--
Status: Patch Available (was: Open)
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
> Attachments: HIVE-17477.1.patch
>
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-17477) PidFilePatternConverter not rolling log according to their pid
[
https://issues.apache.org/jira/browse/HIVE-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bohdan Chupika reassigned HIVE-17477:
-
Assignee: Bohdan Chupika
> PidFilePatternConverter not rolling log according to their pid
> ---
>
> Key: HIVE-17477
> URL: https://issues.apache.org/jira/browse/HIVE-17477
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Reporter: Vlad Gudikov
>Assignee: Bohdan Chupika
>Priority: Major
>
> We can use pid in th filePattern:
> appender.DRFA.filePattern =
> ${sys:hive.log.dir}/${sys:hive.log.file}.%d{-MM-dd}.%pid
> But when its time roll the logs at the end of the RollingAppender just
> renames hive.log by applying the pattern described above and keeps logs that
> are not related to the process this log was named by. The issue is that all
> processes are writing to the same log and we cannot separate them by pid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
