[
https://issues.apache.org/jira/browse/IMPALA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-11948:
------------------------------------
Description:
After IMPALA-11478, The log file of catalogd is full of the following logs even
when the server is idle:
{noformat}
I0226 16:55:27.111258 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: 2ms
I0226 16:55:28.111685 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:28.111811 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:29.112185 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:29.112324 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: 1ms
I0226 16:55:30.112716 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:30.112885 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:31.113215 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:31.113399 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:32.113715 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:32.113834 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:33.114199 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:33.114336 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: {noformat}
getCatalogServerMetrics() is invoked every second by catalogd itself:
{code:cpp}
[[noreturn]] void CatalogServer::RefreshMetrics() {
while (true) {
SleepForMs(REFRESH_METRICS_INTERVAL_MS);
TGetCatalogServerMetricsResponse response;
Status status = catalog_->GetCatalogServerMetrics(&response);
if (!status.ok()) {
LOG(ERROR) << "Error refreshing catalog metrics: " << status.GetDetail();
continue;
}
partial_fetch_rpc_queue_len_metric_->SetValue(
response.catalog_partial_fetch_rpc_queue_len);
TEventProcessorMetrics eventProcessorMetrics = response.event_metrics;
MetastoreEventMetrics::refresh(&eventProcessorMetrics);
}
}
{code}
https://github.com/apache/impala/blob/d0592c0dbf4d634cc26b2e9872a9f62af5249cdd/be/src/catalog/catalog-server.cc#L464-L478
I think it's useless to log these requests. CC [~prozsa] [~csringhofer]
was:
After IMPALA-11478, The log file of catalogd is full of the following logs even
when the server is idle:
{code:noformat}
I0226 16:55:27.111258 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: 2ms
I0226 16:55:28.111685 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:28.111811 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:29.112185 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:29.112324 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: 1ms
I0226 16:55:30.112716 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:30.112885 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:31.113215 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:31.113399 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:32.113715 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:32.113834 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent:
I0226 16:55:33.114199 2816 JniUtil.java:157] getCatalogServerMetrics request:
Get catalog server metrics
I0226 16:55:33.114336 2816 JniUtil.java:161] Finished getCatalogServerMetrics
request: Get catalog server metrics. Time spent: {code}
getCatalogServerMetrics() is invoked every second by catalogd itself:
{code:cpp}
[[noreturn]] void CatalogServer::RefreshMetrics() {
while (true) {
SleepForMs(REFRESH_METRICS_INTERVAL_MS);
TGetCatalogServerMetricsResponse response;
Status status = catalog_->GetCatalogServerMetrics(&response);
if (!status.ok()) {
LOG(ERROR) << "Error refreshing catalog metrics: " << status.GetDetail();
continue;
}
partial_fetch_rpc_queue_len_metric_->SetValue(
response.catalog_partial_fetch_rpc_queue_len);
TEventProcessorMetrics eventProcessorMetrics = response.event_metrics;
MetastoreEventMetrics::refresh(&eventProcessorMetrics);
}
}
{code}
https://github.com/apache/impala/blob/d0592c0dbf4d634cc26b2e9872a9f62af5249cdd/be/src/catalog/catalog-server.cc#L464-L478
I think it's useless to log these requests. CC [~prozsa] [~csringhofer]
> Don't log getCatalogServerMetrics requests in catalogd
> ------------------------------------------------------
>
> Key: IMPALA-11948
> URL: https://issues.apache.org/jira/browse/IMPALA-11948
> Project: IMPALA
> Issue Type: Task
> Components: Frontend
> Reporter: Quanlong Huang
> Priority: Major
>
> After IMPALA-11478, The log file of catalogd is full of the following logs
> even when the server is idle:
> {noformat}
> I0226 16:55:27.111258 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent: 2ms
> I0226 16:55:28.111685 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:28.111811 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:29.112185 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:29.112324 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent: 1ms
> I0226 16:55:30.112716 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:30.112885 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:31.113215 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:31.113399 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:32.113715 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:32.113834 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:33.114199 2816 JniUtil.java:157] getCatalogServerMetrics
> request: Get catalog server metrics
> I0226 16:55:33.114336 2816 JniUtil.java:161] Finished
> getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> {noformat}
> getCatalogServerMetrics() is invoked every second by catalogd itself:
> {code:cpp}
> [[noreturn]] void CatalogServer::RefreshMetrics() {
> while (true) {
> SleepForMs(REFRESH_METRICS_INTERVAL_MS);
> TGetCatalogServerMetricsResponse response;
> Status status = catalog_->GetCatalogServerMetrics(&response);
> if (!status.ok()) {
> LOG(ERROR) << "Error refreshing catalog metrics: " <<
> status.GetDetail();
> continue;
> }
> partial_fetch_rpc_queue_len_metric_->SetValue(
> response.catalog_partial_fetch_rpc_queue_len);
> TEventProcessorMetrics eventProcessorMetrics = response.event_metrics;
> MetastoreEventMetrics::refresh(&eventProcessorMetrics);
> }
> }
> {code}
> https://github.com/apache/impala/blob/d0592c0dbf4d634cc26b2e9872a9f62af5249cdd/be/src/catalog/catalog-server.cc#L464-L478
> I think it's useless to log these requests. CC [~prozsa] [~csringhofer]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
