[
https://issues.apache.org/jira/browse/IMPALA-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-9086:
-----------------------------------
Description:
Hive configurations are not shown in /hadoop-varz page. It helps to check
hive.metastore.warehouse.dir and hive.metastore.warehouse.external.dir used in
FE. We should also add the /hadoop-varz page in catalogd WebUI.
The URL callback is registered in be/src/service/impala-http-handler.cc:
{code:cpp}
webserver->RegisterUrlCallback("/hadoop-varz", "hadoop-varz.tmpl",
MakeCallback(this, &ImpalaHttpHandler::HadoopVarzHandler), true);
{code}
https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/service/impala-http-handler.cc#L133-L134
The handler, ImpalaHttpHandler::HadoopVarzHandler(), is implemented by calling
a JNI method to invoke FE method JniFrontend.getAllHadoopConfigs():
https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L626-L631
We simply create a static object of org.apache.hadoop.conf.Configuration and
use it for each /hadoop-varz http request. We can consier changing its type to
org.apache.hadoop.hive.conf.HiveConf (inherrits Configuration) which might
contain all the hive configurations.
To add the same page for catalogd, we need to register a similar handler in
CatalogServer::RegisterWebpages()
https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/catalog/catalog-server.cc#L434
Also add a new method in JniCatalog.java to return the configurations throught
JNI, add JNI stuffs in catalog.cc and catalog.h (similar to existing JNI
methods, e.g. get_catalog_object_id_).
was:Hive configurations are not shown in /hadoop-varz page. It helps to check
hive.metastore.warehouse.dir and hive.metastore.warehouse.external.dir used in
FE.
> Show Hive configurations in /hadoop-varz page
> ---------------------------------------------
>
> Key: IMPALA-9086
> URL: https://issues.apache.org/jira/browse/IMPALA-9086
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Saurabh Katiyal
> Priority: Major
> Labels: ramp-up, supportability
>
> Hive configurations are not shown in /hadoop-varz page. It helps to check
> hive.metastore.warehouse.dir and hive.metastore.warehouse.external.dir used
> in FE. We should also add the /hadoop-varz page in catalogd WebUI.
> The URL callback is registered in be/src/service/impala-http-handler.cc:
> {code:cpp}
> webserver->RegisterUrlCallback("/hadoop-varz", "hadoop-varz.tmpl",
> MakeCallback(this, &ImpalaHttpHandler::HadoopVarzHandler), true);
> {code}
> https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/service/impala-http-handler.cc#L133-L134
> The handler, ImpalaHttpHandler::HadoopVarzHandler(), is implemented by
> calling a JNI method to invoke FE method JniFrontend.getAllHadoopConfigs():
> https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L626-L631
> We simply create a static object of org.apache.hadoop.conf.Configuration and
> use it for each /hadoop-varz http request. We can consier changing its type
> to org.apache.hadoop.hive.conf.HiveConf (inherrits Configuration) which might
> contain all the hive configurations.
> To add the same page for catalogd, we need to register a similar handler in
> CatalogServer::RegisterWebpages()
> https://github.com/apache/impala/blob/379038f7639731605bca4356337616fa69f35f9d/be/src/catalog/catalog-server.cc#L434
> Also add a new method in JniCatalog.java to return the configurations
> throught JNI, add JNI stuffs in catalog.cc and catalog.h (similar to existing
> JNI methods, e.g. get_catalog_object_id_).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
