[
https://issues.apache.org/jira/browse/IMPALA-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer resolved IMPALA-9609.
-------------------------------------
Fix Version/s: Impala 4.0
Resolution: Implemented
> Minimize Frontend activity in executor only Impalas
> ---------------------------------------------------
>
> Key: IMPALA-9609
> URL: https://issues.apache.org/jira/browse/IMPALA-9609
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Critical
> Fix For: Impala 4.0
>
>
> Currently Impala both starts a JVM and creates a Frontend in executor mode
> impalads:
> JVM:
> https://github.com/apache/impala/blob/04fd9ae268d89b07e2a692a916bf2ddcfb2e351b/be/src/service/impalad-main.cc#L61
> The 3rd argument of is init_jvm. This is actually needed for some executor
> features, e.g. Hive UDFs and HBase.
> Frontend:
> https://github.com/apache/impala/blob/d877dbc572e95f086b37142e1d41231d5b6b7f9f/be/src/runtime/exec-env.cc#L250
> The frontend code itself doesn't really seem to care about whether it serves
> as a coordinator.
> It would be great to remove the Frontend object from executors completely,
> but this may be tricky as Java features used by the executor can have
> widespread dependencies.
> The critical part is to avoid starting services that are both not needed and
> are problematic, e.g. connecting to HMS and Auth services. These can consume
> resource and cause errors if not configured well.
> The executor only setting is only tested by custom_cluster test, so probably
> we'll need to extend those to ensure that HBase/Hive UDFs are not broken.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
