[
https://issues.apache.org/jira/browse/LENS-408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362749#comment-14362749
]
Amareshwari Sriramadasu commented on LENS-408:
----------------------------------------------
Review request : https://reviews.apache.org/r/32043/
After the changes put up, following are all the occurrences of HiveConf
creation in code.
* No occurrences
{noformat}
lens-api amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
lens-cli amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
lens-client amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
lens-driver-jdbc amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
ens-storage-db amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
lens-query-lib amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
lens-examples amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
{noformat}
* lens-cube module :
{noformat}
lens-cube amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java:
this.config = new HiveConf(conf);
src/main/java/org/apache/lens/cube/parse/HQLParser.java: ASTNode ast =
parseHQL("select * from default_table ", new HiveConf());
{noformat}
** There is HiveConf created when creating CubeMetastoreClient. One call per db
** The one HQLParser.java is in main() method. mainly for debugging purpose.
Does not come usual code flow.
* Hive Driver module :
{noformat}
lens-driver-hive amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
src/main/java/org/apache/lens/driver/hive/HiveDriver.java: // resources have
to be added separately on hiveConf again because new HiveConf() overrides
hive.* properties
src/main/java/org/apache/lens/driver/hive/HiveDriver.java: this.hiveConf =
new HiveConf(driverConf, HiveDriver.class);
src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java: //
new HiveConf() is getting created because connection will be different for each
user
src/main/java/org/apache/lens/driver/hive/RemoteThriftConnection.java:
this.conf = new HiveConf(conf);
{noformat}
** One occurrence is during driver initialization
** One occurrence is during open connection with HiveServer - will be called
once per lens session and thread.
* Created LENS-409 for tracking occurrences on lens-ml-lib
* lens-server module :
{noformat}
lens-server amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
src/main/java/org/apache/lens/server/LensServerConf.java: public static
final HiveConf CONF = new HiveConf();
src/main/java/org/apache/lens/server/LensServerConf.java: return new
HiveConf(ConfHolder.CONF);
{noformat}
** Once during server initialization
** One will be called from base class of LensSessionImpl - once per session.
** The one in LensServerConf method is used in tests
* lens-server-api method :
{noformat}
lens-server-api amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java:
hiveConf = new HiveConf(this.conf, this.getClass());
{noformat}
** Once per query.
> Creating HiveConf multiple times in estimate flow should be avoided
> -------------------------------------------------------------------
>
> Key: LENS-408
> URL: https://issues.apache.org/jira/browse/LENS-408
> Project: Apache Lens
> Issue Type: Improvement
> Components: driver-hive, driver-jdbc, server
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Fix For: 2.1
>
>
> We are seeing creation of new HiveConf() is taking around 20 milli seconds on
> the average. And is called in many places in code without much need.
> This issue is to look at all such occurrences and fix them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
