[
https://issues.apache.org/jira/browse/LENS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sriramadasu updated LENS-409:
-----------------------------------------
Summary: Fix usage of new HiveConf() and Hive.get() in ml lib (was: Fix
usage of new HiveConf() in ml lib)
The following occurrences of Hive.get() should also be checked to use proper
configuration or cube metastore client.
{noformat}
lens-ml-lib amareshwari.sr$ git grep 'Hive.get(' src/main/java/
src/main/java/org/apache/lens/ml/ExampleUtils.java: Table tbl =
Hive.get(conf).newTable(database + "." + tableName);
src/main/java/org/apache/lens/ml/ExampleUtils.java:
Hive.get(conf).createTable(tbl, false);
src/main/java/org/apache/lens/ml/ExampleUtils.java:
Hive.get(conf).createPartitions(partitionDesc);
src/main/java/org/apache/lens/ml/TableTestingSpec.java: Hive
metastoreClient = Hive.get(conf);
src/main/java/org/apache/lens/ml/spark/algos/KMeansAlgo.java: Table tbl =
Hive.get(toHiveConf(conf)).getTable(db, table);
src/main/java/org/apache/lens/rdd/LensRDDClient.java: Hive hiveClient =
Hive.get(HIVE_CONF);
src/main/java/org/apache/lens/rdd/LensRDDClient.java: hiveClient =
Hive.get(HIVE_CONF);
{noformat}
> Fix usage of new HiveConf() and Hive.get() in ml lib
> ----------------------------------------------------
>
> Key: LENS-409
> URL: https://issues.apache.org/jira/browse/LENS-409
> Project: Apache Lens
> Issue Type: Improvement
> Components: ml
> Reporter: Amareshwari Sriramadasu
> Assignee: Jaideep Dhok
> Fix For: 2.1
>
>
> We saw new HiveConf() call is costlier : See LENS-408. The following
> occurrences in ml-lib needs to be looked into to see if they are required.
> {noformat}
> lens-ml-lib amareshwari.sr$ git grep 'new HiveConf(' src/main/java/
> src/main/java/org/apache/lens/ml/MLUtils.java: HIVE_CONF = new HiveConf();
> src/main/java/org/apache/lens/ml/ModelLoader.java: FileSystem fs =
> modelPath.getFileSystem(new HiveConf());
> src/main/java/org/apache/lens/ml/spark/algos/BaseSparkAlgo.java: HiveConf
> hiveConf = new HiveConf();
> src/main/java/org/apache/lens/ml/spark/algos/KMeansAlgo.java: HiveConf
> hiveConf = new HiveConf();
> src/main/java/org/apache/lens/rdd/LensRDDClient.java: protected static final
> HiveConf HIVE_CONF = new HiveConf();
> src/main/java/org/apache/lens/server/ml/MLServiceResource.java: HIVE_CONF
> = new HiveConf();
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
