Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review156012
---




lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 52)


classLoaderCache needs to be updated on addDBJar.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 114)


This method should be private .



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 265 - 268)


This method should be private



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 408 - 414)


Should we decide which map to use based on mode (local/hdfs or remote ) 
which is decided on startup



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 416)


addremoteDbResourceEntry and addLocalDbResourceEntry can be private . 
We can also decide and remove these methods and directly add map entries in 
calling code.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 420)


Should we call this method updateLocalDBResourceEntries since its replacing 
all the entries for the DB ?

Simlaraly updateRemoteDbResourceEntries too



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 434)


QueryExecutionServiceImpl#addSessionResourcesToDriver needs to remove old 
jars added to the hive sessions and add the newly uploaded jar to it (there is 
one hive session per User and per DB. Only the current DB sessions should get 
affected)

This implies that there is no guarantee on success of currently running 
queries on the affected Hive sessions



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 436)


Should we also add DB name to this log ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 476)


jarFilePath has the complete path for the jar file. Do we need currentDB 
and baseDir ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 484)


should we check the status of this opeartion and throw exception in case of 
failure.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 505)


should we resuse FileSystem created while starting the service ? This can 
be done for all FieSystem.newInstance calls in this method



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 508)


should we also check the boolean value returned by this call and proceed 
accrodingly



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 522)


Should we use FileSystem.delete(path,false) instead  and check the boolean 
value returned by this call before proceeding further?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 535)


Should we INFO log that jar uploaded to {path}


- Puneet Gupta


On Nov. 8, 2016, 10:18 a.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 8, 2016, 10:18 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   lens-cli/pom.xml 8e5e3eb 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review155416
---




lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
(lines 837 - 849)


Should we add the new properties to default lens server config file along 
with the description on how and when to use them?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 84)


[Suggestion] Should we simplify this method by assuming there will be only 
one level of folders under the base resource directory ?

If we use this knowledge we can sync one DB at a time and paring the urls 
may not be required.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 87)


Can we reuse "dbResourceFs" ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 97 - 109)


Can we add log entries when jars are downloaded



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 143)


This needs to be moved to end so that service state is changed to STARTED 
in the end . 
org.apache.hive.service.AbstractService#start



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 157 - 159)


What will happen in case the scheme is not hdfs? 
Will the flow work gracefully or throw an execption



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 161 - 163)


What do we do for use cases where the user does not need any DB resources ? 
In those cases it may not be an ERROR to not have top resource directory



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 193 - 194)


Can we set dbLocalResTopDir to dbResTopDir when dbResTopDir is local path. 
This check would not be required then. 

OR

If needed we can also store the mode in start method as a boolean 
(local/hdfs) locally and use it in mapCommonResourceEntries(boolean mode) and 
mapDbResourceEntries(boolean mode)



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 210 - 211)


This will never be true since the DB path is got by listing the base 
directory.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 231)


Can we create the loacl fileSystem object only once (may be in start() 
method and) use it elsewhere as hadoop FileSystem object is a heavy and 
mainatins state/cache inside it.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 233 - 237)


Isn't this condition already checked in the calling method ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 265)


Can we just retun the jar path/name or null in case there is no jar in the 
folder and shouldIncrVersion = false



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 272)


Can we reuase FileSystem instance here too ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 278)


Should this be >=1 ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 446)


Should we use LensConfConstants.DB_RESOURCE_COPY_FROM_HDFS property to 
figure out HDFS vs Local ? 

We can rename the property to DB_RESOURCES_IN_HDFS = true/false?


This review only covers the startup flow. addDBJar Flow Review is pending, will 
review it soon.

- Puneet Gupta


On Nov. 8, 2016, 10:18 a.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 8, 2016, 10:18 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Nov. 8, 2016, 10:18 a.m.)


Review request for lens.


Changes
---

- Addressed review.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/pom.xml 8e5e3eb 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
7fc8438 
  lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8f9db2a 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  
lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
 20ec686 
  lens-server/pom.xml d24dc1e 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 21e2a62 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
63eea63 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/main/resources/lensserver-default.xml a00048b 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/session/TestSessionClassLoaders.java
 d66de4c 
  lens-server/src/test/resources/lens-site.xml d96659f 
  pom.xml 29c59d3 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

> On Nov. 4, 2016, 6:35 a.m., Amareshwari Sriramadasu wrote:
> > lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java, 
> > line 97
> > 
> >
> > Database does not exists, but jar can be added?

It was being added to the default db. Changed it to work with specified db.


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review154869
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens UI ... SUCCESS [19.454s]
> [INFO] 
> 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

> On Oct. 26, 2016, 6:01 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java,
> >  line 169
> > 
> >
> > not sure how its corresponding local jar file is added to classloader. 
> > If hdfs jar is getting added, it wont work.
> > 
> > might have to pass the corresponding local jar as the location value 
> > for ResourceEntry.
> 
> Sushil Mohanty wrote:
> In case of base dir is hdfs, first copying the files to local directory 
> and setting local as base directory and respective jar files are picked from 
> here.
> 
>if 
> (resTopDirPath.getFileSystem(getHiveConf()).getScheme().equals("hdfs")) {
>   localResTopDir = 
> getHiveConf().get(LensConfConstants.DATABASE_LOCAL_RESOURCE_DIR,
>   LensConfConstants.DEFAULT_LOCAL_DATABASE_RESOURCE_DIR);
>   downloadJarFilesFromHDFS(resTopDir, localResTopDir);
>   resTopDir = localResTopDir;
> }
> 
> Amareshwari Sriramadasu wrote:
> We cannot change resTopDir which is hdfs to a local path.

Addressed.


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153851
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

> On Oct. 26, 2016, 6:01 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java,
> >  line 169
> > 
> >
> > not sure how its corresponding local jar file is added to classloader. 
> > If hdfs jar is getting added, it wont work.
> > 
> > might have to pass the corresponding local jar as the location value 
> > for ResourceEntry.
> 
> Sushil Mohanty wrote:
> In case of base dir is hdfs, first copying the files to local directory 
> and setting local as base directory and respective jar files are picked from 
> here.
> 
>if 
> (resTopDirPath.getFileSystem(getHiveConf()).getScheme().equals("hdfs")) {
>   localResTopDir = 
> getHiveConf().get(LensConfConstants.DATABASE_LOCAL_RESOURCE_DIR,
>   LensConfConstants.DEFAULT_LOCAL_DATABASE_RESOURCE_DIR);
>   downloadJarFilesFromHDFS(resTopDir, localResTopDir);
>   resTopDir = localResTopDir;
> }

We cannot change resTopDir which is hdfs to a local path.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153851
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

> On Nov. 4, 2016, 6:35 a.m., Amareshwari Sriramadasu wrote:
> > lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java, 
> > line 104
> > 
> >
> > command should be throwing NullPointerException on non existing jar. It 
> > should be a proper exception with message "File does not exist".

I meant 'Command should not be throwing NullPointerException '


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review154869
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens UI 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review154869
---




lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java (line 
97)


Database does not exists, but jar can be added?



lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java (line 
99)


We cannot checkin a jar file into the source code. Please generate jar file 
at compile time. You can see lens-server/pom.xml on how test jars are generated.



lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java (line 
104)


command should be throwing NullPointerException on non existing jar. It 
should be a proper exception with message "File does not exist".



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 149)


Move serverFs also to class level? And shall we call it dbResourceFs 
instead ?



lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
(line 82)


We should access the instance in sessionService, instead of LensServices. 
Inthat case you wont even need it as class variable.



lens-server/src/test/resources/lens-site.xml (line 149)


We should not be doing this. As DatabaseResourceService is getting created 
in SessionService already, this will make two service instances to be created.


- Amareshwari Sriramadasu


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Nov. 3, 2016, 5:31 p.m.)


Review request for lens.


Changes
---

- Addressed review.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
7fc8438 
  lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8f9db2a 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  
lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
 20ec686 
  lens-server/pom.xml d24dc1e 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 21e2a62 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
63eea63 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/main/resources/lensserver-default.xml a00048b 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
 PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml d96659f 
  pom.xml 29c59d3 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

> On Oct. 26, 2016, 6:01 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java,
> >  line 339
> > 
> >
> > Why are we passing null name? We should remove the param, if it is not 
> > required anymore.

Moved NAME to class level variable and CLIService is passed as null. Super 
class BaseLensService expects CLIService as param.


> On Oct. 26, 2016, 6:01 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java,
> >  line 169
> > 
> >
> > not sure how its corresponding local jar file is added to classloader. 
> > If hdfs jar is getting added, it wont work.
> > 
> > might have to pass the corresponding local jar as the location value 
> > for ResourceEntry.

In case of base dir is hdfs, first copying the files to local directory and 
setting local as base directory and respective jar files are picked from here.

   if 
(resTopDirPath.getFileSystem(getHiveConf()).getScheme().equals("hdfs")) {
  localResTopDir = 
getHiveConf().get(LensConfConstants.DATABASE_LOCAL_RESOURCE_DIR,
  LensConfConstants.DEFAULT_LOCAL_DATABASE_RESOURCE_DIR);
  downloadJarFilesFromHDFS(resTopDir, localResTopDir);
  resTopDir = localResTopDir;
}


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153851
---


On Oct. 25, 2016, 10:11 a.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Oct. 25, 2016, 10:11 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  3ae59c6 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml 8e5fe8a 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 4928c1f 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153851
---




lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
(line 135)


Make the argument File instead of String.



lens-client/src/main/java/org/apache/lens/client/LensConnection.java (line 372)


Please add unit test for the client api/cli command



lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
(line 841)


local directory need not be qualified, as it will be always local.



lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 (line 23)


required import?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 90)


Can we store resourceFs which is a FileSystem object as class variable, 
instead of creating newInstance on every download ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 109)


Lets move super.start() towards end of this method.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 117 - 130)


Does server start has to download/refresh jars upon every startup ? It 
would be expensive. and we are doing even if there are no changes to the files.

Should we do size checks across local and remote filesystem before any 
copying? 

Put this enabled/disabled with a configuration?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 123 - 129)


This check has to happen before the above copy.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 169)


not sure how its corresponding local jar file is added to classloader. If 
hdfs jar is getting added, it wont work.

might have to pass the corresponding local jar as the location value for 
ResourceEntry.



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 213)


Can we store this at class level variable instead of reading from conf 
again and again ?



lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (lines 547 - 567)


We should not be filesystem close anywhere, FileSystem's shutdown hook will 
take care of it.



lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 (line 339)


Why are we passing null name? We should remove the param, if it is not 
required anymore.



lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
(line 310)


what is partial state here? I dont see any return with partial state.



lens-server/src/main/resources/lensserver-default.xml (line 58)


Do we need this change? I see DatabaseResourceService is still getting 
added through SessionService.



lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 (line 174)


Can you change logging style to slf4j parameterized style?


- Amareshwari Sriramadasu


On Oct. 25, 2016, 10:11 a.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Oct. 25, 2016, 10:11 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  3ae59c6 
>   
> 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Oct. 25, 2016, 10:11 a.m.)


Review request for lens.


Changes
---

Addressed review comments.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
3ae59c6 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  
lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
 20ec686 
  lens-server/pom.xml d24dc1e 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 9d823da 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 21e2a62 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
63eea63 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/main/resources/lensserver-default.xml 8e5fe8a 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
 PRE-CREATION 
  lens-server/src/test/resources/lens-site.xml d96659f 
  pom.xml 4928c1f 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

> On Oct. 21, 2016, 1:03 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java,
> >  line 576
> > 
> >
> > Just a thought - Should this be part of SessionService?

Yes moved to SessionService from MetastoreService


> On Oct. 21, 2016, 1:03 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  line 751
> > 
> >
> > We shouldnt be checking for scheme starting with hdfs:// as a string. 
> > We should convert the directory as path and check if it is localfile system 
> > or not.

Didn't find any method to check whether FileSystem is local or not, instead 
used FileSystem.getScheme().


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153535
---


On Oct. 20, 2016, 1:24 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Oct. 20, 2016, 1:24 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   lens-server/pom.xml 6dea9a7 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
>  PRE-CREATION 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   lens-server/src/test/resources/lens-site.xml d96659f 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153535
---




lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
(line 837)


Should we qualify the default path now with file:/// ?



lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 (line 576)


Just a thought - Should this be part of SessionService?



lens-server/pom.xml (line 275)


Can you add the dependency to main pom's dependency management section?  
And version, scope and exclusions should go there.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 741)


We should remove calendar time from logging, as logger would already put 
time.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 745 - 764)


We should move all this code to DBResourceService.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 751)


We shouldnt be checking for scheme starting with hdfs:// as a string. We 
should convert the directory as path and check if it is localfile system or not.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 765)


We should remove calendar time from logging, as logger would already put 
time.


- Amareshwari Sriramadasu


On Oct. 20, 2016, 1:24 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Oct. 20, 2016, 1:24 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   lens-server/pom.xml 6dea9a7 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
>  PRE-CREATION 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   lens-server/src/test/resources/lens-site.xml d96659f 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Oct. 20, 2016, 1:24 p.m.)


Review request for lens.


Changes
---

Addressed review comments.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  lens-server/pom.xml 6dea9a7 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 9d823da 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  lens-server/src/test/resources/lens-site.xml d96659f 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

> On Oct. 18, 2016, 2:09 a.m., Puneet Gupta wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  line 1397
> > 
> >
> > Are two commands required for Hive ?

yes, for health check test case.


> On Oct. 18, 2016, 2:09 a.m., Puneet Gupta wrote:
> > lens-client/src/main/java/org/apache/lens/client/LensConnection.java, lines 
> > 395-398
> > 
> >
> > How do we handle non jar type resources both on client and on server ?

We are supporting only jar files as of now. In server side we have check for 
file type.


> On Oct. 18, 2016, 2:09 a.m., Puneet Gupta wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  lines 762-764
> > 
> >
> > How is addJar updating 
> > org.apache.lens.server.session.DatabaseResourceService#classLoaderCache  
> > with calssloader pointing to the newly uploaded jar? Will the new jar be 
> > refletced in exsitin session calss loader as well ?

Added method addLocalDbResourceEntry() which will be taken care in 
loadMappedResources().


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review152818
---


On Oct. 14, 2016, 8:29 a.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Oct. 14, 2016, 8:29 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   lens-server/pom.xml 6dea9a7 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
>  PRE-CREATION 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   lens-server/src/test/resources/lens-site.xml d96659f 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review152818
---




lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
(line 134)


How are we handling HDFS path ? are we downloading this jar to local 
machine (hosting the cli) and then uploading to server ?



lens-client/src/main/java/org/apache/lens/client/LensConnection.java (lines 393 
- 396)


How do we handle non jar type resources both on client and on server ?



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 738)


Should we have logs statements at the start and end of this method as its 
syncronized so that we know why the other addjar call is waiting (if that 
situation ever arises)



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 760)


Do we need to update 
org.apache.lens.server.session.DatabaseResourceService#dbResEntryMap with entry 
in this case?



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 762 - 764)


How is addJar updating 
org.apache.lens.server.session.DatabaseResourceService#classLoaderCache  with 
calssloader pointing to the newly uploaded jar? Will the new jar be refletced 
in exsitin session calss loader as well ?



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 792 - 795)


Should we fail in this case or create the directory and add the jar? I 
feel, the use case where a user creates a new DB from lens CLI and then uploads 
a jar should work without admin intervention.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 801)


Should we show a different message to user which says 
"This database {dbname} does not support jar upload"

The warning message can still have details that jar_oder file is present 
and hence upload is not allowed



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 808 - 812)


Theoratically this case should not arive since the method is syncronized. 
If this does occur, it can be because server was resttarted while a jar was 
still uploading or rename opertaion failed. In this case(s) we should overwrite 
the jar with a warning message. 
Please check once.



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 815 - 827)


should we move this logic to a utility and use it for both 
DatabaseResourceService and CubeMetastoreServiceImpl ?



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 829)


This seems same as "uploadingPath"



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 834)


Can we resuse "dbDir" in all paces where DB path is required



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (lines 840 - 845)


Should we club the two exceptions (catch with multiple exceptions) and just 
say "Execption while uploading jar". The stack trace will have the exception 
type and details anyway. 
Or we can even have smaller try blocks and print the exact execption 
message (Say a try block around IOUtils.copy(is, fos) and the catch says 
execption wile copying jar)



lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 (line 1397)


Are two commands required for Hive ?



lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 (lines 1566 - 1567)


fileDetail needs to me added too ?



lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 (line 1576)


can only JARs be added as resources ? If yes then should we rename the 
method to include jar



lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 (line 1581)


This file size is passed 

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Oct. 14, 2016, 8:29 a.m.)


Review request for lens.


Changes
---

- Added support for HDFS as DB resource path.
- Addressed other review comments.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  lens-server/pom.xml 6dea9a7 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 9d823da 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  lens-server/src/test/resources/lens-site.xml d96659f 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review149956
---




lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 (line 83)


We need to have support for hdfs path as well, otherwise resources wont be 
accessible to hive server if it is running on a different machine.


- Amareshwari Sriramadasu


On Sept. 16, 2016, 6:52 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Sept. 16, 2016, 6:52 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> 05108f0 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java 593cc08 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f1983e 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
>  PRE-CREATION 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   lens-server/src/test/resources/lens-site.xml 7d72928 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens UI ... SUCCESS [19.454s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49:44.002s
> [INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
> [INFO] Final Memory: 178M/2490M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Sushil Mohanty
> 
>

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review149953
---




lens-client/src/main/java/org/apache/lens/client/LensConnection.java (line 378)


we can change log to log.info. And change loggign to parameterized style.



lens-client/src/main/java/org/apache/lens/client/LensConnection.java (line 381)


Why are we adding jar extension here? Shouldnt we take it from resource 
path ?



lens-client/src/main/java/org/apache/lens/client/LensConnection.java (line 382)


Should we add size check here as well ? We can ignore this comment if size 
checks on server are sufficient.


- Amareshwari Sriramadasu


On Sept. 16, 2016, 6:52 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Sept. 16, 2016, 6:52 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> 05108f0 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java 593cc08 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f1983e 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
>  9d823da 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
>  PRE-CREATION 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   lens-server/src/test/resources/lens-site.xml 7d72928 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens UI ... SUCCESS [19.454s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 49:44.002s
> [INFO] Finished at: Sat Sep 17 00:12:08 IST 2016

Review Request 51964: LENS-317: Server side calls for updating database jars

[Sushil Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

Review request for lens.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
05108f0 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 593cc08 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8f1983e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/metastore/MetastoreResource.java
 9d823da 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/metastore/TestDatabaseService.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  lens-server/src/test/resources/lens-site.xml 7d72928 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty