[jira] [Updated] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Sundarrajan updated LENS-831:
--
Fix Version/s: 2.4
Affects Version/s: 2.4
   Status: Patch Available  (was: Open)

> Create a single jar for shipping with hive queries
> --
>
> Key: LENS-831
> URL: https://issues.apache.org/jira/browse/LENS-831
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 2.4
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>  Labels: packaging
> Fix For: 2.4
>
> Attachments: LENS-831.patch
>
>
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 39430: Create a single jar for shipping with hive queries

[Srikanth Sundarrajan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39430/
---

(Updated Oct. 19, 2015, 5:28 a.m.)


Review request for lens.


Changes
---

Added net.sf.opencsv:opencsv as well to the ship-jars


Bugs: LENS-831
https://issues.apache.org/jira/browse/LENS-831


Repository: lens


Description
---

Currently a number of jars have to be shipped along with each query apart from 
user extension jars for udf, serde etc. This has two side effects:

1. Number of ClassLoader on the hive server increases (potentially causing 
PermGen OOM issues)
2. Number of jar localization events in the node manager when executing a task.

The proposal is to bundle these and create a single shaded jar and make it 
available in the binary distribution.


Diffs (updated)
-

  lens-dist/pom.xml bc94209 
  lens-dist/src/main/assembly/bin-dist.xml 8707af9 
  lens-ship-jars/LICENSE PRE-CREATION 
  lens-ship-jars/pom.xml PRE-CREATION 
  lens-ship-jars/src/site/apt/index.apt PRE-CREATION 
  pom.xml 7140038 
  src/site/apt/lenshome/install-and-run.apt 491bbee 

Diff: https://reviews.apache.org/r/39430/diff/


Testing
---

Verified that the shippable jar is created correctly and the same is being 
added to the binary distribution package


Thanks,

Srikanth Sundarrajan

[jira] [Updated] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Sundarrajan updated LENS-831:
--
Labels: packaging  (was: )

> Create a single jar for shipping with hive queries
> --
>
> Key: LENS-831
> URL: https://issues.apache.org/jira/browse/LENS-831
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>  Labels: packaging
>
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 39430: Create a single jar for shipping with hive queries

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39430/#review103096
---



lens-ship-jars/pom.xml (line 70)


Can we include open csv aswell, which is right now coming as transitive 
dependency from lens-query-lib?


- Amareshwari Sriramadasu


On Oct. 19, 2015, 4:46 a.m., Srikanth Sundarrajan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39430/
> ---
> 
> (Updated Oct. 19, 2015, 4:46 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-831
> https://issues.apache.org/jira/browse/LENS-831
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> 
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.
> 
> 
> Diffs
> -
> 
>   lens-dist/pom.xml bc94209 
>   lens-dist/src/main/assembly/bin-dist.xml 8707af9 
>   lens-ship-jars/LICENSE PRE-CREATION 
>   lens-ship-jars/pom.xml PRE-CREATION 
>   lens-ship-jars/src/site/apt/index.apt PRE-CREATION 
>   pom.xml 7140038 
>   src/site/apt/lenshome/install-and-run.apt 491bbee 
> 
> Diff: https://reviews.apache.org/r/39430/diff/
> 
> 
> Testing
> ---
> 
> Verified that the shippable jar is created correctly and the same is being 
> added to the binary distribution package
> 
> 
> Thanks,
> 
> Srikanth Sundarrajan
> 
>

[jira] [Created] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

Srikanth Sundarrajan created LENS-831:
-

 Summary: Create a single jar for shipping with hive queries
 Key: LENS-831
 URL: https://issues.apache.org/jira/browse/LENS-831
 Project: Apache Lens
  Issue Type: Improvement
  Components: build
Reporter: Srikanth Sundarrajan
Assignee: Srikanth Sundarrajan


Currently a number of jars have to be shipped along with each query apart from 
user extension jars for udf, serde etc. This has two side effects:

1. Number of ClassLoader on the hive server increases (potentially causing 
PermGen OOM issues)
2. Number of jar localization events in the node manager when executing a task.

The proposal is to bundle these and create a single shaded jar and make it 
available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Sundarrajan updated LENS-831:
--
Attachment: LENS-831.patch

> Create a single jar for shipping with hive queries
> --
>
> Key: LENS-831
> URL: https://issues.apache.org/jira/browse/LENS-831
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>  Labels: packaging
> Attachments: LENS-831.patch
>
>
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Sundarrajan updated LENS-831:
--
Attachment: LENS-831-V1.patch

Updated patch to include net.sf.opencsv:opencsv

> Create a single jar for shipping with hive queries
> --
>
> Key: LENS-831
> URL: https://issues.apache.org/jira/browse/LENS-831
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 2.4
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>  Labels: packaging
> Fix For: 2.4
>
> Attachments: LENS-831-V1.patch, LENS-831.patch
>
>
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-831) Create a single jar for shipping with hive queries

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962827#comment-14962827
 ] 

Srikanth Sundarrajan commented on LENS-831:
---

Added the following jars to the ship-jar (excluding transitive dependencies)
* hive-metastore
* lens-storage-db
* lens-query-lib
* lens-cube

> Create a single jar for shipping with hive queries
> --
>
> Key: LENS-831
> URL: https://issues.apache.org/jira/browse/LENS-831
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Reporter: Srikanth Sundarrajan
>Assignee: Srikanth Sundarrajan
>  Labels: packaging
> Attachments: LENS-831.patch
>
>
> Currently a number of jars have to be shipped along with each query apart 
> from user extension jars for udf, serde etc. This has two side effects:
> 1. Number of ClassLoader on the hive server increases (potentially causing 
> PermGen OOM issues)
> 2. Number of jar localization events in the node manager when executing a 
> task.
> The proposal is to bundle these and create a single shaded jar and make it 
> available in the binary distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-815) query list api should have an option to filter by driver

[Raju Bairishetti (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raju Bairishetti updated LENS-815:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed patch to master. Thanks [~deepak.barr]

> query list api should have an option to filter by driver
> 
>
> Key: LENS-815
> URL: https://issues.apache.org/jira/browse/LENS-815
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: api, client, server
>Reporter: Rajat Khandelwal
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-815.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-815) query list api should have an option to filter by driver

[Raju Bairishetti (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raju Bairishetti updated LENS-815:
--
Fix Version/s: 2.5
  Component/s: client

> query list api should have an option to filter by driver
> 
>
> Key: LENS-815
> URL: https://issues.apache.org/jira/browse/LENS-815
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: api, client, server
>Reporter: Rajat Khandelwal
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-815.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-615) Refactoring of testQueryCommands test case

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962740#comment-14962740
 ] 

Hadoop QA commented on LENS-615:


Applied patch: 
[rb39191.1.patch|https://issues.apache.org/jira/secure/attachment/12767273/rb39191.1.patch]
 and ran command: mvn clean install. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/137/

> Refactoring of testQueryCommands test case
> --
>
> Key: LENS-615
> URL: https://issues.apache.org/jira/browse/LENS-615
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: client
>Reporter: Himanshu Gahlaut
>Assignee: Gabor Liptak
>  Labels: newbie
> Attachments: rb39191.1.patch, rb39191.patch
>
>
> There are too many test cases running in testQueryCommands test case. It is 
> considerably time consuming to debug a failing test case within  
> testQueryCommands. If these test cases are modularized and broken into 
> separate test cases, it will be much helpful in faster debugging.
> {code}
> @Test
>   public void testQueryCommands() throws Exception {
> client = new LensClient();
> 
> client.setConnectionParam("lens.query.enable.persistent.resultset.indriver", 
> "false");
> setup(client);
> LensQueryCommands qCom = new LensQueryCommands();
> qCom.setClient(client);
> resDir = new File("target/results");
> assertTrue(resDir.exists() || resDir.mkdirs());
> testExecuteSyncQuery(qCom);
> testExecuteAsyncQuery(qCom);
> testSyncResults(qCom);
> testExplainQuery(qCom);
> testExplainFailQuery(qCom);
> testPreparedQuery(qCom);
> testShowPersistentResultSet(qCom);
> testPurgedFinishedResultSet(qCom);
> testFailPreparedQuery(qCom);
> // run all query commands with query metrics enabled.
> client = new LensClient();
> 
> client.setConnectionParam("lens.query.enable.persistent.resultset.indriver", 
> "false");
> client.setConnectionParam("lens.query.enable.metrics.per.query", "true");
> qCom.setClient(client);
> String result = qCom.getAllPreparedQueries("all", "", -1, -1);
> assertEquals(result, "No prepared queries");
> testExecuteSyncQuery(qCom);
> testExecuteAsyncQuery(qCom);
> testSyncResults(qCom);
> testExplainQuery(qCom);
> testExplainFailQuery(qCom);
> testPreparedQuery(qCom);
> testShowPersistentResultSet(qCom);
> testPurgedFinishedResultSet(qCom);
> testFailPreparedQuery(qCom);
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)