date:20150930

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

2015-09-30 Thread Deepak Barr



> On Sept. 30, 2015, 1:37 a.m., Raju Bairishetti wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  line 1203
> > 
> >
> > Why are we changing exceptions here?
> > 
> > You may need to InvalidOperationExcpetion if you keep the older method 
> > declaration (i.e. throws MetaException, UnknownDBException, 
> > HiveSQLException, TException, LensException).
> > 
> > 
> > MetaException, UnknownDBException and InvalidOperationException  all 
> > extends TException.

Raju,

   I didnt quite get you. Since TException extends MetaException, 
UnknownDBException, I felt we can instead throw a general TException. I dint 
get your point about InvalidOperationExcpetion though.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38857/#review101061
---


On Sept. 29, 2015, 6:59 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38857/
> ---
> 
> (Updated Sept. 29, 2015, 6:59 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Currently, To obtain list of native tables, CubeMetastoreService does the 
> following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by 
> looking at the table properties from the Table object. This table object is 
> obtained with another Metastore call. So, If there are 'n' tables, there will 
> be 'n' metastore calls.
> 
> Instead of this, we can directly fetch the list of table objects for our list 
> of table names in a single API call using 
> getMetaStoreClient().getTableObjectsByName() method.
> 
> 
> Diffs
> -
> 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  37cebfe 
> 
> Diff: https://reviews.apache.org/r/38857/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.745 
> s]
> [INFO] Lens ... SUCCESS [  4.601 
> s]
> [INFO] Lens API ... SUCCESS [ 31.397 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 28.560 
> s]
> [INFO] Lens Cube .. SUCCESS [07:04 
> min]
> [INFO] Lens DB storage  SUCCESS [ 27.465 
> s]
> [INFO] Lens Query Library . SUCCESS [ 23.361 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [04:14 
> min]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 50.792 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.551 
> s]
> [INFO] Lens Server  SUCCESS [11:11 
> min]
> [INFO] Lens client  SUCCESS [ 48.547 
> s]
> [INFO] Lens CLI ... SUCCESS [03:45 
> min]
> [INFO] Lens Examples .. SUCCESS [ 11.595 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 13.176 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:03 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  3.752 
> s]
> [INFO] Lens Regression  SUCCESS [ 15.811 
> s]
> [INFO] Lens UI  SUCCESS [ 50.811 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 33:53 min
> [INFO] Finished at: 2015-09-29T22:48:22+05:30
> [INFO] Final Memory: 175M/3158M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

2015-09-30 Thread Raju Bairishetti



> On Sept. 30, 2015, 1:37 a.m., Raju Bairishetti wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  line 1203
> > 
> >
> > Why are we changing exceptions here?
> > 
> > You may need to InvalidOperationExcpetion if you keep the older method 
> > declaration (i.e. throws MetaException, UnknownDBException, 
> > HiveSQLException, TException, LensException).
> > 
> > 
> > MetaException, UnknownDBException and InvalidOperationException  all 
> > extends TException.
> 
> Deepak Barr wrote:
> Raju,
> 
>I didnt quite get you. Since TException extends MetaException, 
> UnknownDBException, I felt we can instead throw a general TException. I dint 
> get your point about InvalidOperationExcpetion though.

I would say keep the existing exceptions(MetaException, UnknownException). 
getTableObjectsByName(...) is throwing InvalidOperationException also. We may 
need to throw InvalidOperationException.

It would be easier to identify the problem from the actual exception instead of 
TException.

++ Rajat


- Raju


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38857/#review101061
---


On Sept. 29, 2015, 6:59 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38857/
> ---
> 
> (Updated Sept. 29, 2015, 6:59 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Currently, To obtain list of native tables, CubeMetastoreService does the 
> following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by 
> looking at the table properties from the Table object. This table object is 
> obtained with another Metastore call. So, If there are 'n' tables, there will 
> be 'n' metastore calls.
> 
> Instead of this, we can directly fetch the list of table objects for our list 
> of table names in a single API call using 
> getMetaStoreClient().getTableObjectsByName() method.
> 
> 
> Diffs
> -
> 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  37cebfe 
> 
> Diff: https://reviews.apache.org/r/38857/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.745 
> s]
> [INFO] Lens ... SUCCESS [  4.601 
> s]
> [INFO] Lens API ... SUCCESS [ 31.397 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 28.560 
> s]
> [INFO] Lens Cube .. SUCCESS [07:04 
> min]
> [INFO] Lens DB storage  SUCCESS [ 27.465 
> s]
> [INFO] Lens Query Library . SUCCESS [ 23.361 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [04:14 
> min]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 50.792 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.551 
> s]
> [INFO] Lens Server  SUCCESS [11:11 
> min]
> [INFO] Lens client  SUCCESS [ 48.547 
> s]
> [INFO] Lens CLI ... SUCCESS [03:45 
> min]
> [INFO] Lens Examples .. SUCCESS [ 11.595 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 13.176 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:03 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  3.752 
> s]
> [INFO] Lens Regression  SUCCESS [ 15.811 
> s]
> [INFO] Lens UI  SUCCESS [ 50.811 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 33:53 min
> [INFO] Finished at: 2015-09-29T22:48:22+05:30
> [INFO] Final Memory: 175M/3158M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

2015-09-30 Thread Deepak Barr



> On Sept. 30, 2015, 6:24 a.m., Puneet Gupta wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  lines 1208-1210
> > 
> >
> > Is it possible to use Filter instead to get required tables in one shot.
> > 
> > org.apache.hadoop.hive.metastore.IMetaStoreClient.listTableNamesByFilter(String,
> >  String, short).
> > 
> > Not sure if we can create a specific filter for this case. To create 
> > such a filetr we should have a common key that exists for both native and 
> > non native tables and has a different value for each.

Hey Puneeet,
   I considered that possibility but it seems the filter support is on limited 
keys. See 
https://hive.apache.org/javadocs/r0.12.0/api/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.html#listTableNamesByFilter(java.lang.String,
 java.lang.String, short)


> On Sept. 30, 2015, 6:24 a.m., Puneet Gupta wrote:
> > lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java,
> >  line 1201
> > 
> >
> > While we are at it, can we also change the name of this private method 
> > to indicate that it gets only NATIVE tables ? 
> > Say: getNaviveTablesFromDB

I agree.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38857/#review101080
---


On Sept. 29, 2015, 6:59 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38857/
> ---
> 
> (Updated Sept. 29, 2015, 6:59 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Currently, To obtain list of native tables, CubeMetastoreService does the 
> following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by 
> looking at the table properties from the Table object. This table object is 
> obtained with another Metastore call. So, If there are 'n' tables, there will 
> be 'n' metastore calls.
> 
> Instead of this, we can directly fetch the list of table objects for our list 
> of table names in a single API call using 
> getMetaStoreClient().getTableObjectsByName() method.
> 
> 
> Diffs
> -
> 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  37cebfe 
> 
> Diff: https://reviews.apache.org/r/38857/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.745 
> s]
> [INFO] Lens ... SUCCESS [  4.601 
> s]
> [INFO] Lens API ... SUCCESS [ 31.397 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 28.560 
> s]
> [INFO] Lens Cube .. SUCCESS [07:04 
> min]
> [INFO] Lens DB storage  SUCCESS [ 27.465 
> s]
> [INFO] Lens Query Library . SUCCESS [ 23.361 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [04:14 
> min]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 50.792 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.551 
> s]
> [INFO] Lens Server  SUCCESS [11:11 
> min]
> [INFO] Lens client  SUCCESS [ 48.547 
> s]
> [INFO] Lens CLI ... SUCCESS [03:45 
> min]
> [INFO] Lens Examples .. SUCCESS [ 11.595 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 13.176 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:03 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  3.752 
> s]
> [INFO] Lens Regression  SUCCESS [ 15.811 
> s]
> [INFO] Lens UI  SUCCESS [ 50.811 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 33:53 min
> [INFO] Finished at: 2015-09-29T22:48:22+05:30
> [INFO] Final Memory: 175M/3158M
> [INFO] 
> 
> 
> 
> Thanks,
> 
>

Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/
---

Review request for lens.


Bugs: LENS-795
https://issues.apache.org/jira/browse/LENS-795


Repository: lens


Description
---

Changes:

1. Missing partition should not matter if the partition is missing on a part 
column not required.
1.1. Look ahead part column's missing partitions are okay
1.2. Fallback partition columns' missing partitions are to be considered as 
missing


Diffs
-

  
lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
 b389a8f3a5257833834f389163b27c7040aedf69 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
68ab5ab8a7441635f7acbe61e4ab9192056118cc 

Diff: https://reviews.apache.org/r/38889/diff/


Testing
---


Thanks,

Rajat Khandelwal

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

2015-09-30 Thread Deepak Barr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38857/
---

(Updated Sept. 30, 2015, 11:30 a.m.)


Review request for lens.


Changes
---

Updated diff - 

1. Reused Metastore client
2. Changed method name
3. Reverted back the exceptions thrown and handled invalidoperationexception.


Repository: lens


Description
---

Currently, To obtain list of native tables, CubeMetastoreService does the 
following -
1. Fetches the list of tables ( one MetastoreClient call)
2. Filters out the cube tables from the list. The filtering happens by looking 
at the table properties from the Table object. This table object is obtained 
with another Metastore call. So, If there are 'n' tables, there will be 'n' 
metastore calls.

Instead of this, we can directly fetch the list of table objects for our list 
of table names in a single API call using 
getMetaStoreClient().getTableObjectsByName() method.


Diffs (updated)
-

  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 37cebfe 

Diff: https://reviews.apache.org/r/38857/diff/


Testing
---

Yes.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules .. SUCCESS [  2.745 s]
[INFO] Lens ... SUCCESS [  4.601 s]
[INFO] Lens API ... SUCCESS [ 31.397 s]
[INFO] Lens API for server and extensions . SUCCESS [ 28.560 s]
[INFO] Lens Cube .. SUCCESS [07:04 min]
[INFO] Lens DB storage  SUCCESS [ 27.465 s]
[INFO] Lens Query Library . SUCCESS [ 23.361 s]
[INFO] Lens Hive Driver ... SUCCESS [04:14 min]
[INFO] Lens Driver for JDBC ... SUCCESS [ 50.792 s]
[INFO] Lens Elastic Search Driver . SUCCESS [ 20.551 s]
[INFO] Lens Server  SUCCESS [11:11 min]
[INFO] Lens client  SUCCESS [ 48.547 s]
[INFO] Lens CLI ... SUCCESS [03:45 min]
[INFO] Lens Examples .. SUCCESS [ 11.595 s]
[INFO] Lens Distribution .. SUCCESS [ 13.176 s]
[INFO] Lens ML Lib  SUCCESS [02:03 min]
[INFO] Lens ML Ext Distribution ... SUCCESS [  3.752 s]
[INFO] Lens Regression  SUCCESS [ 15.811 s]
[INFO] Lens UI  SUCCESS [ 50.811 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 33:53 min
[INFO] Finished at: 2015-09-29T22:48:22+05:30
[INFO] Final Memory: 175M/3158M
[INFO] 


Thanks,

Deepak Barr

[jira] [Work started] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-795 started by Rajat Khandelwal.
-
> Look ahead is failing with partition not found if any process time partitions 
> are absent
> 
>
> Key: LENS-795
> URL: https://issues.apache.org/jira/browse/LENS-795
> Project: Apache Lens
>  Issue Type: Bug
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Sushil Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/#review101108
---

Ship it!


Ship It!

- Sushil Mohanty


On Sept. 30, 2015, 12:14 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38889/
> ---
> 
> (Updated Sept. 30, 2015, 12:14 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-795
> https://issues.apache.org/jira/browse/LENS-795
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes:
> 
> 1. Missing partition should not matter if the partition is missing on a part 
> column not required.
> 1.1. Look ahead part column's missing partitions are okay
> 1.2. Fallback partition columns' missing partitions are to be considered 
> as missing
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
>  b389a8f3a5257833834f389163b27c7040aedf69 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> 68ab5ab8a7441635f7acbe61e4ab9192056118cc 
> 
> Diff: https://reviews.apache.org/r/38889/diff/
> 
> 
> Testing
> ---
> 
> ---
>  T E S T S
> ---
> Running TestSuite
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 508.822 sec 
> <<< FAILURE! - in TestSuite
> trainAndEvalFromDir(org.apache.lens.ml.TestMLRunner)  Time elapsed: 2.21 sec  
> <<< FAILURE!
> java.lang.AssertionError: expected [SUCCESSFUL] but found [FAILED]
>   at org.testng.Assert.fail(Assert.java:94)
>   at org.testng.Assert.failNotEquals(Assert.java:494)
>   at org.testng.Assert.assertEquals(Assert.java:123)
>   at org.testng.Assert.assertEquals(Assert.java:165)
>   at 
> org.apache.lens.ml.TestMLRunner.trainAndEvalFromDir(TestMLRunner.java:132)
> 
> 
> Results :
> 
> Failed tests: 
>   TestMLRunner.trainAndEvalFromDir:132 expected [SUCCESSFUL] but found 
> [FAILED]
> 
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules . SUCCESS [1:08.377s]
> [INFO] Lens .. SUCCESS [38.321s]
> [INFO] Lens API .. SUCCESS [1:54.490s]
> [INFO] Lens API for server and extensions  SUCCESS [1:44.559s]
> [INFO] Lens Cube . SUCCESS 
> [26:20.812s]
> [INFO] Lens DB storage ... SUCCESS [2:37.722s]
> [INFO] Lens Query Library  SUCCESS [49.289s]
> [INFO] Lens Hive Driver .. SUCCESS [9:33.923s]
> [INFO] Lens Driver for JDBC .. SUCCESS [4:48.278s]
> [INFO] Lens Elastic Search Driver  SUCCESS [1:00.712s]
> [INFO] Lens Server ... SUCCESS 
> [43:09.453s]
> [INFO] Lens client ... SUCCESS [3:28.307s]
> [INFO] Lens CLI .. SUCCESS 
> [12:21.291s]
> [INFO] Lens Examples . SUCCESS [29.909s]
> [INFO] Lens Distribution . SUCCESS [1:04.494s]
> [INFO] Lens ML Lib ... FAILURE [9:25.823s]
> [INFO] Lens ML Ext Distribution .. SKIPPED
> [INFO] Lens Regression ... SKIPPED
> [INFO] Lens UI ... SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 2:01:17.504s
> [INFO] Finished at: Wed Sep 30 11:56:50 UTC 2015
> [INFO] Final Memory: 136M/347M
> [INFO] 
> 
> 
> 
> The failure is unrelated and seems intermittent. When run separately, the 
> test case passed. Running again though.
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/
---

(Updated Sept. 30, 2015, 6:26 p.m.)


Review request for lens.


Bugs: LENS-795
https://issues.apache.org/jira/browse/LENS-795


Repository: lens


Description
---

Changes:

1. Missing partition should not matter if the partition is missing on a part 
column not required.
1.1. Look ahead part column's missing partitions are okay
1.2. Fallback partition columns' missing partitions are to be considered as 
missing


Diffs (updated)
-

  
lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
 b389a8f3a5257833834f389163b27c7040aedf69 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
68ab5ab8a7441635f7acbe61e4ab9192056118cc 

Diff: https://reviews.apache.org/r/38889/diff/


Testing
---

---
 T E S T S
---
Running TestSuite
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 508.822 sec <<< 
FAILURE! - in TestSuite
trainAndEvalFromDir(org.apache.lens.ml.TestMLRunner)  Time elapsed: 2.21 sec  
<<< FAILURE!
java.lang.AssertionError: expected [SUCCESSFUL] but found [FAILED]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:494)
at org.testng.Assert.assertEquals(Assert.java:123)
at org.testng.Assert.assertEquals(Assert.java:165)
at 
org.apache.lens.ml.TestMLRunner.trainAndEvalFromDir(TestMLRunner.java:132)


Results :

Failed tests: 
  TestMLRunner.trainAndEvalFromDir:132 expected [SUCCESSFUL] but found [FAILED]

Tests run: 6, Failures: 1, Errors: 0, Skipped: 0

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [1:08.377s]
[INFO] Lens .. SUCCESS [38.321s]
[INFO] Lens API .. SUCCESS [1:54.490s]
[INFO] Lens API for server and extensions  SUCCESS [1:44.559s]
[INFO] Lens Cube . SUCCESS [26:20.812s]
[INFO] Lens DB storage ... SUCCESS [2:37.722s]
[INFO] Lens Query Library  SUCCESS [49.289s]
[INFO] Lens Hive Driver .. SUCCESS [9:33.923s]
[INFO] Lens Driver for JDBC .. SUCCESS [4:48.278s]
[INFO] Lens Elastic Search Driver  SUCCESS [1:00.712s]
[INFO] Lens Server ... SUCCESS [43:09.453s]
[INFO] Lens client ... SUCCESS [3:28.307s]
[INFO] Lens CLI .. SUCCESS [12:21.291s]
[INFO] Lens Examples . SUCCESS [29.909s]
[INFO] Lens Distribution . SUCCESS [1:04.494s]
[INFO] Lens ML Lib ... FAILURE [9:25.823s]
[INFO] Lens ML Ext Distribution .. SKIPPED
[INFO] Lens Regression ... SKIPPED
[INFO] Lens UI ... SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 2:01:17.504s
[INFO] Finished at: Wed Sep 30 11:56:50 UTC 2015
[INFO] Final Memory: 136M/347M
[INFO] 


The failure is unrelated and seems intermittent. When run separately, the test 
case passed. Running again though.


Thanks,

Rajat Khandelwal

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Raju Bairishetti


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/#review101107
---



lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
(line 696)


can we make it private instead of package private?

Patch looks good to me otherwise.


- Raju Bairishetti


On Sept. 30, 2015, 9:50 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38889/
> ---
> 
> (Updated Sept. 30, 2015, 9:50 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-795
> https://issues.apache.org/jira/browse/LENS-795
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes:
> 
> 1. Missing partition should not matter if the partition is missing on a part 
> column not required.
> 1.1. Look ahead part column's missing partitions are okay
> 1.2. Fallback partition columns' missing partitions are to be considered 
> as missing
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
>  b389a8f3a5257833834f389163b27c7040aedf69 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> 68ab5ab8a7441635f7acbe61e4ab9192056118cc 
> 
> Diff: https://reviews.apache.org/r/38889/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/
---

(Updated Sept. 30, 2015, 5:44 p.m.)


Review request for lens.


Bugs: LENS-795
https://issues.apache.org/jira/browse/LENS-795


Repository: lens


Description
---

Changes:

1. Missing partition should not matter if the partition is missing on a part 
column not required.
1.1. Look ahead part column's missing partitions are okay
1.2. Fallback partition columns' missing partitions are to be considered as 
missing


Diffs
-

  
lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
 b389a8f3a5257833834f389163b27c7040aedf69 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
68ab5ab8a7441635f7acbe61e4ab9192056118cc 

Diff: https://reviews.apache.org/r/38889/diff/


Testing (updated)
---

---
 T E S T S
---
Running TestSuite
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 508.822 sec <<< 
FAILURE! - in TestSuite
trainAndEvalFromDir(org.apache.lens.ml.TestMLRunner)  Time elapsed: 2.21 sec  
<<< FAILURE!
java.lang.AssertionError: expected [SUCCESSFUL] but found [FAILED]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:494)
at org.testng.Assert.assertEquals(Assert.java:123)
at org.testng.Assert.assertEquals(Assert.java:165)
at 
org.apache.lens.ml.TestMLRunner.trainAndEvalFromDir(TestMLRunner.java:132)


Results :

Failed tests: 
  TestMLRunner.trainAndEvalFromDir:132 expected [SUCCESSFUL] but found [FAILED]

Tests run: 6, Failures: 1, Errors: 0, Skipped: 0

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [1:08.377s]
[INFO] Lens .. SUCCESS [38.321s]
[INFO] Lens API .. SUCCESS [1:54.490s]
[INFO] Lens API for server and extensions  SUCCESS [1:44.559s]
[INFO] Lens Cube . SUCCESS [26:20.812s]
[INFO] Lens DB storage ... SUCCESS [2:37.722s]
[INFO] Lens Query Library  SUCCESS [49.289s]
[INFO] Lens Hive Driver .. SUCCESS [9:33.923s]
[INFO] Lens Driver for JDBC .. SUCCESS [4:48.278s]
[INFO] Lens Elastic Search Driver  SUCCESS [1:00.712s]
[INFO] Lens Server ... SUCCESS [43:09.453s]
[INFO] Lens client ... SUCCESS [3:28.307s]
[INFO] Lens CLI .. SUCCESS [12:21.291s]
[INFO] Lens Examples . SUCCESS [29.909s]
[INFO] Lens Distribution . SUCCESS [1:04.494s]
[INFO] Lens ML Lib ... FAILURE [9:25.823s]
[INFO] Lens ML Ext Distribution .. SKIPPED
[INFO] Lens Regression ... SKIPPED
[INFO] Lens UI ... SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 2:01:17.504s
[INFO] Finished at: Wed Sep 30 11:56:50 UTC 2015
[INFO] Final Memory: 136M/347M
[INFO] 


The failure is unrelated and seems intermittent. When run separately, the test 
case passed. Running again though.


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936773#comment-14936773
 ] 

Rajat Khandelwal commented on LENS-795:
---

Created https://reviews.apache.org/r/38892/

> Look ahead is failing with partition not found if any process time partitions 
> are absent
> 
>
> Key: LENS-795
> URL: https://issues.apache.org/jira/browse/LENS-795
> Project: Apache Lens
>  Issue Type: Bug
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Rajat Khandelwal (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936773#comment-14936773
 ] 

Rajat Khandelwal edited comment on LENS-795 at 9/30/15 12:54 PM:
-

Created https://reviews.apache.org/r/38889/


was (Author: prongs):
Created https://reviews.apache.org/r/38892/

> Look ahead is failing with partition not found if any process time partitions 
> are absent
> 
>
> Key: LENS-795
> URL: https://issues.apache.org/jira/browse/LENS-795
> Project: Apache Lens
>  Issue Type: Bug
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

2015-09-30 Thread Raju Bairishetti


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38889/#review101109
---

Ship it!


Test failed in ML module. Tracking already at 
[LENS-660|https://issues.apache.org/jira/browse/LENS-660].

- Raju Bairishetti


On Sept. 30, 2015, 12:14 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38889/
> ---
> 
> (Updated Sept. 30, 2015, 12:14 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-795
> https://issues.apache.org/jira/browse/LENS-795
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes:
> 
> 1. Missing partition should not matter if the partition is missing on a part 
> column not required.
> 1.1. Look ahead part column's missing partitions are okay
> 1.2. Fallback partition columns' missing partitions are to be considered 
> as missing
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/PartitionRangesForPartitionColumns.java
>  b389a8f3a5257833834f389163b27c7040aedf69 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> 68ab5ab8a7441635f7acbe61e4ab9192056118cc 
> 
> Diff: https://reviews.apache.org/r/38889/diff/
> 
> 
> Testing
> ---
> 
> ---
>  T E S T S
> ---
> Running TestSuite
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 508.822 sec 
> <<< FAILURE! - in TestSuite
> trainAndEvalFromDir(org.apache.lens.ml.TestMLRunner)  Time elapsed: 2.21 sec  
> <<< FAILURE!
> java.lang.AssertionError: expected [SUCCESSFUL] but found [FAILED]
>   at org.testng.Assert.fail(Assert.java:94)
>   at org.testng.Assert.failNotEquals(Assert.java:494)
>   at org.testng.Assert.assertEquals(Assert.java:123)
>   at org.testng.Assert.assertEquals(Assert.java:165)
>   at 
> org.apache.lens.ml.TestMLRunner.trainAndEvalFromDir(TestMLRunner.java:132)
> 
> 
> Results :
> 
> Failed tests: 
>   TestMLRunner.trainAndEvalFromDir:132 expected [SUCCESSFUL] but found 
> [FAILED]
> 
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules . SUCCESS [1:08.377s]
> [INFO] Lens .. SUCCESS [38.321s]
> [INFO] Lens API .. SUCCESS [1:54.490s]
> [INFO] Lens API for server and extensions  SUCCESS [1:44.559s]
> [INFO] Lens Cube . SUCCESS 
> [26:20.812s]
> [INFO] Lens DB storage ... SUCCESS [2:37.722s]
> [INFO] Lens Query Library  SUCCESS [49.289s]
> [INFO] Lens Hive Driver .. SUCCESS [9:33.923s]
> [INFO] Lens Driver for JDBC .. SUCCESS [4:48.278s]
> [INFO] Lens Elastic Search Driver  SUCCESS [1:00.712s]
> [INFO] Lens Server ... SUCCESS 
> [43:09.453s]
> [INFO] Lens client ... SUCCESS [3:28.307s]
> [INFO] Lens CLI .. SUCCESS 
> [12:21.291s]
> [INFO] Lens Examples . SUCCESS [29.909s]
> [INFO] Lens Distribution . SUCCESS [1:04.494s]
> [INFO] Lens ML Lib ... FAILURE [9:25.823s]
> [INFO] Lens ML Ext Distribution .. SKIPPED
> [INFO] Lens Regression ... SKIPPED
> [INFO] Lens UI ... SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 2:01:17.504s
> [INFO] Finished at: Wed Sep 30 11:56:50 UTC 2015
> [INFO] Final Memory: 136M/347M
> [INFO] 
> 
> 
> 
> The failure is unrelated and seems intermittent. When run separately, the 
> test case passed. Running again though.
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38857: LENS-796 : Slow response times for /metastore/nativetables API

[jira] [Work started] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

[jira] [Commented] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

[jira] [Comment Edited] (LENS-795) Look ahead is failing with partition not found if any process time partitions are absent

Re: Review Request 38889: LENS-795: Look ahead is failing with partition not found if any process time partitions are absent

13 matches

Site Navigation

Mail list logo

Footer information