Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-14 Thread Rajat Khandelwal


> On Oct. 14, 2016, 3:31 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java,
> >  line 916
> > 
> >
> > On what basis server retry policy is decided? Is it one at server level 
> > for all queries ?

`Server retry policy "decider"` is one at server level for all queries. Upon 
query failure and driver refusing to retry, this decider is consulted to give a 
`Server Retry Policy`. At that point, it's purely a function of the error 
message.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152645
---


On Oct. 13, 2016, 11:57 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 13, 2016, 11:57 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> e41077cffd1fed891e12dbefe7650abed8d73bb7 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  365a619619dcde3ef847c2955b6e818515f870a2 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-14 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152645
---


Fix it, then Ship it!





lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 (lines 212 - 215)






lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
 (line 911)


On what basis server retry policy is decided? Is it one at server level for 
all queries ?


- Amareshwari Sriramadasu


On Oct. 13, 2016, 6:27 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 13, 2016, 6:27 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> e41077cffd1fed891e12dbefe7650abed8d73bb7 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  365a619619dcde3ef847c2955b6e818515f870a2 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-13 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 13, 2016, 11:57 a.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
e41077cffd1fed891e12dbefe7650abed8d73bb7 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 365a619619dcde3ef847c2955b6e818515f870a2 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-12 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 12, 2016, 6:31 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
e41077cffd1fed891e12dbefe7650abed8d73bb7 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 365a619619dcde3ef847c2955b6e818515f870a2 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-12 Thread Rajat Khandelwal


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, line 113
> > 
> >
> > can you update state machine on user doc with the change?
> 
> Rajat Khandelwal wrote:
> Would need to generate the png file again.

Will update seperately. Adding here and submitting patch results in the patch 
not getting applied because some binary diff is present. Created LENS-1356 for 
the same.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
---


On Oct. 12, 2016, 5:57 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 12, 2016, 5:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> e41077cffd1fed891e12dbefe7650abed8d73bb7 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  365a619619dcde3ef847c2955b6e818515f870a2 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-12 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 12, 2016, 5:57 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
e41077cffd1fed891e12dbefe7650abed8d73bb7 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
2ad7f76c9c25469a2e39122ed96d1afcb837a6a0 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 365a619619dcde3ef847c2955b6e818515f870a2 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 fc24fc6364f8e9770ddfe4029ec6bede46df64f7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-12 Thread Rajat Khandelwal


> On Oct. 12, 2016, 4:51 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java,
> >  line 55
> > 
> >
> > test disabled?

Commented to run the test in isolation. Forgot to uncomment. Will do.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152294
---


On Oct. 10, 2016, 12:14 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 10, 2016, 12:14 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  e4984798751d012f0a9d01980df833c206155546 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
>  e160f58bfb45cffb05b9f6be19db3bc7605dde57 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-12 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review152294
---




lens-server/src/test/java/org/apache/lens/server/query/retry/QueryRetryTest.java
 (line 55)


test disabled?


- Amareshwari Sriramadasu


On Oct. 10, 2016, 6:44 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 10, 2016, 6:44 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  e4984798751d012f0a9d01980df833c206155546 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
>  e160f58bfb45cffb05b9f6be19db3bc7605dde57 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
>  5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-10 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 10, 2016, 12:14 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-09 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 10, 2016, 11:14 a.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-09 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 9, 2016, 11:42 a.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-07 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 7, 2016, 11:41 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-07 Thread Rajat Khandelwal


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java,
> >  lines 76-77
> > 
> >
> > This is enforcing drivertype in the file names. This will be a problem 
> > with -default.xml files, if people create driver type to be different than 
> > whats expected to be.
> > 
> > Instead you can try using driver.getClass().getName().toLowercase() ?
> 
> Rajat Khandelwal wrote:
> I'm fine with either, but unable to see the merit in using one above the 
> other.

I changed it to using class name. But there is a slight inconsistency. 

Now, if drivers are configured like 
"mockHive:org.apache.lens.driver.hive.HiveDriver", then the `configure` method 
looks for `hivedriver-site.xml` instead of `mockhivedriver-site.xml`, which is 
fine. 

But the code that I added for unification of properties does the other thing. 
e.g. for `retry.policy`, first candidate is 
`lens.driver..retry.policy`, followed by 
`lens.driver.retry.policy`, followed by `retry.policy`. Now, driverType is 
`mockHive`, so basically it's looking for `lens.driver.mockHive.retry.policy` 
inside a file named `hivedriver-site.xml`. While I think that it should instead 
look for `lens.driver.hive.retry.policy`, but now, the only way to extract 
`hive` is through getting classname. 

I've changed the logic to now check 4 candidates instead of 3 as mentioned in 
the above paragraph. The new candidate now be 
`lens.driver..retry.policy`, and it'll be checked after 
`lens.driver..retry.policy`.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
---


On Oct. 7, 2016, 4:51 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 7, 2016, 4:51 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-07 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 7, 2016, 4:51 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 e160f58bfb45cffb05b9f6be19db3bc7605dde57 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-07 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review151768
---




lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 (line 80)


Is this required? If not, lets remove it.

Also, why does removal of a driver set selectedDriver to null?


- Amareshwari Sriramadasu


On Oct. 6, 2016, 12:29 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Oct. 6, 2016, 12:29 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 0d8810f6b7dfd9fb0173a4be6966f1a561107066 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java
>  daf01ac64616da72ee55c00f71c2c0aa33651ee4 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
>  d80fba9aee91130b69a2a511d7f157c01a261ac4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  e4984798751d012f0a9d01980df833c206155546 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
>  5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
>  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-06 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 6, 2016, 5:59 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-05 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 5, 2016, 1:40 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-05 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 5, 2016, 1:39 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-10-04 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Oct. 4, 2016, 1:51 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
0d8810f6b7dfd9fb0173a4be6966f1a561107066 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
daf01ac64616da72ee55c00f71c2c0aa33651ee4 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 1acbb13c3a24c0e7b41d799db0d307c5acfa8c6b 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
f805ec6ebccd18de3bfcd0f1ab8626f7330dd6f1 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-regression/src/main/java/org/apache/lens/regression/core/constants/DriverConfig.java
 d80fba9aee91130b69a2a511d7f157c01a261ac4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 e4984798751d012f0a9d01980df833c206155546 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-30 Thread Rajat Khandelwal


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > checkstyle/src/main/resources/checkstyle.xml, line 65
> > 
> >
> > hmm, what file is extending this? Need to fix by doing some refactoring 
> > if required.

Query execution service impl is now 3558 lines. Will see if any refactoring can 
be done.


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java, lines 
> > 303-304
> > 
> >
> > Adding resources specific to driver is removed?

This is being done in super class's method.


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 365
> > 
> >
> > hiveConf would be required here.

Adding overlay in the next line. `super.configure` is supposed to add driver 
specific configurations and this can be asked by `getConf()`


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 368
> > 
> >
> > hivedriver-site.xml is not added in old code.

Refer to the above comment, super class making sure `getConf()` returns driver 
specific configuration. And this line is to make sure the overlay takes effect.


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java,
> >  lines 76-77
> > 
> >
> > This is enforcing drivertype in the file names. This will be a problem 
> > with -default.xml files, if people create driver type to be different than 
> > whats expected to be.
> > 
> > Instead you can try using driver.getClass().getName().toLowercase() ?

I'm fine with either, but unable to see the merit in using one above the other.


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java,
> >  line 82
> > 
> >
> > removing from context map? We might lose some details like plan and 
> > etc. May be should be moved another data structure

Those things will anyway get lost if the query succeeds on another driver. If 
there are no other drivers, this is restored.


> On Sept. 30, 2016, 12:16 p.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java, line 113
> > 
> >
> > can you update state machine on user doc with the change?

Would need to generate the png file again.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
---


On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 29, 2016, 2:39 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-30 Thread Rajat Khandelwal


> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java,
> >  line 41
> > 
> >
> > why do we need clear() functionality ?

The file FailureContext is renamed to `StatusUpdateFailureContext`. It already 
had `clear` method. The method isn't propagated to the interface, but is 
retained in the implementation. The other implementation of the interface 
doesn't have a `clear` method. 

And to answer your question, it's used in 
`QueryContext#updateDriverStatus(BackOffRetryHandler)`.


> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java,
> >  line 27
> > 
> >
> > when can priority of the query can change? and why?

Again, pre-existing code, only moved to another package. AFAIK, it's not being 
used anywhere.


> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java,
> >  line 385
> > 
> >
> > loadDrivers() called separately?

Yes. The method `loadDriversAndSelectors` was basically a bunch of code and a 
call to `loadSelectors`. So extracted that bunch of code into a method 
`loadDrivers`. Both are called sequencially in `init`


> On Sept. 30, 2016, 12:34 p.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java,
> >  line 905
> > 
> >
> > Would need re-run for jdbc queries, as connection would have closed.

I don't think any additional code is required for that. query status will be 
updated to FAILING, and then retry policy will be consulted. The retry policy 
should be taking care of forcing a retry in case a failure is due to connection 
close.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150993
---


On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 29, 2016, 2:39 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> 3bf5e8f7dc9d032e46d06c3245048096acb4572e 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-30 Thread Rajat Khandelwal


> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java,
> >  line 22
> > 
> >
> > Will SingleRetryHandler have delay for the next try?
> 
> Rajat Khandelwal wrote:
> Will add more policies. This is one of the simplest of policies which has 
> only one retry, without any delay.

Have generalized the policy to call it ImmediateRetryPolicy. The constructor 
takes an integer argument which is the number of retries.


> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java,
> >  lines 210-216
> > 
> >
> > All these should be defined as transient? Or there is recovery upon 
> > restarts?
> 
> Rajat Khandelwal wrote:
> The failed attempts will have recovery upon restart
> 
> For retry policies, they need not be stored, but it might result in a 
> query being retried more than the normal amount, in case the query failed and 
> is going through a retry and fails again on restart. Then the retry cycle 
> will restart, causing the retries happened before restart to become 
> redundant. Which should be fine assuming retries will not be that frequent.

Upon second thuoght, I've kept all these non-transient. These will be persisted 
and recovered upon restart, which will maintain consistencey of retries across 
restart. Have added a test case for the same.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
---


On Sept. 29, 2016, 2:39 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 29, 2016, 2:39 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> 3bf5e8f7dc9d032e46d06c3245048096acb4572e 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-30 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150993
---




lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java
 (line 41)


why do we need clear() functionality ?



lens-server-api/src/main/java/org/apache/lens/server/api/query/events/PriorityChange.java
 (line 27)


when can priority of the query can change? and why?



lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java
 (line 25)


Add javadoc for all the methods.



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
 


loadDrivers() called separately?



lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
 (line 900)


Would need re-run for jdbc queries, as connection would have closed.


- Amareshwari Sriramadasu


On Sept. 29, 2016, 9:09 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 29, 2016, 9:09 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> 3bf5e8f7dc9d032e46d06c3245048096acb4572e 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
>  88fbe43318315394873cabb9f5dd39f09f6c47f4 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-30 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review150991
---




checkstyle/src/main/resources/checkstyle.xml (line 65)


hmm, what file is extending this? Need to fix by doing some refactoring if 
required.



lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java (line 112)


can you update state machine on user doc with the change?



lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 


Adding resources specific to driver is removed?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 


hiveConf would be required here.



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 336)


hivedriver-site.xml is not added in old code.



lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 (lines 76 - 77)


This is enforcing drivertype in the file names. This will be a problem with 
-default.xml files, if people create driver type to be different than whats 
expected to be.

Instead you can try using driver.getClass().getName().toLowercase() ?



lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 (line 82)


removing from context map? We might lose some details like plan and etc. 
May be should be moved another data structure


- Amareshwari Sriramadasu


On Sept. 29, 2016, 9:09 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 29, 2016, 9:09 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   checkstyle/src/main/resources/checkstyle.xml 
> 17e55ea1071c06c73e8bf387cd69acca83bc 
>   lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
> PRE-CREATION 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
> 7d079a9ed99e3d4940d5f26370547683a6bba465 
>   lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
> daedbf9ea387cbde3ef39363c2beef3f96725223 
>   
> lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
> 04d9ad6dee942c705b0cdf071f49f1305696da65 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
> a681bca7fd6e46fe992901e04e025a8a3106 
>   
> lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml
>  70f9c932efc090095273566eaa52d380a2f23842 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> fceabeae9217676774a6978212cbc9ee9ca120f7 
>   lens-driver-es/src/test/resources/hive-site.xml 
> e49c876289d5352203d2523ddb9134d745c197e5 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> 84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
>   lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
> 3bf5e8f7dc9d032e46d06c3245048096acb4572e 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
>  3c34eb191966dff3ac3032eb79282336b5c6789b 
>   
> lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
>  82b56470e2de09f7c41d1226ea59cffdd8d26a26 
>   
> lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java
>  6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8cf617b7181b9a305ea2c68f68c610f823ee75d7 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
>  70a34b0c0517ed3b7af46120906fb77ae57870ac 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-29 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Sept. 29, 2016, 2:39 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  checkstyle/src/main/resources/checkstyle.xml 
17e55ea1071c06c73e8bf387cd69acca83bc 
  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-api/src/test/java/org/apache/lens/api/jaxb/YAMLToStringStrategyTest.java 
04d9ad6dee942c705b0cdf071f49f1305696da65 
  lens-api/src/test/resources/toString/org.apache.lens.api.query.LensQuery.xml 
a681bca7fd6e46fe992901e04e025a8a3106 
  
lens-api/src/test/resources/toString/org.apache.lens.api.query.QueryStatus.xml 
70f9c932efc090095273566eaa52d380a2f23842 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-es/src/test/resources/hive-site.xml 
e49c876289d5352203d2523ddb9134d745c197e5 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
3bf5e8f7dc9d032e46d06c3245048096acb4572e 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 
81c53be905d015bd14c0a05de5c7a43f8a6c5d84 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java
 af8c8ee062c4c43ae987cea7316a36d84215d389 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java 
bdffbc2f37bae7a59add1ee3823de87887483e5e 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-28 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Sept. 28, 2016, 7:47 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
3bf5e8f7dc9d032e46d06c3245048096acb4572e 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueuedForRetry.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
 e0f1376d48457d53c5c9bafa673304df4b54cc50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java
 12d9562566e71477db3cf00e1e0ed84d43eeb6cc 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java
 PRE-CREATION 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-28 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Sept. 28, 2016, 7:55 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/query/FailedAttempt.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 
7d079a9ed99e3d4940d5f26370547683a6bba465 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
fceabeae9217676774a6978212cbc9ee9ca120f7 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
  lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java 
3bf5e8f7dc9d032e46d06c3245048096acb4572e 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java
 3c34eb191966dff3ac3032eb79282336b5c6789b 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverConfiguration.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/PriorityChange.java
 eaf3fee953e1efbcf151fc844a9441b74b84cc3e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryAccepted.java
 8980b619c9feb86be326bf799fecc35c1579c763 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryCancelled.java
 a473a47606df84e2b5b204478ad63e63e69b0c23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryClosed.java 
3837087db6e51c1606573511bfc02b6108d1ef3d 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEnded.java 
e80da6d2a417e1274fb186153ec01615c2da565b 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryEvent.java 
81c53be905d015bd14c0a05de5c7a43f8a6c5d84 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecuted.java
 af8c8ee062c4c43ae987cea7316a36d84215d389 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryFailed.java 
bdffbc2f37bae7a59add1ee3823de87887483e5e 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryLaunched.java
 5fcdd73899e3d8c8368fdc42ff6cc9912323e5ee 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryQueued.java 
5f347ad0e751d9cfc6b647d57d0acc25fd8dee01 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRejected.java
 c4e60e3d72108bb556f6cd9a5887230b39d67a02 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryRunning.java
 52aa50d60cd2c1c03a76351830de2942ee84687d 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-27 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Sept. 27, 2016, 5:29 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  contrib/clients/python/pom.xml a849038ba4c6b52fc898e3a7ae41c604e9452de1 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
  
lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/MaxJDBCConnectionCheckConstraint.java
 82b56470e2de09f7c41d1226ea59cffdd8d26a26 
  
lens-driver-jdbc/src/test/java/org/apache/lens/driver/jdbc/TestJdbcDriver.java 
6e9086f9e8dc302ad2171d7f5636a24cb79dcb50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FailureContext.java
 70a34b0c0517ed3b7af46120906fb77ae57870ac 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/OperationRetryHandlerFactory.java
 88fbe43318315394873cabb9f5dd39f09f6c47f4 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/StatusUpdateFailureContext.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
 e0f1376d48457d53c5c9bafa673304df4b54cc50 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java
 12d9562566e71477db3cf00e1e0ed84d43eeb6cc 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/BackOffRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/DefaultRetryPolicyDecider.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FailureContext.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/FibonacciExponentialBackOffRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/ImmediateRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/retry/RetryPolicyDecider.java
 PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
8261d8aa19eb26e0336846d32c5bcd2162ddcb14 
  
lens-server-api/src/test/java/org/apache/lens/server/api/common/TestExponentialBackOffRetryHandler.java
 5f407af815c0e43c51d185a30cb0b639e7696af7 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-26 Thread Rajat Khandelwal

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/
---

(Updated Sept. 26, 2016, 8:29 p.m.)


Review request for lens.


Bugs: LENS-743
https://issues.apache.org/jira/browse/LENS-743


Repository: lens


Description
---

There have to be retries for query failures for transient errors like network 
errors (Hive server not reachable/ Metastore not reachable/ DB not reachable). 
Retries should be available for each phase - submission, execution, updating 
status, fetching results and formatting.

Right now, any such failure results in marking query as failed.


Diffs (updated)
-

  contrib/clients/python/pom.xml a849038ba4c6b52fc898e3a7ae41c604e9452de1 
  lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java 
daedbf9ea387cbde3ef39363c2beef3f96725223 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
84d9933da506e3f1bff2d3e0ea403ed1edf0fe35 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8cf617b7181b9a305ea2c68f68c610f823ee75d7 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
 17bfba02244bca7141268ed12e4a41400e2bec82 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
 e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
 03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
 033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java 
e472de0a452ffbbcb217503c260aa26fe7f6ed99 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
 5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
 b58fcf964cedb8dc302099ae9c806b06acafa2d3 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 b584c6a6d414a6f0dac509e67c8944dbd2373325 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/ChainedComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/FIFOQueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/MoreRetriesFirstComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryCostComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/comparators/QueryPriorityComparator.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java
 PRE-CREATION 
  lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
8261d8aa19eb26e0336846d32c5bcd2162ddcb14 
  
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java 
168b3cc8a2e32e4fddb66f480af38e77adef62c2 
  
lens-server-api/src/test/java/org/apache/lens/server/api/query/comparators/ChainedComparatorTest.java
 PRE-CREATION 
  lens-server/pom.xml 6dea9a7be5181db1ff2f0f2e4407d4208fbb5464 
  
lens-server/src/main/java/org/apache/lens/server/query/FIFOQueryComparator.java 
75c11460f7a0c2c5a52d6fec5e5f4b763643f3a4 
  lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 
0aada406963f7b421e8b9726a72abcd85c63ddd4 
  lens-server/src/main/java/org/apache/lens/server/query/QueryComparator.java 
67dda6bf8a74dc4a21b7f4148e5e3af9a51721d0 
  
lens-server/src/main/java/org/apache/lens/server/query/QueryCostComparator.java 
2702581fc88a560dbb420ec3a7de1cf442542bd7 
  
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
 0bb106ed77306094710c126fc652d5c56a908d85 
  
lens-server/src/main/java/org/apache/lens/server/query/QueryPriorityComparator.java
 2c6d904a5fd35d39b5339275013386ae7852bb73 
  
lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java
 48291b94d35dfcf3dde6b3667b2aae28d81f9366 
  
lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java
 PRE-CREATION 
  

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-22 Thread Rajat Khandelwal


> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java,
> >  lines 210-216
> > 
> >
> > All these should be defined as transient? Or there is recovery upon 
> > restarts?

The failed attempts will have recovery upon restart

For retry policies, they need not be stored, but it might result in a query 
being retried more than the normal amount, in case the query failed and is 
going through a retry and fails again on restart. Then the retry cycle will 
restart, causing the retries happened before restart to become redundant. Which 
should be fine assuming retries will not be that frequent.


> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java,
> >  line 35
> > 
> >
> > Should we have errorcode as well here ?

Where will the error code be extracted from? Right now, the failed attempt is 
extracted from driver status, which only has error message.


> On Sept. 22, 2016, 11:53 a.m., Amareshwari Sriramadasu wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java,
> >  line 22
> > 
> >
> > Will SingleRetryHandler have delay for the next try?

Will add more policies. This is one of the simplest of policies which has only 
one retry, without any delay.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
---


On Sept. 20, 2016, 11:56 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 20, 2016, 11:56 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f1983ee10e9e4a429dc1c8ceb5d4a65bab975c1 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
>  5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
>  b58fcf964cedb8dc302099ae9c806b06acafa2d3 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
>  b584c6a6d414a6f0dac509e67c8944dbd2373325 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ChainedComparator.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java
>  PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
> 8261d8aa19eb26e0336846d32c5bcd2162ddcb14 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 
> 997b5a80ab1f9516ad6aaeb431f08a61f386180a 
>   
> 

Re: Review Request 52088: LENS-743: Query failure retries for transient errors

2016-09-22 Thread Amareshwari Sriramadasu

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52088/#review149955
---




lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java
 (line 22)


Will SingleRetryHandler have delay for the next try?



lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java
 (line 35)


Should we have errorcode as well here ?



lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
 (lines 210 - 216)


All these should be defined as transient? Or there is recovery upon 
restarts?


- Amareshwari Sriramadasu


On Sept. 20, 2016, 6:26 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52088/
> ---
> 
> (Updated Sept. 20, 2016, 6:26 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-743
> https://issues.apache.org/jira/browse/LENS-743
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> There have to be retries for query failures for transient errors like network 
> errors (Hive server not reachable/ Metastore not reachable/ DB not 
> reachable). Retries should be available for each phase - submission, 
> execution, updating status, fetching results and formatting.
> 
> Right now, any such failure results in marking query as failed.
> 
> 
> Diffs
> -
> 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f1983ee10e9e4a429dc1c8ceb5d4a65bab975c1 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/BackOffRetryHandler.java
>  17bfba02244bca7141268ed12e4a41400e2bec82 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/FibonacciExponentialBackOffRetryHandler.java
>  e7fb8ce0194803317aaa0e1c5f4e3e0fb6bbca8c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/NoRetryHandler.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/common/SingleRetryHandler.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/AbstractLensDriver.java
>  03079e2dc353fa028f3c5d5a03a8ae2dbf728004 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
>  033f6777b4be7a1b6d6cdc350e42a94ade1acbca 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java
>  e472de0a452ffbbcb217503c260aa26fe7f6ed99 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java
>  5ff59bd0da713651a7cc5261dbd18cd88e91fc76 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FailedAttempt.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
>  b58fcf964cedb8dc302099ae9c806b06acafa2d3 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
>  b584c6a6d414a6f0dac509e67c8944dbd2373325 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ChainedComparator.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/DefaultQueryRetryPolicyDecider.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/retry/QueryRetryPolicyDecider.java
>  PRE-CREATION 
>   lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java 
> 8261d8aa19eb26e0336846d32c5bcd2162ddcb14 
>   lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java 
> 997b5a80ab1f9516ad6aaeb431f08a61f386180a 
>   
> lens-server/src/main/java/org/apache/lens/server/query/MoreRetriesFirstComparator.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
>  dafe93dde2ec8a040b855261f00723be6cd7b26e 
>   
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java
>  48291b94d35dfcf3dde6b3667b2aae28d81f9366 
>   
> lens-server/src/main/java/org/apache/lens/server/query/constraint/RetryPolicyToConstraingAdapter.java
>  PRE-CREATION 
>   lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java 
> e620dc561774b396b6a498fb5b2e0f4001aaa46f 
> 
> Diff: https://reviews.apache.org/r/52088/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>