[
https://issues.apache.org/jira/browse/LENS-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066260#comment-15066260
]
Rajat Khandelwal commented on LENS-899:
---------------------------------------
Hi
After facing more than a few complications, I'm dropping the idea of
*Attempt*s, and picking up the idea of *Failed Attempt*s. I'd already made a
lot of changes for introducing *Attempt*s. I'll describe the problems below,
but for those interested, diff can be seen at
http://github.com/prongs/lens/compare/master...LENS-899.attempts.
Now I'll describe the two approaches and the reasons for picking one over the
other.
h2. Attempts
The drivers will no longer execute the query, they'll just launch an *Attempt*,
The bookkeeping of *Attempts* will be done by Query Service now. An *Attempt*
will be an autonomous entity. So the driver will return an *Attempt* on
_executeAsync_ and that instance will be capable of performing _close_,
_cancel_, _updateStatus_ on itself without any outside help. The *Attempt* will
have the driver status and other driver-specific informations. With these
things in mind, I created the interface
*LensDriver.Attempt*(https://github.com/prongs/lens/blob/LENS-899.attempts/lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java#L194).
The drivers themselves have implementations of this and return their specific
impmentation on _executeAsync_. After launch, query service interacts with the
*Attempt* instead of relying on the driver.
Turns out, the *Attempt* can't be made autonomous. The driver is the bridge
class that allowe LENS to communicate with the underlying execution engine.
Since a Driver's implementation of Attempt needs to be able to communicate with
the underlying execution engine in order to be able to _close_, _update
status_, or _cancel_ itself, the implementation will either
* have the instance of the driver with it -- This will make the separation
even more entangled.
* Be a non-static inner class inside driver class.
So I went with the inner-class approach, which too falls apart when a restart
happens and a new instance of outer driver class is made. After restart, the
attempts are also read and attempt objects are created, but they are orphans
now, since they don't have the instance of outer driver class they were created
with.
Discarding this approach too, I tried to keep *Attempt* and its implementations
and change the communication between query service and drivers. Now, the
*LensDriver* will have all the same methods, but it'll return *Attempt* on
_executeAsync_ and take *Attempt* inside _close_, _updateStatus_, _cancel_ etc.
This doesn't exactly clean up the code since everywhere I have to cast
*LensDriver.Attempt* to driver's own implementation of *Attempt*. Generics also
didn't help much.
Having no other ideas for the *Attempt* approach, I decided to pivot the
approach to *FailedAttempt*s described below:
h2. Failed Attempts
The driver interface stays the same save for the addition of one method
allowing the driver to decide whether the query should be retried or not. All
the drivers extending *AbstractLensDriver* by default say no to retries.
When a query fails, the selected driver is consulted on whether the retry
should happen. If yes, then the query is not removed from _launchedQueries_,
and re-submitted on the driver. Before re-submit, the important members of
driver status are extracted out in a *FailedAttempt* instance, and the driver
status is cleared. The *FailedAttempt* has no _write_ usecases, it only has
_read_ usecases, so as the instance is created, it's persisted to the database.
All the failed attempts reside inside a list inside *QueryContext*. In most
cases, the list will be empty and the server code paths will not cross the new
code.
The database has one more table *_failed_attempts_*. This contains all the
failed attempts for which a re-try was attempted. This won't contain the last
attempt, whether it failed or not. The table has query handle as the join key.
No fields are added in *_finished_queries_*.
The code with the approach is under review at https://reviews.apache.org/r/41506
> Create Attempt framework
> ------------------------
>
> Key: LENS-899
> URL: https://issues.apache.org/jira/browse/LENS-899
> Project: Apache Lens
> Issue Type: Sub-task
> Components: server
> Reporter: Rajat Khandelwal
> Assignee: Rajat Khandelwal
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
