[
https://issues.apache.org/jira/browse/LENS-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14722651#comment-14722651
]
Amareshwari Sriramadasu commented on LENS-576:
----------------------------------------------
bq. I realize that serializing QueryOutputFormatter may not be the best idea.
This object is quite heavy to serialize.
+1
bq. What we can do is - we will persist all the queries in the SQL database as
soon as they are finished. This way, persistent resultsets will always be
obtained from SQL db. Also, we will still keep
"lens.server.max.finished.queries" queries in memory and once the queue
overflows, we can simply purge the query from the queue in memory.
Right now, one of the reasons finished queries are available in server is
because of some event threads accessing some information that is not persisted.
If we can remove that dependency (for ex : LENS-693), we can always immediately
persist the finished query.
If finished queries are immediately persisted, server stop should also persist
the queries.
But there might be scenarios corresponding to server crashes, where finished
queries are not persisted - in those cases we should still have a way to read
them back from snapshotted state. QueryOutputFormatter may not be serializable,
but we should serialize information required to construct the finished query
back - similar to the data put in SQL database.
> Cannot retrieve query results on lens server restart
> ----------------------------------------------------
>
> Key: LENS-576
> URL: https://issues.apache.org/jira/browse/LENS-576
> Project: Apache Lens
> Issue Type: Bug
> Reporter: Deepak Barr
> Assignee: Deepak Barr
> Priority: Critical
> Fix For: 2.4
>
>
> In the event of lens server restart, We can not obtain the query result by
> calling "/queryuiapi/queries/{queryHandle}/httpresultset" API as
> LensPersistentResult does not get created. This is because
> QueryOutputFormatter variable in QueryContext object is always null.This
> QueryOutputFormatter is important to create LensPersistedResultSet (see
> getResultset() method in QueryExecutionServiceImpl class) .
> Reason : One can see that QueryOutputFormatter variable in QueryContext class
> is not serialized. Therefore, on server restart, this variable is always null
> in QueryContext for all queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
