[
https://issues.apache.org/jira/browse/LENS-576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805025#comment-14805025
]
Puneet Gupta commented on LENS-576:
-----------------------------------
Summary of discussion with Deepak [~deepak.barr] and Rajat [~prongs] .
Overall we have realized two problems
1. How to plug-in serialization at higher level
A— move/copy required 4 fields from QueryOutputFormatter to QueryContext
B— serialize those four fields separately in
QueryExecutionServiceImpl.writeExternal(ObjectOutput)
C— Make QueryContext.queryOutputFormatter non transient and make specific
parts in all formatters transient
D— New approach : Let QueryOutputFormatter implement Externalizable and let
the AbstractOutputFormatter handle Externalizable implementation in one place
for all implementations of QueryOutputFormatter ( We will have to have
definition for the required 4 members in this AbstractOutputFormatter in this
case)
****We can choose an approach. I think D might be cleanest and easiest if its
feasible.
2. How to handle serialization of members that come from third party libraries
(Hive/Hadoop) that are not implementing Serializable.This needs to be handled
for finalOutputPath and LensResultSetMetadata as pointed by Deepak.
A. Option1
Can we make LensResultSetMetadata implement Externalizable and the use custom
serialization ( use JSON string representation for ColumnDescriptor for
serailization or create a custom class for ColumnDescriptor which can be
serialized as suggested by Deepak )?
B. Option2
We can keep it in sync with Query Purge flow
(QueryExecutionServiceImpl.QueryPurger) where we are using JSON string
representation for serialization of metadata to DB .
>finishedQuery.setMetadataClass(metadata.getClass().getName());
>finishedQuery.setMetadata(MAPPER.writeValueAsString(metadata));
Option B seems better for now as we have a reference in code already.
> Cannot retrieve query results on lens server restart
> ----------------------------------------------------
>
> Key: LENS-576
> URL: https://issues.apache.org/jira/browse/LENS-576
> Project: Apache Lens
> Issue Type: Bug
> Reporter: Deepak Barr
> Assignee: Deepak Barr
> Priority: Critical
> Fix For: 2.4
>
>
> In the event of lens server restart, We can not obtain the query result by
> calling "/queryuiapi/queries/{queryHandle}/httpresultset" API as
> LensPersistentResult does not get created. This is because
> QueryOutputFormatter variable in QueryContext object is always null.This
> QueryOutputFormatter is important to create LensPersistedResultSet (see
> getResultset() method in QueryExecutionServiceImpl class) .
> Reason : One can see that QueryOutputFormatter variable in QueryContext class
> is not serialized. Therefore, on server restart, this variable is always null
> in QueryContext for all queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
