[
https://issues.apache.org/jira/browse/LENS-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14300957#comment-14300957
]
Aniruddha Gangopadhyay commented on LENS-251:
---------------------------------------------
Lens by default assumes data availability until present time. However, to
ensure that the query will have access to the latest data depends on the
partitions being added to respective storage tables. Or, in an altogether
different scenario, where there is no notion of time partitioned data, however,
there is a dynamic time range for which data is available in that storage table
(say last N days or hours, so on). We already have a notion of static time
range associated with partitioned storage tables (inferred by the partitions
added). We need the notion of dynamic time range associated with partitioned as
well as non-partitioned tables. By introducing dynamism in time range of data
available with a partitioned storage table, we effectively talk about a sliding
window which calls for dynamically adding newer partitions and dropping older
ones based on the time window. For non partitioned storage tables, its more
about being aware of the time window for which data is available in that
storage table.
> Streaming Updates in storage
> ----------------------------
>
> Key: LENS-251
> URL: https://issues.apache.org/jira/browse/LENS-251
> Project: Apache Lens
> Issue Type: New Feature
> Components: cube
> Reporter: Sharad Agarwal
>
> For certain stores that allows streaming ingestion, to make the data
> available immediately for querying, we need the notion of streaming update
> period or some such.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
