[
https://issues.apache.org/jira/browse/LENS-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354781#comment-14354781
]
Hudson commented on LENS-281:
-----------------------------
SUCCESS: Integrated in Lens-Commit #629 (See
[https://builds.apache.org/job/Lens-Commit/629/])
LENS-281 : Create Partition Timeline and keep it cached and stored as storage
table properties (Rajat Khandelwal via amareshwari) (amareshwari: rev
803448a18e901a5c3dd9e4a70aa03935be05790b)
*
lens-cube/src/main/java/org/apache/lens/cube/metadata/CaseInsensitiveStringHashMap.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/StorageUtil.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java
*
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimeline.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/Storage.java
* lens-cube/src/test/java/org/apache/lens/cube/parse/TestStorageUtil.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java
*
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/PartitionTimelineFactory.java
*
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/StoreAllPartitionTimeline.java
*
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java
*
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
* lens-cube/src/test/java/org/apache/lens/cube/metadata/TestTimePartition.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java
*
lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
* lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/ORTimeRangeWriter.java
*
lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeUtils.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/BetweenTimeRangeWriter.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRange.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/StorageConstants.java
*
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/EndsAndHolesPartitionTimeline.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/FactPartition.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java
* lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/TimeRangeWriter.java
* lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
> Create Partition Timeline and keep it cached and stored as storage table
> properties
> -----------------------------------------------------------------------------------
>
> Key: LENS-281
> URL: https://issues.apache.org/jira/browse/LENS-281
> Project: Apache Lens
> Issue Type: Sub-task
> Components: cube
> Reporter: Amareshwari Sriramadasu
> Assignee: Rajat Khandelwal
> Fix For: 2.1
>
> Attachments: rb31154.patch
>
>
> For each fact-storage table, for each of fact-storage table's update periods,
> for each of fact-storage table's time partitioning column, maintain a
> Timeline data structure which records all the time partitions existing for
> the combination in memory. This info will be cached in memory for server
> lifecycle. For the first time, it will be evaluated from all existing
> partitions and stored in table properties. For next server restarts, this
> will be read from the properties. all partition
> operations(add/delete/latest/...) will update the timeline and use it as best
> as they can.
> For the Timeline Data structure, the following implementations come to mind:
> * Store list of all partitions.
> * Store first, last and in-between holes.
> * Store as an interval tree/segment tree.
> As part of this jira, I intend to provide an interface, atleast one
> implementation and (maybe) configurability of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
