[
https://issues.apache.org/jira/browse/BOOKKEEPER-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229964#comment-15229964
]
Flavio Junqueira commented on BOOKKEEPER-916:
---------------------------------------------
I really like the discussion about storage tiers. I'd like us to have a deeper
discussion about it before endeavoring into a feature implementation. Here are
some thoughts.
Long term storage and storage tiers have different aspects and implications.
For long term storage, we probably want to have say archival bookies and have
regular bookies replicating data there when they go cold rather than have
clients directly writing into the cold storage layer. I'm assuming that the
writes into cold storage are not necessarily fast and consequently you don't
want to have it in the critical path of addEntry requests. Related to this
observation is how often ledger data goes cold and that one needs to store long
term. In the applications I've seen, it doesn't happen often, ledgers are
rather short-lived, but I'd like to hear opinions.
On storage tiers, I assume we may want to be able to offer different ways of
storing the data to client applications based on performance constraints,
reliability, isolation. I think that making a distinction between SSDs and HDDs
is a good one, and one tier we might want to care about because it might speed
up things quite a bit for us is non-volatile RAM.
> Placement policy to accomodate different types of ledger strorages
> ------------------------------------------------------------------
>
> Key: BOOKKEEPER-916
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-916
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
> Affects Versions: 4.3.2
> Reporter: Venkateswararao Jujjuri (JV)
> Assignee: Venkateswararao Jujjuri (JV)
>
> As we start to use bookkeeper as long term storage, it may not be right use
> of resources to keep all copies of entry (write ensemble) on efficient
> storage. This feature is to come up with an intelligent placement that
> distributes entry copies across different classes of storage.
> Simply put, say we have SSD based ledger storage and HDD based ledger storage
> on each system. Instead of putting all copies of entries either on SSD or on
> HDD, this placement policy maintains one copy on SSD and others on HDD.
> - Have at least one copy on SSD and others on HDD.
> - Writer need to be aware of this classification
> - Replication logic need to be aware of this logic.
> - While reading attempt to read from SSD first.
> - Reader also need to be aware of this logic.
> This will push bookkeeper towards the long term storage, also can be a
> stepping store towards introducing storage tiers in the future.
> This has dependency/relation to
> https://issues.apache.org/jira/browse/BOOKKEEPER-912.
> https://issues.apache.org/jira/browse/BOOKKEEPER-915
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
