Hi Bob,
Pluggable persistence sounds like a great feature! Help cleaning up this
HDFS feature to be more pluggable is also most welcome :)
I'm interested to hear more about what your ideas are for pluggable
persistence - such as whether you're thinking about swapping out the
persistence at an individual node level (redundancy is still managed be
geode) or at the cluster level (like this HDFS layer, redundancy of the
persistent data is managed elsewhere). I'd love to see your proposal!
-Dan
On Fri, Mar 18, 2016 at 4:24 PM, robert geiger <rob...@ampool.io> wrote:
> +1
>
> Also would like to see the storage layer move to a pluggable model for
> stores other than disk (we would like to contribute this). Would be willing
> to take on the work of turing HDFS into a separate pluggable module as part
> of this effort. If responses are positive will open a Jira to capture the
> pluggable store proposal.
>
> Bob
>
>
>
> On 3/18/16, 4:15 PM, "William Markito" <wmark...@pivotal.io> wrote:
>
> >+1 to move it to "HDFS feature branch". I'd rather have a eviction
> >class(es) specific to HDFS.
> >
> >On Fri, Mar 18, 2016 at 4:03 PM, Dan Smith <dsm...@pivotal.io> wrote:
> >
> >> While looking to the HDFS related changes, I noticed that a new custom
> >> eviction feature was also added related to those changes. Unlike the
> >> already existing CustomExpiry which returns an expiration time for a
> single
> >> key, this takes an EvictionCriteria that is polled periodically and
> returns
> >> return a list of keys to evict.
> >>
> >> I noticed we currently have no tests for this so I'm not sure if it
> >> actually works or not. Is this something we actually want in geode or
> >> should it get removed? My inclination is to move to the HDFS branch,
> >> asssuming we create one, since it came in with that functionality. And
> then
> >> not merge it back to develop until there are tests associated with it.
> >>
> >> -Dan
> >>
> >> /**
> >> * Set custom {@link EvictionCriteria} for the region with start time
> and
> >> * interval of evictor task to be run in milliseconds, or evict
> incoming
> >> rows
> >> * in case both start and frequency are specified as zero.
> >> *
> >> * @param criteria
> >> * an {@link EvictionCriteria} to be used for eviction for
> HDFS
> >> * persistent regions
> >> * @param start
> >> * the start time at which periodic evictor task should be
> first
> >> * fired to apply the provided {@link EvictionCriteria}; if
> this
> >> is
> >> * zero then current time is used for the first invocation of
> >> evictor
> >> * @param interval
> >> * the periodic frequency at which to run the evictor task
> after
> >> the
> >> * initial start; if this is if both start and frequency are
> >> zero
> >> * then {@link EvictionCriteria} is applied on incoming
> >> insert/update
> >> * to determine whether it is to be retained
> >> */
> >> public RegionFactory<K, V> setCustomEvictionAttributes(
> >> EvictionCriteria<K, V> criteria, long start, long interval) {
> >>
> >> /**
> >> * Interface implemented by an EVICTION BY CRITERIA of
> >> * {@link CustomEvictionAttributes}. This will be invoked by periodic
> >> evictor
> >> * task that will get the keys to be evicted using this and then destroy
> >> from
> >> * the region to which this is attached.
> >> *
> >> * @author swale
> >> * @since gfxd 1.0
> >> */
> >> public interface EvictionCriteria<K, V> {
> >>
> >> /**
> >> * Get the (key, routing object) of the entries to be evicted from
> region
> >> * satisfying EVICTION BY CRITERIA at this point of time.
> >> * <p>
> >> * The returned Map.Entry object by the Iterator may be reused
> internally
> >> so
> >> * caller must extract the key, routing object from the entry on each
> >> * iteration.
> >> */
> >> Iterator<Map.Entry<K, Object>> getKeysToBeEvicted(long currentMillis,
> >> Region<K, V> region);
> >>
> >> /**
> >> * Last moment check if an entry should be evicted or not applying the
> >> * EVICTION BY CRITERIA again under the region entry lock in case the
> >> entry
> >> * has changed after the check in {@link #getKeysToBeEvicted}.
> >> */
> >> boolean doEvict(EntryEvent<K, V> event);
> >>
> >> /**
> >> * Return true if this eviction criteria is equivalent to the other
> one.
> >> This
> >> * is used to ensure that custom eviction is configured identically on
> >> all the
> >> * nodes of a cluster hosting the region to which this eviction
> criteria
> >> has
> >> * been attached.
> >> */
> >> boolean isEquivalent(EvictionCriteria<K, V> other);
> >> }
> >>
> >
> >
> >
> >--
> >
> >~/William
>
