[
https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075376#comment-15075376
]
stack commented on HBASE-15016:
-------------------------------
Ok. I tried you patch. Here is some feedback.
So, RegionStoresProxy is meant to be a list of all the 'services' or functions
a Store needs of its encompassing Region. I'd think that a Region would
Implement this Interface or maybe if we want to keep this all contained inside
this package, yeah, you might have a protected method on Region that returns
the provider.
So, I'd think that we'd want the Store to get all its Region services via this
new facility rather than just a few else we'll be in an intermediate place with
half-done stuff. So, if I go to HStore and comment out HRegion, I see that
HRegion is needed to do the following:
{code}
@Override
public long getMemstoreFlushSize() {
// TODO: Why is this in here? The flushsize of the region rather than the
store? St.Ack
return this.region.memstoreFlushSizeLB;
}
...
To get the CP Host
info.setRegionCoprocessorHost(this.region.getCoprocessorHost());
....
To get an executor pool....
// initialize the thread pool for closing store files in parallel.
ThreadPoolExecutor storeFileCloserThreadPool = this.region
.getStoreFileOpenAndCloseThreadPool("StoreFileCloserThread-"
+ this.getColumnFamilyName());
.....
For some functionality that actually doesn't work....
InetSocketAddress[] favoredNodes = null;
if (region.getRegionServerServices() != null) {
favoredNodes = region.getRegionServerServices().getFavoredNodesForRegion(
region.getRegionInfo().getEncodedName());
}
.....
To write a compaction marker to the WAL
HRegionInfo info = this.region.getRegionInfo();
CompactionDescriptor compactionDescriptor =
ProtobufUtil.toCompactionDescriptor(info,
family.getName(), inputPaths, outputPaths,
fs.getStoreDir(getFamily().getNameAsString()));
WALUtil.writeCompactionMarker(region.getWAL(), this.region.getTableDesc(),
this.region.getRegionInfo(), compactionDescriptor, region.getMVCC());
.....
For some weird reporting reason....
this.region.reportCompactionRequestStart(request.isMajor());
....
and here: this.region.reportCompactionRequestEnd(cr.isMajor(),
cr.getFiles().size(), cr.getSize());
....
and for these:
@Override
public RegionCoprocessorHost getCoprocessorHost() {
return this.region.getCoprocessorHost();
}
@Override
public HRegionInfo getRegionInfo() {
return this.fs.getRegionInfo();
}
@Override
public boolean areWritesEnabled() {
return this.region.areWritesEnabled();
}
@Override
public long getSmallestReadPoint() {
return this.region.getSmallestReadPoint();
}
{code}
You'd add blockUpdates/unblockUpdates which seems reasonable and
getWalSequenceId, though you are getting a new sequenceid.... which we should
look at carefully... and then for three reads on memstore state that I think
need better specification:
{code}
public long addAndGetGlobalMemstoreSize(long size) {
return this.region.addAndGetGlobalMemstoreSize(size);
}
public long addAndGetGlobalMemstoreFluctuatingSize(long size) {
return this.memstoreFluctuatingSize.addAndGet(size);
}
public long getGlobalMemstoreActiveSize() {
{code}
I'm not clear on differences between global memstore size, fluctuating size,
and active size. I think users will be confused. Anyways, the Store being
able to add/subtract to the Region-level memory usage accounting is totally
reasonable.
I have other review but maybe I should just press on and complete a version of
this patch of yours to illustrate better what I mean by the review? If that
would help, just say and I'll have a go at it.
> StoreServices facility in Region
> --------------------------------
>
> Key: HBASE-15016
> URL: https://issues.apache.org/jira/browse/HBASE-15016
> Project: HBase
> Issue Type: Sub-task
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-15016-V01.patch, HBASE-15016-V02.patch,
> HBASE-15016-V03.patch
>
>
> The default implementation of a memstore ensures that between two flushes the
> memstore size increases monotonically. Supporting new memstores that store
> data in different formats (specifically, compressed), or that allows to
> eliminate data redundancies in memory (e.g., via compaction), means that the
> size of the data stored in memory can decrease even between two flushes. This
> requires memstores to have access to facilities that manipulate region
> counters and synchronization.
> This subtasks introduces a new region interface -- StoreServices, through
> which store components can access these facilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
