[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420309#comment-15420309
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
bq. Makes sense. So were you able to observe the problems that we highlighted
in the previous discussions - We got two problems one is the invalid negative
size for memstore and the 'Too many open files'?
Were you able to get this patch running with the PE tool and the configs that
we had updated in the earlier discussions?
We are yet unable to use the PE tool. Even on DefaultMemStore it doesn't work
properly. We are currently running with YCSB, stressing the writes. We didn't
get the exceptions that you mentioned, neither "negative size" nor "too many
open files".
However, we were able to see the problem of "too many open files" without
getting the exception. From here, the quick fix of limiting the number of
segments in the pipeline. This to be managed by a proper policy later.
Later today I am going to publish the patch where we have instrumentation
measuring the time it takes to do the speculative scan. Will you be able to
give it a run on your PE tool? You can by the way take a look if you are still
seeing the "negative size" exception. This one (if still exists) can later be
investigated while dealing with the sizes issues.
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch,
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch,
> HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch,
> HBASE-14921-V09-CAO.patch, InitialCellArrayMapEvaluation.pdf,
> IntroductiontoNewFlatandCompactMemStore.pdf, MemStoreSizes.pdf,
> MemstoreItrCountissue.patch, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
