[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-10-27 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15611902#comment-15611902 ] Eshcar Hillel commented on HBASE-16417: --- The report of the first round of experiment is ready

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-10-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614876#comment-15614876 ] Eshcar Hillel commented on HBASE-16417: --- First I plan to experiment with what we implemented so far

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15725746#comment-15725746 ] Eshcar Hillel commented on HBASE-16417: --- Thanks [~ram_krish]. AFAIK 2.0 is not released yet. I

[jira] [Comment Edited] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741396#comment-15741396 ] Eshcar Hillel edited comment on HBASE-17294 at 12/12/16 9:11 AM: - Dear

[jira] [Created] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
Eshcar Hillel created HBASE-17294: - Summary: External Configuration for Memory Compaction Key: HBASE-17294 URL: https://issues.apache.org/jira/browse/HBASE-17294 Project: HBase Issue Type:

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Description: We would like to have a single external knob to control memstore compaction.

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Status: Patch Available (was: Open) > External Configuration for Memory Compaction >

[jira] [Assigned] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-17294: - Assignee: Eshcar Hillel > External Configuration for Memory Compaction >

[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741396#comment-15741396 ] Eshcar Hillel commented on HBASE-17294: --- Dear [~saint@gmail.com], [~anoop.hbase], [~ram_krish],

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Attachment: HBASE-17294-V01.patch > External Configuration for Memory Compaction >

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-13 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Attachment: HBASE-17294-V03.patch > External Configuration for Memory Compaction >

[jira] [Created] (HBASE-17316) Addendum to HBASE-17294

2016-12-14 Thread Eshcar Hillel (JIRA)
Eshcar Hillel created HBASE-17316: - Summary: Addendum to HBASE-17294 Key: HBASE-17316 URL: https://issues.apache.org/jira/browse/HBASE-17316 Project: HBase Issue Type: Sub-task

[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17316: -- Attachment: HBASE-17316.patch > Addendum to HBASE-17294 > --- > >

[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17316: -- Status: Patch Available (was: Open) > Addendum to HBASE-17294 > --- > >

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Release Note: This patch provides a single external knob to control memstore compaction.

[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747734#comment-15747734 ] Eshcar Hillel commented on HBASE-17316: --- Updated the two tests that failed during the commit of

[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747741#comment-15747741 ] Eshcar Hillel commented on HBASE-17294: --- Opened HBASE-17316 with a new patch which updates the two

[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294

2016-12-14 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748035#comment-15748035 ] Eshcar Hillel commented on HBASE-17316: --- QA passed. [~stack] please consider committing this

[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction

2016-12-13 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744726#comment-15744726 ] Eshcar Hillel commented on HBASE-17294: --- Working on a new patch following your comments. Please

[jira] [Updated] (HBASE-17294) External Configuration for Memory Compaction

2016-12-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17294: -- Attachment: HBASE-17294-V02.patch > External Configuration for Memory Compaction >

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-07 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728092#comment-15728092 ] Eshcar Hillel commented on HBASE-16417: --- I am not suggesting to set eager as default. We can have

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-12 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820586#comment-15820586 ] Eshcar Hillel commented on HBASE-17434: --- No failures this time ;) Patch V06 fixes some typo in the

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-11 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V06.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724919#comment-15724919 ] Eshcar Hillel commented on HBASE-16417: --- Thanks for the comments [~anoop.hbase]. The scope of the

[jira] [Updated] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-16417: -- Attachment: HBASE-16417-benchmarkresults-20161205.pdf > In-Memory MemStore Policy for

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722264#comment-15722264 ] Eshcar Hillel commented on HBASE-16417: --- Attached new benchmarks results. This time we focus on

[jira] [Created] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
Eshcar Hillel created HBASE-17434: - Summary: New Synchronization Scheme for Compaction Pipeline Key: HBASE-17434 URL: https://issues.apache.org/jira/browse/HBASE-17434 Project: HBase Issue

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805533#comment-15805533 ] Eshcar Hillel commented on HBASE-17434: --- The current patch implements copy-on-write in the

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805369#comment-15805369 ] Eshcar Hillel commented on HBASE-17379: --- Created a new Jira HBASE-17434 to fix the bug of

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Status: Patch Available (was: Open) > New Synchronization Scheme for Compaction Pipeline >

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V01.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805355#comment-15805355 ] Eshcar Hillel commented on HBASE-17434: --- The design of the new scheme is discussed in HBASE-17379.

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-06 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805502#comment-15805502 ] Eshcar Hillel commented on HBASE-17434: --- The problem reported in HBASE-17379 can be fixed only in

[jira] [Created] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-03 Thread Eshcar Hillel (JIRA)
Eshcar Hillel created HBASE-17407: - Summary: Correct update of maxFlushedSeqId in HRegion Key: HBASE-17407 URL: https://issues.apache.org/jira/browse/HBASE-17407 Project: HBase Issue Type:

[jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794617#comment-15794617 ] Eshcar Hillel commented on HBASE-17407: --- In HRegion::internalFlushCacheAndCommit() we have the

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794623#comment-15794623 ] Eshcar Hillel commented on HBASE-17379: --- Moved the discussion to HBASE-17407. > Lack of

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794563#comment-15794563 ] Eshcar Hillel commented on HBASE-17379: --- Thanks for looking into this [~Apache9]. I am opening a

[jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794590#comment-15794590 ] Eshcar Hillel commented on HBASE-17407: --- Continuing the discussion started at

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795419#comment-15795419 ] Eshcar Hillel commented on HBASE-17373: --- Last patch removed all unused code. Please commit. >

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-02 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V04.patch > Reverse the order of snapshot creation in the

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-02 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V04.patch > Reverse the order of snapshot creation in the

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-02 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792539#comment-15792539 ] Eshcar Hillel commented on HBASE-17373: --- These are two different problems. Here the problem was

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-02 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792644#comment-15792644 ] Eshcar Hillel commented on HBASE-17379: --- Looking at the current trunk getTailSize() is only invoked

[jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795278#comment-15795278 ] Eshcar Hillel commented on HBASE-17407: --- Updating the WAL is not done in one atomic step; It

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-03 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V05.patch > Reverse the order of snapshot creation in the

[jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800723#comment-15800723 ] Eshcar Hillel commented on HBASE-17407: --- Thanks [~stack] your explanation is very helpful (but for

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-04 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800664#comment-15800664 ] Eshcar Hillel commented on HBASE-17379: --- I went over the patch. Still it seems much more

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800687#comment-15800687 ] Eshcar Hillel commented on HBASE-17379: --- I went over the trunk code again, and I now see that

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800700#comment-15800700 ] Eshcar Hillel commented on HBASE-17379: --- Meanwhile, until we decide which way to go here is a link

[jira] [Comment Edited] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-05 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800687#comment-15800687 ] Eshcar Hillel edited comment on HBASE-17379 at 1/5/17 8:10 AM: --- I went over

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-04 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797549#comment-15797549 ] Eshcar Hillel commented on HBASE-17373: --- [~ram_krish] I think you now have good understanding of

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811438#comment-15811438 ] Eshcar Hillel commented on HBASE-17434: --- Following HBASE-17373 the drain method is not needed

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V03.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811443#comment-15811443 ] Eshcar Hillel commented on HBASE-17434: --- V03 changes version to be volatile since it is long and

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811674#comment-15811674 ] Eshcar Hillel commented on HBASE-17434: --- This is the same as in DefaultMemstote - the active

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811593#comment-15811593 ] Eshcar Hillel commented on HBASE-17434: --- The comment by FindBugs can be ignored since the version

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-09 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811590#comment-15811590 ] Eshcar Hillel commented on HBASE-17434: --- The logic and timing of moving segments into the pipeline

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization for Get Operation

2016-12-31 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15790072#comment-15790072 ] Eshcar Hillel commented on HBASE-17339: --- [~stack] I see the scheme described in HBASE-10247. I

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2016-12-31 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15789991#comment-15789991 ] Eshcar Hillel commented on HBASE-17373: --- Since HBASE-17379 is not committed yet, would it be

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization for Get Operation

2016-12-31 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15790029#comment-15790029 ] Eshcar Hillel commented on HBASE-17339: --- Thanks [~enis]. What is the advantage of checking

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-31 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15789987#comment-15789987 ] Eshcar Hillel commented on HBASE-17379: --- The synchronization scheme is as follows: - use read-write

[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore

2016-12-31 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15790008#comment-15790008 ] Eshcar Hillel commented on HBASE-16421: --- [~ram_krish] have you tried using batching? if so what is

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V03.patch > Reverse the order of snapshot creation in the

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15790952#comment-15790952 ] Eshcar Hillel commented on HBASE-17379: --- Here is an even simpler synchronization scheme. [~stack]

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791089#comment-15791089 ] Eshcar Hillel commented on HBASE-17373: --- [~stack], [~ted_yu], other commiter watchers, please

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791210#comment-15791210 ] Eshcar Hillel commented on HBASE-17373: --- A rebase due to revert of HBASE-17081. > Reverse the

[jira] [Comment Edited] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15790952#comment-15790952 ] Eshcar Hillel edited comment on HBASE-17379 at 1/1/17 12:12 PM: Here is

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791034#comment-15791034 ] Eshcar Hillel commented on HBASE-17379: --- Regardless of the new clone, there are additional issues

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-02 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792871#comment-15792871 ] Eshcar Hillel commented on HBASE-17379: --- The current implementation of a compacting memstore only

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2017-01-01 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791368#comment-15791368 ] Eshcar Hillel commented on HBASE-17373: --- I don't know what is the policy or rule of thumb for

[jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion

2017-01-04 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797653#comment-15797653 ] Eshcar Hillel commented on HBASE-17407: --- Here is what I read: {code} void abortCacheFlush(final

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2017-01-04 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797876#comment-15797876 ] Eshcar Hillel commented on HBASE-17379: --- bq. Before HBASE-17081 - you mean the problem was there?

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-08 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V02.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-08 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15810016#comment-15810016 ] Eshcar Hillel commented on HBASE-17434: --- I also prefer read-write lock over an exclusive lock when

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-08 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15810038#comment-15810038 ] Eshcar Hillel commented on HBASE-17434: --- Making a copy inside the building block methods and

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-10 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V05.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-10 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814318#comment-15814318 ] Eshcar Hillel commented on HBASE-17434: --- Thanks. So here are my answers (some are already in RB):

[jira] [Updated] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-10 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17434: -- Attachment: HBASE-17434-V04.patch > New Synchronization Scheme for Compaction Pipeline >

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-10 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814832#comment-15814832 ] Eshcar Hillel commented on HBASE-17434: --- Java does not have pointers. It has primitive variables

[jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline

2017-01-10 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815057#comment-15815057 ] Eshcar Hillel commented on HBASE-17434: --- With the possibility of non-atomic initialization of a

[jira] [Created] (HBASE-17339) Scan-Memory-First Optimization

2016-12-19 Thread Eshcar Hillel (JIRA)
Eshcar Hillel created HBASE-17339: - Summary: Scan-Memory-First Optimization Key: HBASE-17339 URL: https://issues.apache.org/jira/browse/HBASE-17339 Project: HBase Issue Type: Improvement

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization

2016-12-19 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761344#comment-15761344 ] Eshcar Hillel commented on HBASE-17339: --- Benchmark results presented in HBASE-16417 show that this

[jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions

2016-12-19 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761351#comment-15761351 ] Eshcar Hillel commented on HBASE-16417: --- Opened a new Jira with the patch for scan-memory-first

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization

2016-12-20 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763596#comment-15763596 ] Eshcar Hillel commented on HBASE-17339: --- Thanks all for commenting. Indeed this optimization would

[jira] [Updated] (HBASE-17339) Scan-Memory-First Optimization for Get Operation

2016-12-20 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17339: -- Summary: Scan-Memory-First Optimization for Get Operation (was: Scan-Memory-First

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization

2016-12-20 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763689#comment-15763689 ] Eshcar Hillel commented on HBASE-17339: --- Bellow is the summary of when the optimization can and

[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk

2016-12-18 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760460#comment-15760460 ] Eshcar Hillel commented on HBASE-17081: --- Just some clarification w.r.t. which memstore is running:

[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk

2016-12-19 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15760756#comment-15760756 ] Eshcar Hillel commented on HBASE-17081: --- bq. enabling BASIC by default for all cases am not sure if

[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782921#comment-15782921 ] Eshcar Hillel commented on HBASE-16421: --- *maxscanlength*: for scans, what is the maximum number of

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782959#comment-15782959 ] Eshcar Hillel commented on HBASE-17379: --- You use read lock in flattening method since it changes

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V02.patch > Reverse the order of snapshot creation in the

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782971#comment-15782971 ] Eshcar Hillel commented on HBASE-17373: --- added ASF license to new files in patch. > Reverse the

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782637#comment-15782637 ] Eshcar Hillel commented on HBASE-17379: --- Hi [~ted_yu] some of the changes you suggest here may

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783314#comment-15783314 ] Eshcar Hillel commented on HBASE-17379: --- If you think about it the bug was there all the time since

[jira] [Commented] (HBASE-17339) Scan-Memory-First Optimization for Get Operation

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783076#comment-15783076 ] Eshcar Hillel commented on HBASE-17339: --- If I understand correctly then when clients put cells into

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783082#comment-15783082 ] Eshcar Hillel commented on HBASE-17379: --- Seems the test failed with timeout, might be a deadlock.

[jira] [Commented] (HBASE-17379) Lack of synchronization in CompactionPipeline#getScanners()

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782785#comment-15782785 ] Eshcar Hillel commented on HBASE-17379: --- I agree that using a read write lock would be a valid

[jira] [Commented] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782802#comment-15782802 ] Eshcar Hillel commented on HBASE-17373: --- It seems that parameterized tests can pass the parameters

[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782866#comment-15782866 ] Eshcar Hillel commented on HBASE-16421: --- [~ram_krish] when running YCSB you can have properties

[jira] [Updated] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore

2016-12-28 Thread Eshcar Hillel (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-17373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17373: -- Attachment: HBASE-17373-V01.patch > Reverse the order of snapshot creation in the

<    1   2   3   4   5   6   7   8   >