[jira] [Commented] (HBASE-14586) Remove extraneous ${argLine} references in pom

2015-10-09 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951463#comment-14951463 ] Duo Zhang commented on HBASE-14586: --- Can you please verify if jacoco coverage is still working after

[jira] [Commented] (HBASE-14806) Missing sources.jar for several modules when building HBase

2015-11-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003860#comment-15003860 ] Duo Zhang commented on HBASE-14806: --- I tried locally with command {noformat} mvn clean install

[jira] [Updated] (HBASE-14806) Missing sources.jar for several modules when building HBase

2015-11-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14806: -- Attachment: (was: HBASE-14806.patch) > Missing sources.jar for several modules when building HBase

[jira] [Updated] (HBASE-14806) Missing sources.jar for several modules when building HBase

2015-11-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14806: -- Attachment: HBASE-14806.patch > Missing sources.jar for several modules when building HBase >

[jira] [Commented] (HBASE-11236) Last flushed sequence id is ignored by ServerManager

2015-08-31 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724638#comment-14724638 ] Duo Zhang commented on HBASE-11236: --- I think this could happen without HBASE-13811 where [~stack] fix a

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-07 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046422#comment-15046422 ] Duo Zhang commented on HBASE-14004: --- I do not think we can benefit a lot from storing acked length to

[jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy

2015-12-02 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037148#comment-15037148 ] Duo Zhang commented on HBASE-14906: --- {quote} About config in table description, I think we need to

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-02 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037338#comment-15037338 ] Duo Zhang commented on HBASE-14790: --- I read the code in {{NameNode}} and {{DFSOutputStream}} and I

[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-04 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234 ] Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:06 AM: Considering these

[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-04 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234 ] Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:06 AM: Considering these

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15039718#comment-15039718 ] Duo Zhang commented on HBASE-14790: --- {quote} 2. dn1 received the WAL entry, and it is read by

[jira] [Comment Edited] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-04 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041234#comment-15041234 ] Duo Zhang edited comment on HBASE-14790 at 12/4/15 8:02 AM: Considering these

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15040809#comment-15040809 ] Duo Zhang commented on HBASE-14790: --- [~chenheng] We should make a trade off here. I do not think

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037472#comment-15037472 ] Duo Zhang commented on HBASE-14790: --- Oh, I think we could not fix HBASE-14004 without changing the

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-04 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041321#comment-15041321 ] Duo Zhang commented on HBASE-14790: --- {quote} This is clean up of a broken WAL? This is being able to

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15039770#comment-15039770 ] Duo Zhang commented on HBASE-14790: ---

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041147#comment-15041147 ] Duo Zhang commented on HBASE-14790: --- And I found that, {{hsync}} and {{hflush}} have different ack

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-06 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044319#comment-15044319 ] Duo Zhang commented on HBASE-14004: --- {quote} Is it means some entries may be already in place (you

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-06 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044321#comment-15044321 ] Duo Zhang commented on HBASE-14004: --- {quote} If one hlush failed, we close old WAL and write entries

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043246#comment-15043246 ] Duo Zhang commented on HBASE-14004: --- Sounds great, an incremental unique id of WAL entry is better

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-04 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042639#comment-15042639 ] Duo Zhang commented on HBASE-14790: --- We just dig into the implementation of {{DFSOutputStream}} and

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042744#comment-15042744 ] Duo Zhang commented on HBASE-14004: --- {quote} Fixing HBase writing path that we should retry logging WAL

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043664#comment-15043664 ] Duo Zhang commented on HBASE-14004: --- {quote} This will require big change in how replication works but

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038849#comment-15038849 ] Duo Zhang commented on HBASE-14790: --- [~zhz] I was wrong,the pipeline recovery is always needed in the

[jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15039587#comment-15039587 ] Duo Zhang commented on HBASE-14906: --- So there is no exact lower bound in global config? > Improvements

[jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy

2015-12-03 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15039657#comment-15039657 ] Duo Zhang commented on HBASE-14906: --- I'm OK with it. +1 on the newest patch. > Improvements on

[jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy

2015-12-02 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035610#comment-15035610 ] Duo Zhang commented on HBASE-14906: --- +1 on the idea. Only one thing, I think we should still use

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035230#comment-15035230 ] Duo Zhang commented on HBASE-14790: --- basically works. I added a test for it

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-07 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046395#comment-15046395 ] Duo Zhang commented on HBASE-14004: --- {quote} If e don't persist this acked length, during failover we

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-07 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046398#comment-15046398 ] Duo Zhang commented on HBASE-14004: --- {quote} 3. Replication queue recovery starts on another RS, where

[jira] [Commented] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

2015-12-07 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046405#comment-15046405 ] Duo Zhang commented on HBASE-14004: --- {quote} I don't think finally be closed is enough, since the

[jira] [Updated] (HBASE-14969) Add throughput controller for flush

2015-12-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14969: -- Status: Patch Available (was: Open) > Add throughput controller for flush >

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2015-12-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061315#comment-15061315 ] Duo Zhang commented on HBASE-14969: --- Will review the patch later on rb and do you have some perf

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052136#comment-15052136 ] Duo Zhang commented on HBASE-14949: --- Oh, seems fine. {{DefaultMemStore}} uses a

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052064#comment-15052064 ] Duo Zhang commented on HBASE-14949: --- {quote} SequenceId has a region scope. If you play an edit into a

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052072#comment-15052072 ] Duo Zhang commented on HBASE-14949: --- And the only thing I am still a little worry about is how memstore

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052182#comment-15052182 ] Duo Zhang commented on HBASE-14949: --- {quote} Should we add a test that asserts this

[jira] [Commented] (HBASE-15047) Try spin lock for MVCC completion

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074495#comment-15074495 ] Duo Zhang commented on HBASE-15047: --- I think this is just a spin, not a spin lock? And usually I do

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15073391#comment-15073391 ] Duo Zhang commented on HBASE-14949: --- Talked with [~chenheng] offline. The actual difficulty here is how

[jira] [Commented] (HBASE-15047) Try spin lock for MVCC completion

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074590#comment-15074590 ] Duo Zhang commented on HBASE-15047: --- OK, I should add a limitation 'in production'. In micro benchmark,

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074690#comment-15074690 ] Duo Zhang commented on HBASE-14949: --- {quote} If we move sync WAL before update memstore, HBASE-14004

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074641#comment-15074641 ] Duo Zhang commented on HBASE-14790: ---

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074699#comment-15074699 ] Duo Zhang commented on HBASE-14949: --- {quote} This only happens when all datanodes crash, right? {quote}

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-12-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074735#comment-15074735 ] Duo Zhang commented on HBASE-14790: --- {quote} Soon as you have a basic WAL Duo Zhang, I can try get some

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-11-30 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15033281#comment-15033281 ] Duo Zhang commented on HBASE-14790: --- Thanks for the information. Here I'm trying to use only one

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2015-11-30 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15031479#comment-15031479 ] Duo Zhang commented on HBASE-14790: --- Now I‘m trying to implement a fan-out async output stream first.

[jira] [Commented] (HBASE-15968) Strong semantics of versions

2016-06-07 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319918#comment-15319918 ] Duo Zhang commented on HBASE-15968: --- We will use the sequence id of the HFile as the cell's sequence id

[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it

2016-05-31 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15309049#comment-15309049 ] Duo Zhang commented on HBASE-15921: --- [~stack] I think the Future here is the netty one? It has the same

[jira] [Commented] (HBASE-16012) Major compaction can't work because left scanner read point in RegionServer

2016-06-20 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340855#comment-15340855 ] Duo Zhang commented on HBASE-16012: --- [~lhofhansl] The exception is thrown inside the constructor. We

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-01-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102910#comment-15102910 ] Duo Zhang commented on HBASE-14969: --- In general, since we have a new package 'controller' and flush

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-01-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102889#comment-15102889 ] Duo Zhang commented on HBASE-14969: --- [~liyu] Could you please upload the newest patch to review board?

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-01-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103724#comment-15103724 ] Duo Zhang commented on HBASE-14969: --- {quote} My only concern is that this will require user to modify

[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+

2016-06-27 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352049#comment-15352049 ] Duo Zhang commented on HBASE-16110: --- Need a reviewboard link? > AsyncFS WAL doesn't work with Hadoop

[jira] [Updated] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted

2016-06-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16135: -- Attachment: HBASE-16135.patch > PeerClusterZnode under rs of removed peer may never be deleted >

[jira] [Created] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted

2016-06-28 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-16135: - Summary: PeerClusterZnode under rs of removed peer may never be deleted Key: HBASE-16135 URL: https://issues.apache.org/jira/browse/HBASE-16135 Project: HBase

[jira] [Updated] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted

2016-06-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16135: -- Status: Patch Available (was: Open) > PeerClusterZnode under rs of removed peer may never be deleted

[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted

2016-06-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352777#comment-15352777 ] Duo Zhang commented on HBASE-16135: --- I include the code in the patch of HBASE-14476 because the multi

[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+

2016-06-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348959#comment-15348959 ] Duo Zhang commented on HBASE-16110: --- Yeah it does not work. Need some changes. Is hadoop 2.8.0 out? Or

[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+

2016-06-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349097#comment-15349097 ] Duo Zhang commented on HBASE-16110: --- Got it. Let me fix this. > AsyncFS WAL doesn't work with Hadoop

[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+

2016-06-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349466#comment-15349466 ] Duo Zhang commented on HBASE-16110: --- And for hadoop 3, [~stack] and I are trying to push AsyncFSOutput

[jira] [Commented] (HBASE-14949) Skip duplicate entries when replay WAL.

2016-02-08 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136811#comment-15136811 ] Duo Zhang commented on HBASE-14949: --- {quote} It will be better if we use a switch which store in System

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-08 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136985#comment-15136985 ] Duo Zhang commented on HBASE-14790: --- [~eclark] This is the implementation of {{ProtobufLogWriter.sync}}

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133896#comment-15133896 ] Duo Zhang commented on HBASE-14790: --- I have got a 5 regionservers test cluster and run pe tool with

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Description: This is a problem introduced by HBASE-13825 where we change the exception type in catch

[jira] [Commented] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142304#comment-15142304 ] Duo Zhang commented on HBASE-15252: --- [~stack] Oh yes, it is used for replication...

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Attachment: HBASE-15252-testcase.patch Attach a testcase to reproduce the bug. In this testcase I

[jira] [Comment Edited] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142216#comment-15142216 ] Duo Zhang edited comment on HBASE-15252 at 2/11/16 4:03 AM: Attach a testcase

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available)

[jira] [Commented] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142227#comment-15142227 ] Duo Zhang commented on HBASE-15252: --- Changing the exception type back to IPBE can solve the

[jira] [Resolved] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-15252. --- Resolution: Fixed Pushed the addendum to 0.98. > Data loss when replaying wal if HDFS timeout >

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Attachment: HBASE-15252-addendum-0.98.patch Fix compile error for 0.98 on hadoop-1.1 > Data loss when

[jira] [Reopened] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-15252: --- The patch for 0.98 breaks hadoop-1.1 compatibility. > Data loss when replaying wal if HDFS timeout >

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Parent Issue: HBASE-14790 (was: HBASE-14004) > Resolve name conflict when splitting if there are

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: HBASE-14949-v3.patch New patch solve the name conflict when split. > Resolve name

[jira] [Assigned] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-14949: - Assignee: Duo Zhang > Resolve name conflict when splitting if there are duplicated WAL entries

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Summary: Resolve name conflict when splitting if there are duplicated WAL entries (was: Skip

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-11 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Attachment: HBASE-15252.patch So let's fix this issue first. The readNext thing could be talked at

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-11 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Fix Version/s: 0.98.18 1.0.4 1.1.4

[jira] [Updated] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-11 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15252: -- Attachment: HBASE-15252-v1.patch New patch addressing the secure log reader issue. > Data loss when

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135501#comment-15135501 ] Duo Zhang commented on HBASE-14790: --- [~stack] Oh there is a WALPE tool, I didn't know it before, I have

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135485#comment-15135485 ] Duo Zhang commented on HBASE-14790: --- [~eclark] AFAIK, the ack of hflush only means that datanode has

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-05 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135606#comment-15135606 ] Duo Zhang commented on HBASE-14790: --- Fine. Let's do it in HBase. A little problem is that if we get an

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Description: The AsyncFSHLog introduced in HBASE-14790 may write same WAL entries to different WAL

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-01-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116786#comment-15116786 ] Duo Zhang commented on HBASE-14969: --- OK, Let's finish this... Will push to master and branch-1 later

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-02-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125915#comment-15125915 ] Duo Zhang commented on HBASE-14969: --- [~stack] Yes, {{testFlushThroughputTuning}} fails consistently

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-02-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125931#comment-15125931 ] Duo Zhang commented on HBASE-14969: --- The problem here is that {{hbase:meta}} is assigned to the RS so

[jira] [Updated] (HBASE-14969) Add throughput controller for flush

2016-02-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14969: -- Status: Patch Available (was: Reopened) > Add throughput controller for flush >

[jira] [Updated] (HBASE-14969) Add throughput controller for flush

2016-02-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14969: -- Attachment: HBASE-14969-addendum-branch-1.patch Patch for branch-1. On master, we have this

[jira] [Updated] (HBASE-14969) Add throughput controller for flush

2016-01-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14969: -- Affects Version/s: 1.3.0 Component/s: regionserver > Add throughput controller for flush >

[jira] [Updated] (HBASE-14969) Add throughput controller for flush

2016-01-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14969: -- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and branch-1.

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: (was: HBASE-15264-v8.patch) > Implement a fan out HDFS OutputStream >

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15162906#comment-15162906 ] Duo Zhang commented on HBASE-15264: --- Let me see what's wrong with the javadoc... > Implement a fan out

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15162923#comment-15162923 ] Duo Zhang commented on HBASE-15264: --- Oh the problem is the @link tag. If I remove it the warning will

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v8.patch > Implement a fan out HDFS OutputStream >

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v8.patch Retry. The bot seems broken. > Implement a fan out HDFS OutputStream

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: (was: HBASE-15264-v8.patch) > Implement a fan out HDFS OutputStream >

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-23 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158505#comment-15158505 ] Duo Zhang commented on HBASE-15264: --- Commit this on master is a bit simple for me since I haven't

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-23 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160060#comment-15160060 ] Duo Zhang commented on HBASE-15264: --- Sure, I will fix the javadoc issue. For checkstyle, I could fix

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-23 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v8.patch Fix the javadoc issue, but it will introduce a checkstyle issue(line

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-23 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v7.patch Add more comments. > Implement a fan out HDFS OutputStream >

<    1   2   3   4   5   6   7   8   9   10   >