Hi Huaxiang,

Given that you already use this feature in production with no problems, I 
created a short patch to remove the “be careful” warnings from the HBase 
documentation.

PTAL if you agree.

https://github.com/apache/hbase/pull/4301

Thanks,
Andor




> On 2022. Mar 29., at 18:52, Huaxiang Sun <huaxiang...@apache.org> wrote:
> 
> This is great, thanks for the testing results!
> 
> Huaxiang
> 
> On 2022/03/29 13:29:48 Andor Molnar wrote:
>> Works!
>> 
>> I enabled async wal replication with the suggested option and ITBLL ran 
>> successfully.
>> 
>> generator step: 
>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList generator 15 
>> 1000000 /tmp/hbase-itbll
>> 
>> verification step:
>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList verify 
>> /tmp/hbase-itbll-verify 15
>> 
>> Both succeeded. I also confirmed that meta replicas are written and read by 
>> the clients, so must be in load balance mode.
>> 
>> Thanks for the help!
>> 
>> Andor
>> 
>> 
>> 
>>> On 2022. Mar 27., at 6:59, Huaxiang Sun <huaxiang...@apache.org> wrote:
>>> 
>>> It makes sense to turn on async wal replication when 
>>> hbase.meta.replicas.use = true. Let me run couple rounds of itbll with 
>>> hbase.region.replica.replication.catalog.enabled (lastest 2.4 and 2.5.0 
>>> candidates) to get more confidence before proposing turn on async wal 
>>> replication for meta.
>>> 
>>> Thanks,
>>> Huaxiang
>>> 
>>> On 2022/03/26 04:03:15 Andrew Purtell wrote:
>>>> Just to be clear when I say "it seems pointless to have meta replicas which
>>>> do not actually receive updates (by default)", what I should have said is
>>>> 'timely updates', because a long delay in updating meta might as well be a
>>>> missed update.
>>>> 
>>>> On Fri, Mar 25, 2022 at 9:01 PM Andrew Purtell <apurt...@apache.org> wrote:
>>>> 
>>>>>> "Async WAL replication for META is added as a new feature in 2.4.0. It
>>>>> is still under active development. Use with caution. Set
>>>>> hbase.region.replica.replication.catalog.enabled to enable async WAL
>>>>> Replication for META region replicas. It is off by default."
>>>>> 
>>>>> Do we still need this warning?
>>>>> 
>>>>> Should hbase.region.replica.replication.catalog.enabled have a default of
>>>>> 'true' (enabled) if hbase.meta.replicas.use = true ? Otherwise, it seems
>>>>> pointless to have meta replicas which do not actually receive updates (by
>>>>> default).
>>>>> 
>>>>> 
>>>>> On Fri, Mar 25, 2022 at 10:51 AM Huaxiang Sun <huaxiang...@apache.org>
>>>>> wrote:
>>>>> 
>>>>>> Hi Andor,
>>>>>> 
>>>>>>  I get what you are saying. The HFile refreshing is the old way for
>>>>>> replica regions to refresh hfiles periodically, default is 5 minutes. In
>>>>>> this itbll case, we need to have the wal replication enabled for meta
>>>>>> replica. Please check out,
>>>>>> 
>>>>>> https://hbase.apache.org/book.html#_async_wal_replication_for_meta_table_as_of_hbase_2_4_0.
>>>>>> Basically, you need to set
>>>>>> "hbase.region.replica.replication.catalog.enabled" to true in the
>>>>>> configuration and rerun itbll. Otherwise, all meta changes at the primary
>>>>>> meta region wont be updated at the replica meta regions and it will 
>>>>>> result
>>>>>> in itbll failures.
>>>>>> 
>>>>>>  Hope this helps,
>>>>>> 
>>>>>>  Huaxiang
>>>>>> 
>>>>>> 
>>>>>> On 2022/03/25 13:46:42 Andor Molnar wrote:
>>>>>>> Hi Huaxiang,
>>>>>>> 
>>>>>>> We use 2.4.6 for the tests.
>>>>>>> 
>>>>>>> I run itbll with the following command:
>>>>>>> 
>>>>>>> hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
>>>>>> generator 15 1000000 /tmp/hbase-itbll
>>>>>>> 
>>>>>>> for the generator step and essentially jobs have failed. We can see the
>>>>>> meta request are spanning out to replicas, but writes start failing after
>>>>>> this due to the stale cache which is not getting updated.
>>>>>>> 
>>>>>>> Would you please tell me more about ‘hfile refresh’ and how to
>>>>>> configure it?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Andor
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On 2022. Mar 24., at 17:43, Huaxiang Sun <huaxiang...@apache.org>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi Andor,
>>>>>>>> 
>>>>>>>> Which 2.4 release do you test in your lab? We use this feature at
>>>>>> production cluster with 2.4.5.
>>>>>>>> At server side, we use hfile refresh instead of wal replication. I
>>>>>> used to run itbll for each release with this feature enabled. How did you
>>>>>> find the errors, did itbll fail?
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Huaxiang
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best regards,
>>>>> Andrew
>>>>> 
>>>>> Unrest, ignorance distilled, nihilistic imbeciles -
>>>>>   It's what we’ve earned
>>>>> Welcome, apocalypse, what’s taken you so long?
>>>>> Bring us the fitting end that we’ve been counting on
>>>>>  - A23, Welcome, Apocalypse
>>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Best regards,
>>>> Andrew
>>>> 
>>>> Unrest, ignorance distilled, nihilistic imbeciles -
>>>>   It's what we’ve earned
>>>> Welcome, apocalypse, what’s taken you so long?
>>>> Bring us the fitting end that we’ve been counting on
>>>>  - A23, Welcome, Apocalypse
>>>> 
>> 
>> 

Reply via email to