[jira] [Comment Edited] (HBASE-14070) Hybrid Logical Clocks for HBase
[ https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142478#comment-16142478 ] Amit Patel edited comment on HBASE-14070 at 8/26/17 12:15 AM: -- Just as a status update, I'll be finishing my internship so I will be much less active in the future (I still hope to maintain some degree of involvement with the work). I am extremely grateful for the help and mentorship that I have received from the community. At the moment there are still a few outstanding issues that should be addressed before merging the work into master: * Performance of HLC (Clock#now, Clock#update are expensive, not as big of an issue if HLC is only used for meta but certainly is for user tables) * Huge number of tests timing out in in the pre-commit/[public branch|https://builds.apache.org/job/HBASE-14070.HLC/] builds (but haven't been able to replicate the test time-out and they pass just fine if ran locally) In terms of enabling HLC on user tables, remaining issues would include: * Some tests that are still explicitly setting the timestamp and need to be converted to instead manipulate the current time via mocking the clocks * Bulk load does not update the HLC Some additional brainstorming/discussions includes: * [Deprecate setting of timestamp in client for HLC|https://issues.apache.org/jira/browse/HBASE-18642] * [Add 'Transaction ID' to Result for HLC|https://issues.apache.org/jira/browse/HBASE-18643] was (Author: amit.patel): Just as a status update, I'll be finishing my internship so I will be much less active in the future. I am extremely grateful for the help and support that I have received from the community. At the moment there are still a few outstanding issues that should be addressed before merging the work into master: * Performance of HLC (Clock#now, Clock#update are expensive, not as big of an issue if HLC is only used for meta but certainly is for user tables) * Huge number of tests timing out in in the pre-commit/[public branch|https://builds.apache.org/job/HBASE-14070.HLC/] builds (but haven't been able to replicate the test time-out and they pass just fine if ran locally) In terms of enabling HLC on user tables, remaining issues would include: * Some tests that are still explicitly setting the timestamp and need to be converted to instead manipulate the current time via mocking the clocks * Bulk load does not update the HLC Some additional brainstorming/discussions includes: * [Deprecate setting of timestamp in client for HLC|https://issues.apache.org/jira/browse/HBASE-18642] * [Add 'Transaction ID' to Result for HLC|https://issues.apache.org/jira/browse/HBASE-18643] > Hybrid Logical Clocks for HBase > --- > > Key: HBASE-14070 > URL: https://issues.apache.org/jira/browse/HBASE-14070 > Project: HBase > Issue Type: New Feature >Reporter: Enis Soztutar >Assignee: Amit Patel > Attachments: HBASE-14070.master.001.patch, > HybridLogicalClocksforHBaseandPhoenix.docx, > HybridLogicalClocksforHBaseandPhoenix.pdf > > > HBase and Phoenix uses systems physical clock (PT) to give timestamps to > events (read and writes). This works mostly when the system clock is strictly > monotonically increasing and there is no cross-dependency between servers > clocks. However we know that leap seconds, general clock skew and clock drift > are in fact real. > This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of > hybrid physical clock + a logical clock. HLC is best of both worlds where it > keeps causality relationship similar to logical clocks, but still is > compatible with NTP based physical system clock. HLC can be represented in > 64bits. > A design document is attached and also can be found here: > https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit# -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase
[ https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142478#comment-16142478 ] Amit Patel commented on HBASE-14070: Just as a status update, I'll be finishing my internship so I will be much less active in the future. I am extremely grateful for the help and support that I have received from the community. At the moment there are still a few outstanding issues that should be addressed before merging the work into master: * Performance of HLC (Clock#now, Clock#update are expensive, not as big of an issue if HLC is only used for meta but certainly is for user tables) * Huge number of tests timing out in in the pre-commit/[public branch|https://builds.apache.org/job/HBASE-14070.HLC/] builds (but haven't been able to replicate the test time-out and they pass just fine if ran locally) In terms of enabling HLC on user tables, remaining issues would include: * Some tests that are still explicitly setting the timestamp and need to be converted to instead manipulate the current time via mocking the clocks * Bulk load does not update the HLC Some additional brainstorming/discussions includes: * [Deprecate setting of timestamp in client for HLC|https://issues.apache.org/jira/browse/HBASE-18642] * [Add 'Transaction ID' to Result for HLC|https://issues.apache.org/jira/browse/HBASE-18643] > Hybrid Logical Clocks for HBase > --- > > Key: HBASE-14070 > URL: https://issues.apache.org/jira/browse/HBASE-14070 > Project: HBase > Issue Type: New Feature >Reporter: Enis Soztutar >Assignee: Amit Patel > Attachments: HBASE-14070.master.001.patch, > HybridLogicalClocksforHBaseandPhoenix.docx, > HybridLogicalClocksforHBaseandPhoenix.pdf > > > HBase and Phoenix uses systems physical clock (PT) to give timestamps to > events (read and writes). This works mostly when the system clock is strictly > monotonically increasing and there is no cross-dependency between servers > clocks. However we know that leap seconds, general clock skew and clock drift > are in fact real. > This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of > hybrid physical clock + a logical clock. HLC is best of both worlds where it > keeps causality relationship similar to logical clocks, but still is > compatible with NTP based physical system clock. HLC can be represented in > 64bits. > A design document is attached and also can be found here: > https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit# -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18686) Improve HLC implementation performance
[ https://issues.apache.org/jira/browse/HBASE-18686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18686: --- Description: Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the HybridLogicalClock#now and HybridLogicalClock#update methods are far too costly in multithreaded scenarios especially due to synchronization around the entire methods. Thus we need an improved implementation of Clock#now and Clock#update such that it would perform better in the presence of multiple threads. was: Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods. Thus we need an improved implementation of Clock#now and Clock#update such that it would perform better in the presence of multiple threads. > Improve HLC implementation performance > -- > > Key: HBASE-18686 > URL: https://issues.apache.org/jira/browse/HBASE-18686 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Priority: Critical > > Based on [performance benchmarks on the > clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], > the HybridLogicalClock#now and HybridLogicalClock#update methods are far too > costly in multithreaded scenarios especially due to synchronization around > the entire methods. > Thus we need an improved implementation of Clock#now and Clock#update such > that it would perform better in the presence of multiple threads. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18686) Improve HLC implementation performance
[ https://issues.apache.org/jira/browse/HBASE-18686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18686: --- Description: Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods. Thus we need an improved implementation of Clock#now and Clock#update such that it would perform better in the presence of multiple threads. was:Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods. Thus we need an improved implementation of Clock#now and Clock#update such that it would perform better in the presence of multiple threads. > Improve HLC implementation performance > -- > > Key: HBASE-18686 > URL: https://issues.apache.org/jira/browse/HBASE-18686 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Priority: Critical > > Based on [performance benchmarks on the > clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], > the Clock#now and Clock#update methods of the HLC implementation are costly > in multithreaded scenarios especially due to synchronization around the > entire methods. > Thus we need an improved implementation of Clock#now and Clock#update such > that it would perform better in the presence of multiple threads. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18686) Improve HLC implementation performance
[ https://issues.apache.org/jira/browse/HBASE-18686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18686: --- Description: Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods. Thus we need an improved implementation of Clock#now and Clock#update such that it would perform better in the presence of multiple threads. (was: Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods.) > Improve HLC implementation performance > -- > > Key: HBASE-18686 > URL: https://issues.apache.org/jira/browse/HBASE-18686 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Priority: Critical > > Based on [performance benchmarks on the > clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], > the Clock#now and Clock#update methods of the HLC implementation are costly > in multithreaded scenarios especially due to synchronization around the > entire methods. Thus we need an improved implementation of Clock#now and > Clock#update such that it would perform better in the presence of multiple > threads. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18686) Improve HLC implementation performance
[ https://issues.apache.org/jira/browse/HBASE-18686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18686: --- Priority: Critical (was: Blocker) > Improve HLC implementation performance > -- > > Key: HBASE-18686 > URL: https://issues.apache.org/jira/browse/HBASE-18686 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Priority: Critical > > Based on [performance benchmarks on the > clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], > the Clock#now and Clock#update methods of the HLC implementation are costly > in multithreaded scenarios especially due to synchronization around the > entire methods. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18686) Improve HLC implementation performance
Amit Patel created HBASE-18686: -- Summary: Improve HLC implementation performance Key: HBASE-18686 URL: https://issues.apache.org/jira/browse/HBASE-18686 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Priority: Blocker Based on [performance benchmarks on the clocks|https://issues.apache.org/jira/browse/HBASE-18542?focusedCommentId=16141932=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16141932], the Clock#now and Clock#update methods of the HLC implementation are costly in multithreaded scenarios especially due to synchronization around the entire methods. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141932#comment-16141932 ] Amit Patel edited comment on HBASE-18542 at 8/25/17 5:54 PM: - I redid the benchmarks using JMH as suggested [~enis] (thanks for the examples!). I ran tests on Clock#now and Clock#update for each clock implementation with 1, 4, and then 16 threads. The hybrid logical clock implementation doesn't look too good and does not scale with additional threads compared to the system and system monotonic clock implementation. The main culprit is the use of the synchronized blocks in the HLC's Clock#now and Clock#update so these results would represent the worst case. Results: https://docs.google.com/spreadsheets/d/1ndAVSwQ4yBqqu28_9eREg9qqDrpcXVt7TfTJOQWUq8Q/edit?usp=sharing Overall, there is still work that needs to be done to improve performance for the hybrid logical clock implementation. was (Author: amit.patel): I redid the benchmarks using JMH as suggested [~enis] (thanks for the examples!). I ran tests on Clock#now and Clock#update for each clock implementation with 1, 4, and then 16 threads. The hybrid logical clock implementation doesn't look too good and does not scale with additional threads compared to the system and system monotonic clock implementation. The main culprit is the use of the synchronized blocks in the HLC's Clock#now and Clock#update so these results would represent the worst case. https://docs.google.com/spreadsheets/d/1ndAVSwQ4yBqqu28_9eREg9qqDrpcXVt7TfTJOQWUq8Q/edit?usp=sharing Overall, there is still work that needs to be done to improve performance for the hybrid logical clock implementation. > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141932#comment-16141932 ] Amit Patel commented on HBASE-18542: I redid the benchmarks using JMH as suggested [~enis] (thanks for the examples!). I ran tests on Clock#now and Clock#update for each clock implementation with 1, 4, and then 16 threads. The hybrid logical clock implementation doesn't look too good and does not scale with additional threads compared to the system and system monotonic clock implementation. The main culprit is the use of the synchronized blocks in the HLC's Clock#now and Clock#update so these results would represent the worst case. https://docs.google.com/spreadsheets/d/1ndAVSwQ4yBqqu28_9eREg9qqDrpcXVt7TfTJOQWUq8Q/edit?usp=sharing Overall, there is still work that needs to be done to improve performance for the hybrid logical clock implementation. > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139209#comment-16139209
]
Amit Patel edited comment on HBASE-18509 at 8/23/17 10:58 PM:
--
The refactoring messed up some of the tests that were mocking the clocks. I've
submitted a patch that fixes this in the meantime.
[^HBASE-18509.HBASE-14070.HLC.005.patch]
was (Author: amit.patel):
The refactoring messed up some of the tests that were mocking the clocks. I've
submitted a patch that fixes this in the meantime.
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch, HBASE-18509.HBASE-14070.HLC.005.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - (done in 004.patch) --Maybe encapsulate all clocks in RS/Master into a new
> class. Then RSServices can just have getClocks() function.--
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18668) [HLC] Fix TestCellACLWithMultipleVersions timing out
[ https://issues.apache.org/jira/browse/HBASE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18668: --- Summary: [HLC] Fix TestCellACLWithMultipleVersions timing out (was: Fix TestCellACLWithMultipleVersions timing out) > [HLC] Fix TestCellACLWithMultipleVersions timing out > > > Key: HBASE-18668 > URL: https://issues.apache.org/jira/browse/HBASE-18668 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Attachments: HBASE-18668.HBASE-14070.HLC.001.patch > > > The patch submitted > [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] > in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by > making some changes in AccessController. This test was timing out but due to > a different issue than the other (thousands) of tests that are timing out so > it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18668) Fix TestCellACLWithMultipleVersions timing out
[ https://issues.apache.org/jira/browse/HBASE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18668: --- Status: Patch Available (was: In Progress) > Fix TestCellACLWithMultipleVersions timing out > -- > > Key: HBASE-18668 > URL: https://issues.apache.org/jira/browse/HBASE-18668 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Attachments: HBASE-18668.HBASE-14070.HLC.001.patch > > > The patch submitted > [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] > in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by > making some changes in AccessController. This test was timing out but due to > a different issue than the other (thousands) of tests that are timing out so > it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18668) Fix TestCellACLWithMultipleVersions timing out
[ https://issues.apache.org/jira/browse/HBASE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18668 started by Amit Patel. -- > Fix TestCellACLWithMultipleVersions timing out > -- > > Key: HBASE-18668 > URL: https://issues.apache.org/jira/browse/HBASE-18668 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Attachments: HBASE-18668.HBASE-14070.HLC.001.patch > > > The patch submitted > [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] > in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by > making some changes in AccessController. This test was timing out but due to > a different issue than the other (thousands) of tests that are timing out so > it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18668) Fix TestCellACLWithMultipleVersions timing out
[ https://issues.apache.org/jira/browse/HBASE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18668: --- Attachment: HBASE-18668.HBASE-14070.HLC.001.patch > Fix TestCellACLWithMultipleVersions timing out > -- > > Key: HBASE-18668 > URL: https://issues.apache.org/jira/browse/HBASE-18668 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > Attachments: HBASE-18668.HBASE-14070.HLC.001.patch > > > The patch submitted > [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] > in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by > making some changes in AccessController. This test was timing out but due to > a different issue than the other (thousands) of tests that are timing out so > it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18668) Fix TestCellACLWithMultipleVersions timing out
[ https://issues.apache.org/jira/browse/HBASE-18668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18668: --- Summary: Fix TestCellACLWithMultipleVersions timing out (was: Fix TestCellACLWithMultipleVersions failing) > Fix TestCellACLWithMultipleVersions timing out > -- > > Key: HBASE-18668 > URL: https://issues.apache.org/jira/browse/HBASE-18668 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel > > The patch submitted > [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] > in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by > making some changes in AccessController. This test was timing out but due to > a different issue than the other (thousands) of tests that are timing out so > it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18668) Fix TestCellACLWithMultipleVersions failing
Amit Patel created HBASE-18668: -- Summary: Fix TestCellACLWithMultipleVersions failing Key: HBASE-18668 URL: https://issues.apache.org/jira/browse/HBASE-18668 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel The patch submitted [here|https://issues.apache.org/jira/browse/HBASE-18508?focusedCommentId=16133233=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16133233] in HBASE-18508 fixes TestCellACLWithMultipleVersions from timing out by making some changes in AccessController. This test was timing out but due to a different issue than the other (thousands) of tests that are timing out so it should be moved here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel reopened HBASE-18509:
The refactoring messed up some of the tests that were mocking the clocks. I've
submitted a patch that fixes this in the meantime.
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch, HBASE-18509.HBASE-14070.HLC.005.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - (done in 004.patch) --Maybe encapsulate all clocks in RS/Master into a new
> class. Then RSServices can just have getClocks() function.--
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18542: --- Status: Patch Available (was: In Progress) > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18661) [HLC] Move clocks to a separate hbase-clocks module
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18661 started by Amit Patel. -- > [HLC] Move clocks to a separate hbase-clocks module > --- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch > > > I think we can move all clocks related code out of h-common and h-server into > a new package which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency structure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18661) [HLC] Move clocks to a separate hbase-clocks module
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18661: --- Status: Patch Available (was: In Progress) > [HLC] Move clocks to a separate hbase-clocks module > --- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch > > > I think we can move all clocks related code out of h-common and h-server into > a new package which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency structure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18661) [HLC] Move clocks to a separate hbase-clocks module
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18661: --- Attachment: HBASE-18661.HBASE-14070.HLC.001.patch > [HLC] Move clocks to a separate hbase-clocks module > --- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch > > > I think we can move all clocks related code out of h-common and h-server into > a new package which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency structure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18661) [HLC] Move clocks to a separate hbase-clocks module
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel reassigned HBASE-18661: -- Assignee: Amit Patel (was: Appy) > [HLC] Move clocks to a separate hbase-clocks module > --- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > > I think we can move all clocks related code out of h-common and h-server into > a new package which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency structure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Attachment: HBASE-18509.HBASE-14070.HLC.005.patch
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch, HBASE-18509.HBASE-14070.HLC.005.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - (done in 004.patch) --Maybe encapsulate all clocks in RS/Master into a new
> class. Then RSServices can just have getClocks() function.--
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Status: Patch Available (was: In Progress)
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Attachment: HBASE-18509.HBASE-14070.HLC.004.patch
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137392#comment-16137392
]
Amit Patel commented on HBASE-18509:
Patch 3 [^HBASE-18509.HBASE-14070.HLC.003.patch] includes the new Clocks class
(forgot to add it in the patch before)
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Attachment: HBASE-18509.HBASE-14070.HLC.003.patch
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137300#comment-16137300
]
Amit Patel commented on HBASE-18509:
Patch 2 [^HBASE-18509.HBASE-14070.HLC.002.patch] takes care of encapsulating
all clocks in RS/Master into a new class (Clocks).
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Attachment: HBASE-18509.HBASE-14070.HLC.002.patch
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - Move to separate module/package/both
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137130#comment-16137130 ] Amit Patel commented on HBASE-18508: Also retry with [^HBASE-18508.HBASE-14070.HLC.006.patch] (which is the exact same as [^HBASE-18508.HBASE-14070.HLC.002.patch]) to verify that TestCellACLWithMultipleVersions passes. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch, HBASE-18508.HBASE-14070.HLC.005.patch, > HBASE-18508.HBASE-14070.HLC.006.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.006.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch, HBASE-18508.HBASE-14070.HLC.005.patch, > HBASE-18508.HBASE-14070.HLC.006.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137070#comment-16137070 ] Amit Patel commented on HBASE-18508: Retry with empty patch [^HBASE-18508.HBASE-14070.HLC.005.patch] after possible fixes to test timeout was pulled into the HBASE-14070.HLC from master > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch, HBASE-18508.HBASE-14070.HLC.005.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.005.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch, HBASE-18508.HBASE-14070.HLC.005.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136211#comment-16136211 ] Amit Patel edited comment on HBASE-18542 at 8/22/17 3:29 AM: - Added patch [^HBASE-18542.HBASE-14070.HLC.001.patch] that benchmarks performance of Clock#now and Clock#update under both single threaded and multi threaded cases. Below is a summary of the performance (where the number is the multiplicative factor of how much longer it took compared to the system clock). ||System clock||System monotonic||Hybrid logical clock|| |Clock#now single threaded|1.01|1.41| |Clock#update single threaded|1.01|1.36| |Clock#now multi threaded|1.23|1.76| |Clock#update multi threaded|2.28|2.20| was (Author: amit.patel): Added patch [^HBASE-18542.HBASE-14070.HLC.001.patch] that benchmarks performance of Clock#now and Clock#update under both single threaded and multi threaded cases. Below is a summary of the performance (where the number is the multiplicative factor of how much longer it took compared to the system clock). ||System clock||System monotonic|Hybrid logical clock|| |Clock#now single threaded|1.01|1.41| |Clock#update single threaded|1.01|1.36| |Clock#now multi threaded|1.23|1.76| |Clock#update multi threaded|2.28|2.20| > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136211#comment-16136211 ] Amit Patel commented on HBASE-18542: Added patch [^HBASE-18542.HBASE-14070.HLC.001.patch] that benchmarks performance of Clock#now and Clock#update under both single threaded and multi threaded cases. Below is a summary of the performance (where the number is the multiplicative factor of how much longer it took compared to the system clock). ||System clock||System monotonic|Hybrid logical clock|| |Clock#now single threaded|1.01|1.41| |Clock#update single threaded|1.01|1.36| |Clock#now multi threaded|1.23|1.76| |Clock#update multi threaded|2.28|2.20| > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18542: --- Attachment: HBASE-18542.HBASE-14070.HLC.001.patch > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18542.HBASE-14070.HLC.001.patch > > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18542 started by Amit Patel. -- > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel reassigned HBASE-18542: -- Assignee: Amit Patel > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135845#comment-16135845 ] Amit Patel commented on HBASE-18508: Retry, wrong patch name... [^HBASE-18508.HBASE-14070.HLC.004.patch] > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.004.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.004.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135607#comment-16135607 ] Amit Patel edited comment on HBASE-18508 at 8/21/17 8:26 PM: - Retry with empty patch [^0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch] was (Author: amit.patel): Retry with empty patch [^Empty_patch1.patch] > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: (was: Empty_patch1.patch) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: > 0001-HBASE-18508-empty-patch-added-new-line-in-TestClockW.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135607#comment-16135607 ] Amit Patel edited comment on HBASE-18508 at 8/21/17 8:19 PM: - Retry with empty patch [^Empty_patch1.patch] was (Author: amit.patel): Retry [^HBASE-18508.HBASE-14070.HLC.004.patch] > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: Empty_patch1.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: Empty_patch1.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: Empty_patch1.patch, > HBASE-18508.HBASE-14070.HLC.001.patch, HBASE-18508.HBASE-14070.HLC.002.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: (was: HBASE-18508.HBASE-14070.HLC.004.patch) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135607#comment-16135607 ] Amit Patel commented on HBASE-18508: Retry > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.004.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135607#comment-16135607 ] Amit Patel edited comment on HBASE-18508 at 8/21/17 6:58 PM: - Retry [^HBASE-18508.HBASE-14070.HLC.004.patch] was (Author: amit.patel): Retry > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.004.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.004.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch, > HBASE-18508.HBASE-14070.HLC.003.patch, HBASE-18508.HBASE-14070.HLC.004.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18643) Add 'Transaction ID' to Result for HLC
[ https://issues.apache.org/jira/browse/HBASE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18643: --- Description: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. This change would be similar to the way [Couchbase uses HLC|https://developer.couchbase.com/documentation/server/current/xdcr/xdcr-timestamp-based-conflict-resolution.html#timestamp-based-conflict-resolution__hybrid-logical-clock] where the timestamp of a mutation (denoted CAS) is stored in the result and sent back to the client. was: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. This change would be similar to the way Couchbase uses HLC where the timestamp of a mutation (denoted CAS) is stored in the result and sent back to the client. > Add 'Transaction ID' to Result for HLC > -- > > Key: HBASE-18643 > URL: https://issues.apache.org/jira/browse/HBASE-18643 > Project: HBase > Issue Type: Brainstorming >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > > In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and > [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss > using HLC and deprecating the client from being able to set timestamps for > mutations. However, one important use case for setting the timestamps (which > represent transaction IDs) is for transactions. What we could do is add a > transaction ID to the result object that is the hybrid logical timestamp of a > particular batch operation. The transaction ID should be considered opaque > and clients should not make assumptions for how the ID is changed. > This change would be similar to the way [Couchbase uses > HLC|https://developer.couchbase.com/documentation/server/current/xdcr/xdcr-timestamp-based-conflict-resolution.html#timestamp-based-conflict-resolution__hybrid-logical-clock] > where the timestamp of a mutation (denoted CAS) is stored in the result and > sent back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18643) Add 'Transaction ID' to Result for HLC
[ https://issues.apache.org/jira/browse/HBASE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18643: --- Description: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. This change would be similar to the way Couchbase uses HLC where the timestamp of a mutation (denoted CAS) is stored in the result and sent back to the client. was: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. > Add 'Transaction ID' to Result for HLC > -- > > Key: HBASE-18643 > URL: https://issues.apache.org/jira/browse/HBASE-18643 > Project: HBase > Issue Type: Brainstorming >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > > In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and > [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss > using HLC and deprecating the client from being able to set timestamps for > mutations. However, one important use case for setting the timestamps (which > represent transaction IDs) is for transactions. What we could do is add a > transaction ID to the result object that is the hybrid logical timestamp of a > particular batch operation. The transaction ID should be considered opaque > and clients should not make assumptions for how the ID is changed. > This change would be similar to the way Couchbase uses HLC where the > timestamp of a mutation (denoted CAS) is stored in the result and sent back > to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18642) Deprecate setting of timestamp in client for HLC
[ https://issues.apache.org/jira/browse/HBASE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18642: --- Description: With using [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070], clients should no longer set the timestamp for mutations (i.e., put/delete) because the server will do time stamping instead. Therefore we should look into deprecating setTimestamp as well as other methods that allow clients to set timestamps. (was: With using [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070], clients should no longer set the timestamp for mutations because the server will do time stamping instead. Therefore we should look into deprecating setTimestamp as well as other methods that allow clients to set timestamps. ) > Deprecate setting of timestamp in client for HLC > > > Key: HBASE-18642 > URL: https://issues.apache.org/jira/browse/HBASE-18642 > Project: HBase > Issue Type: Brainstorming >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > > With using [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070], > clients should no longer set the timestamp for mutations (i.e., put/delete) > because the server will do time stamping instead. Therefore we should look > into deprecating setTimestamp as well as other methods that allow clients to > set timestamps. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18643) Add 'Transaction ID' to Result for HLC
[ https://issues.apache.org/jira/browse/HBASE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18643: --- Description: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. was: In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642 title|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. > Add 'Transaction ID' to Result for HLC > -- > > Key: HBASE-18643 > URL: https://issues.apache.org/jira/browse/HBASE-18643 > Project: HBase > Issue Type: Brainstorming >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > > In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and > [HBASE-18642|https://issues.apache.org/jira/browse/HBASE-18642], we discuss > using HLC and deprecating the client from being able to set timestamps for > mutations. However, one important use case for setting the timestamps (which > represent transaction IDs) is for transactions. What we could do is add a > transaction ID to the result object that is the hybrid logical timestamp of a > particular batch operation. > The transaction ID should be considered opaque and clients should not make > assumptions for how the ID is changed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18643) Add 'Transaction ID' to Result for HLC
Amit Patel created HBASE-18643: -- Summary: Add 'Transaction ID' to Result for HLC Key: HBASE-18643 URL: https://issues.apache.org/jira/browse/HBASE-18643 Project: HBase Issue Type: Brainstorming Reporter: Amit Patel Assignee: Amit Patel Priority: Minor In [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070] and [HBASE-18642 title|https://issues.apache.org/jira/browse/HBASE-18642], we discuss using HLC and deprecating the client from being able to set timestamps for mutations. However, one important use case for setting the timestamps (which represent transaction IDs) is for transactions. What we could do is add a transaction ID to the result object that is the hybrid logical timestamp of a particular batch operation. The transaction ID should be considered opaque and clients should not make assumptions for how the ID is changed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18642) Deprecate setting of timestamp in client for HLC
Amit Patel created HBASE-18642: -- Summary: Deprecate setting of timestamp in client for HLC Key: HBASE-18642 URL: https://issues.apache.org/jira/browse/HBASE-18642 Project: HBase Issue Type: Brainstorming Reporter: Amit Patel Assignee: Amit Patel Priority: Minor With using [HBASE-14070|https://issues.apache.org/jira/browse/HBASE-14070], clients should no longer set the timestamp for mutations because the server will do time stamping instead. Therefore we should look into deprecating setTimestamp as well as other methods that allow clients to set timestamps. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133594#comment-16133594 ] Amit Patel edited comment on HBASE-18508 at 8/18/17 8:38 PM: - Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.003.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. was (Author: amit.patel): Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.001.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133594#comment-16133594 ] Amit Patel commented on HBASE-18508: Resubmitted same patch [^HBASE-18508.HBASE-14070.HLC.001.patch] after a fix to previously^^ failing test TestHeapSize#testSizes was pushed to the HBASE-14070.HLC branch. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.003.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch, HBASE-18508.HBASE-14070.HLC.003.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133233#comment-16133233 ] Amit Patel commented on HBASE-18508: Submitted another patch [^HBASE-18508.HBASE-14070.HLC.002.patch] that fixes TestCellACLWithMultipleVersions, hopefully this + rebasing the HBASE-14070.HLC branch gets us closer to fixing all the tests. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.002.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch, > HBASE-18508.HBASE-14070.HLC.002.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel updated HBASE-18509:
---
Attachment: HBASE-18509.HBASE-14070.HLC.001.patch
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch
>
>
> Track all types of cleanups here:
> - Rename classes to more consistent naming: SystemClock,
> SystemMonotonicClock, HLClock
> - Move implementations out from Clock interface. It's a simple interface of 6
> fns but very overloaded right now with everything put inside it.
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-18509 started by Amit Patel.
--
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch
>
>
> Track all types of cleanups here:
> - Rename classes to more consistent naming: SystemClock,
> SystemMonotonicClock, HLClock
> - Move implementations out from Clock interface. It's a simple interface of 6
> fns but very overloaded right now with everything put inside it.
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Patel reassigned HBASE-18509:
--
Assignee: Amit Patel
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
>
> Track all types of cleanups here:
> - Rename classes to more consistent naming: SystemClock,
> SystemMonotonicClock, HLClock
> - Move implementations out from Clock interface. It's a simple interface of 6
> fns but very overloaded right now with everything put inside it.
> - Maybe encapsulate all clocks in RS/Master into a new class. Then RSServices
> can just have getClocks() function.
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Patch Available (was: In Progress) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126519#comment-16126519 ] Amit Patel edited comment on HBASE-18508 at 8/14/17 10:34 PM: -- Added patch [^HBASE-18508.HBASE-14070.HLC.001.patch] that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. was (Author: amit.patel): Added patch that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.001.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Open (was: Patch Available) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: (was: HBASE-18508.HBASE-14070.HLC.001.patch) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Patch Available (was: In Progress) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18508 started by Amit Patel. -- > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126519#comment-16126519 ] Amit Patel commented on HBASE-18508: Added patch that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18508 started by Amit Patel. -- > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.001.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel reassigned HBASE-18508: -- Assignee: Amit Patel > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18539) Remove usages of masterSystemTime
[ https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18539: --- Attachment: HBASE-18539.HBASE-14070.HLC.003.patch > Remove usages of masterSystemTime > - > > Key: HBASE-18539 > URL: https://issues.apache.org/jira/browse/HBASE-18539 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18539.HBASE-14070.HLC.001.patch, > HBASE-18539.HBASE-14070.HLC.002.patch, HBASE-18539.HBASE-14070.HLC.003.patch > > > After removing the use of the master's system timestamp for meta updates in > [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that > we pass the timestamps during region open/close in > [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also > remove all legacy usages of masterSystemTime. > This was initially introduced in > [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the > master's current system time during an open region RPC to fix a case where > certain meta table updates could be eclipsed in very rare circumstances. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18539) Remove usages of masterSystemTime
[ https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18539: --- Description: After removing the use of the master's system timestamp for meta updates in [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that we pass the timestamps during region open/close in [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also remove all legacy usages of masterSystemTime. This was initially introduced in [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the master's current system time during an open region RPC to fix a case where certain meta table updates could be eclipsed in very rare circumstances. was: After removing the use of the master's system timestamp for meta updates in [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that we pass the timestamps during region open/close in [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also remove all legacy usages of masterSystemTime. This was initially introduced in [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the master's current system time during an open region RPC. > Remove usages of masterSystemTime > - > > Key: HBASE-18539 > URL: https://issues.apache.org/jira/browse/HBASE-18539 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18539.HBASE-14070.HLC.001.patch, > HBASE-18539.HBASE-14070.HLC.002.patch > > > After removing the use of the master's system timestamp for meta updates in > [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that > we pass the timestamps during region open/close in > [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also > remove all legacy usages of masterSystemTime. > This was initially introduced in > [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the > master's current system time during an open region RPC to fix a case where > certain meta table updates could be eclipsed in very rare circumstances. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18539) Remove usages of masterSystemTime
[ https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18539: --- Description: After removing the use of the master's system timestamp for meta updates in [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that we pass the timestamps during region open/close in [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also remove all legacy usages of masterSystemTime. This was initially introduced in [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the master's current system time during an open region RPC. was:After removing the use of the master's system timestamp for meta updates in [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that we pass the timestamps during region open/close in [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also remove all legacy usages of masterSystemTime. > Remove usages of masterSystemTime > - > > Key: HBASE-18539 > URL: https://issues.apache.org/jira/browse/HBASE-18539 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18539.HBASE-14070.HLC.001.patch, > HBASE-18539.HBASE-14070.HLC.002.patch > > > After removing the use of the master's system timestamp for meta updates in > [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that > we pass the timestamps during region open/close in > [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also > remove all legacy usages of masterSystemTime. > This was initially introduced in > [HBASE-13709|https://issues.apache.org/jira/browse/HBASE-13709] to send the > master's current system time during an open region RPC. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18539) Remove usages of masterSystemTime
[ https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18539: --- Attachment: HBASE-18539.HBASE-14070.HLC.002.patch > Remove usages of masterSystemTime > - > > Key: HBASE-18539 > URL: https://issues.apache.org/jira/browse/HBASE-18539 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18539.HBASE-14070.HLC.001.patch, > HBASE-18539.HBASE-14070.HLC.002.patch > > > After removing the use of the master's system timestamp for meta updates in > [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that > we pass the timestamps during region open/close in > [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also > remove all legacy usages of masterSystemTime. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18539) Remove usages of masterSystemTime
[ https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18539: --- Attachment: HBASE-18539.HBASE-14070.HLC.001.patch > Remove usages of masterSystemTime > - > > Key: HBASE-18539 > URL: https://issues.apache.org/jira/browse/HBASE-18539 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18539.HBASE-14070.HLC.001.patch > > > After removing the use of the master's system timestamp for meta updates in > [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that > we pass the timestamps during region open/close in > [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also > remove all legacy usages of masterSystemTime. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18539) Remove usages of masterSystemTime
Amit Patel created HBASE-18539: -- Summary: Remove usages of masterSystemTime Key: HBASE-18539 URL: https://issues.apache.org/jira/browse/HBASE-18539 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel Priority: Trivial After removing the use of the master's system timestamp for meta updates in [HBASE-18328|https://issues.apache.org/jira/browse/HBASE-18328] and now that we pass the timestamps during region open/close in [HBASE-18395|https://issues.apache.org/jira/browse/HBASE-18395], we can also remove all legacy usages of masterSystemTime. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18497: --- Attachment: HBASE-18497.HBASE-14070.HLC.003.patch > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Attachment: HBASE-18510.HBASE-14070.HLC.002.patch > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18497: --- Attachment: HBASE-18497.HBASE-14070.HLC.001.patch > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Attachment: HBASE-18510.HBASE-14070.HLC.001.patch > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18510) Update clock on replaying recovered edits
Amit Patel created HBASE-18510: -- Summary: Update clock on replaying recovered edits Key: HBASE-18510 URL: https://issues.apache.org/jira/browse/HBASE-18510 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel Priority: Minor This task covers updating the clock on recovery in HRegion#replayRecoveredEdits and includes region-level tests. Credit for the baseline of this work all goes to our Sai Teja Ranuva. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Description: This task covers updating the clock on recovery in HRegion#replayRecoveredEdits and includes region-level tests. Credit for the baseline of this work all goes to our [~saitejar]. was: This task covers updating the clock on recovery in HRegion#replayRecoveredEdits and includes region-level tests. Credit for the baseline of this work all goes to our Sai Teja Ranuva. > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18497: --- Attachment: (was: HBASE-18497.HBASE-14070.HLC.001.patch) > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.002.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18497: --- Attachment: HBASE-18497.HBASE-14070.HLC.002.patch > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18497: --- Attachment: HBASE-18497.HBASE-14070.HLC.001.patch > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18497) Add clock type to proto message for HLC region open/close
Amit Patel created HBASE-18497: -- Summary: Add clock type to proto message for HLC region open/close Key: HBASE-18497 URL: https://issues.apache.org/jira/browse/HBASE-18497 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel Priority: Minor This task covers the work for adding clock type to the protobuf message NodeTime used for sending timestamps of each clock during region open and close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18395: --- Attachment: HBASE-18395.HBASE-14070.HLC.006.patch > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch, HBASE-18395.HBASE-14070.HLC.005.patch, > HBASE-18395.HBASE-14070.HLC.006.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18395: --- Attachment: (was: HBASE-18395.HBASE-14070.HLC.006.patch) > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch, HBASE-18395.HBASE-14070.HLC.005.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18395: --- Attachment: HBASE-18395.HBASE-14070.HLC.006.patch > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch, HBASE-18395.HBASE-14070.HLC.005.patch, > HBASE-18395.HBASE-14070.HLC.006.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18496) Fix TestTimestampType failing on isLikelyOfType tests
[ https://issues.apache.org/jira/browse/HBASE-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18496: --- Attachment: HBASE-18496.HBASE-14070.HLC.001.patch > Fix TestTimestampType failing on isLikelyOfType tests > - > > Key: HBASE-18496 > URL: https://issues.apache.org/jira/browse/HBASE-18496 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18496.HBASE-14070.HLC.001.patch > > > The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were > failing after the initial commit of core HLC so this patch fixes that. Note > that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false > if the input timestamp is from before the first half of the first day of 1970. > Changes > * Offset date by second day of year > * Fixed instance of not properly setting ZonedDateTime in > TestTimestampType#testPhysicalIsLikelyOfType > * Changed timestamp for year 2016 in TimestampType to be UTC+0 instead of > UTC+8 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18496) Fix TestTimestampType failing on isLikelyOfType tests
[ https://issues.apache.org/jira/browse/HBASE-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18496: --- Description: The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were failing after the initial commit of core HLC so this patch fixes that. Note that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false if the input timestamp is from before the first half of the first day of 1970. Changes * Offset date by second day of year * Fixed instance of not properly setting ZonedDateTime in TestTimestampType#testPhysicalIsLikelyOfType * Changed timestamp for year 2016 in TimestampType to be UTC+0 instead of UTC+8 was:The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were failing after the initial commit of core HLC so this patch fixes that. Note that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false if the input timestamp is from before the first half of the first day of 1970. > Fix TestTimestampType failing on isLikelyOfType tests > - > > Key: HBASE-18496 > URL: https://issues.apache.org/jira/browse/HBASE-18496 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > > The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were > failing after the initial commit of core HLC so this patch fixes that. Note > that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false > if the input timestamp is from before the first half of the first day of 1970. > Changes > * Offset date by second day of year > * Fixed instance of not properly setting ZonedDateTime in > TestTimestampType#testPhysicalIsLikelyOfType > * Changed timestamp for year 2016 in TimestampType to be UTC+0 instead of > UTC+8 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18496) Fix TestTimestampType failing on isLikelyOfType tests
Amit Patel created HBASE-18496: -- Summary: Fix TestTimestampType failing on isLikelyOfType tests Key: HBASE-18496 URL: https://issues.apache.org/jira/browse/HBASE-18496 Project: HBase Issue Type: Sub-task Reporter: Amit Patel Assignee: Amit Patel Priority: Trivial The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were failing after the initial commit of core HLC so this patch fixes that. Note that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false if the input timestamp is from before the first half of the first day of 1970. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18395: --- Attachment: HBASE-18395.HBASE-14070.HLC.005.patch > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch, HBASE-18395.HBASE-14070.HLC.005.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18395: --- Attachment: HBASE-18395.HBASE-14070.HLC.004.patch > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100823#comment-16100823 ] Amit Patel edited comment on HBASE-18395 at 7/25/17 9:37 PM: - As of [^HBASE-18395.HBASE-14070.HLC.003.patch], it doesn't yet include proper mocking of the clocks in TestClockWithCluster and passing of the timestamp type in the proto messages (see [here|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16100734=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16100734] for details on the potential issue). TODO: The tests in TestClockWithCluster would ideally also advance (for example) the physical clock of the region server's HL clock by a fixed duration before region assignment/unassignmnet and then assert that the master server's clock was advanced by said fixed duration (and vice versa). TODO: implement updating clock on region open with highest timestamp found in region's files was (Author: amit.patel): As of [^HBASE-18395.HBASE-14070.HLC.003.patch], it doesn't yet include proper mocking of the clocks in TestClockWithCluster and passing of the timestamp type in the proto messages (see [here|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16100734=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16100734] for details on the potential issue). TODO: The tests in TestClockWithCluster would ideally also advance (for example) the physical clock of the region server's HL clock by a fixed duration before region assignment/unassignmnet and then assert that the master server's clock was advanced by said fixed duration (and vice versa). > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
