[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987237#comment-14987237 ] Jeroen van Bemmel commented on HBASE-8927: -- A shift in system time ( e.g. ntpdate -u or date -s on Linux ) can cause a Zookeeper server to close all client connections due to session timeout; it becomes unavailable for several minutes until it recovers. Using System.nanoTime() (in the right places) can potentially make the system immune to such local time shifts, on platforms that support the monotonically increasing clock > Use nano time instead of mili time everywhere > - > > Key: HBASE-8927 > URL: https://issues.apache.org/jira/browse/HBASE-8927 > Project: HBase > Issue Type: Bug >Reporter: stack > Labels: phoenix > Attachments: 8927-WIP-TEST.txt, 8927.txt > > > Less collisions and we are paying the price of a long anyways so might as > well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544308#comment-14544308 ] Lars Hofhansl commented on HBASE-8927: -- Perhaps an easier config would be TIMESTAMP_RESOLUTION from a enum with MILLISECONDS, MICROSECONDS, NANOSECONDS. Default in 0.98, 1.x would be always MILLISECONDS. In 2.0 old tables default to MILLISECONDS, new tables could default to MICRO or NANOSECONDS. Note that we would not actually have this resolution, yet, but rather have space in the timestamp (by multiplying the current milliseconds by either 1000 or 100). Is that better? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544410#comment-14544410 ] Andrew Purtell commented on HBASE-8927: --- bq. MILLISECONDS, MICROSECONDS, NANOSECONDS Just to state this a bit differently, these are symbolic constants for timestamp multipliers. Sounds good to me. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544833#comment-14544833
]
Hadoop QA commented on HBASE-8927:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12733022/8927-WIP-TEST.txt
against master branch at commit 9ba7337ac82d13b22a1b0c40edaba7873c0bd795.
ATTACHMENT ID: 12733022
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1901 checkstyle errors (more than the master's current 1898 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ LOG.warn(\+TIMESTAMP_RESOLUTION+\ parameter cannot be updated!
Ignore., new Exception());
+ private static long determineTTLFromFamily(final HColumnDescriptor family,
final HTableDescriptor table) {
+
htd.setTimestampResolution(JInteger.valueOf(arg.delete(TIMESTAMP_RESOLUTION)))
if arg[TIMESTAMP_RESOLUTION]
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestDefaultMemStore
org.apache.hadoop.hbase.regionserver.TestReversibleScanners
org.apache.hadoop.hbase.regionserver.TestStoreScanner
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.ResourceCheckerJUnitListener.testFinished(ResourceCheckerJUnitListener.java:183)
at
org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingTTL.testScannerSelection(TestScannerSelectionUsingTTL.java:116)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/14052//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/14052//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/14052//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/14052//console
This message is automatically generated.
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Labels: phoenix
Attachments: 8927-WIP-TEST.txt, 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544868#comment-14544868 ] Lars Hofhansl commented on HBASE-8927: -- Also need to deal with per-cell TTL. (this would all so much easier if we could just change the EnvironmentEdge globally - but then we couldn't grandfather in any old tables) Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927-WIP-TEST.txt, 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350822#comment-14350822 ] stack commented on HBASE-8927: -- [~lhofhansl] There is no config by default if I read this right. New tables will get a non-1 multiplier. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350806#comment-14350806 ] Lars Hofhansl commented on HBASE-8927: -- Coming back to this... I think in order to make TTL work we need to be able to at least distinguish old and new tables and/or column families. I still think the easiest is to add TIMESTAMP_MULTIPLIER as a table or CF option. That way we: # can handle old tables (the multiplier would default to 1) # handle new table (there the multiplier could default to 1000 - for microsecs, or 100 - for nanosec) # allow folks to tweak their their time range against their resolution # (could even allow to set the multiplier to 0, indicating we're not dealing with time at all in the TS fields, and then we can disallow TTL) # when checking the TTL, we'd scale the TTL with this value, and it would do the right thing. Note that we would not actually have a higher resolution per se, we'd just make space in the KVs. We can either set the free bits to 0, or fill them with a random numbers, or do some nanotime trickery. It's a new option (sorry [~stack]), but I do not see another way out. Thoughts? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350820#comment-14350820 ] stack commented on HBASE-8927: -- I like it. Simple. [~lhofhansl] Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219860#comment-14219860 ] Lars Hofhansl commented on HBASE-8927: -- Should we pick this up again? bq. At a minimum, I think we should just left shift millis so we open up the lower bytes for external transactions, etc. to use, if we do nothing else on this issue. Agreed. We'd probably need to add a timestamp multiplier or timestamp shift option to table or column family. That way we can grandfather in old tables and in all cases scale the TTL value accordingly. I think this is the only concerns, the timestamps never actually have to changed in the existing/old KVs (even when stored in the WAL). Care would have to be taken when the multiplier is changed to higher value and there's a TTL on the table already, that would probably not work easily. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220262#comment-14220262 ] stack commented on HBASE-8927: -- bq. Should we pick this up again? I like this issue. Hard part is ensuring we don't break TTLs going between old and new ts types (as you pointed out above). Was thinking we'd have all timestamping go via the environmentedge thingy... would add a compare on it. It would do the ttl math cognizant of the ts typing (my guess is that left shift would be less expensive doing this compare than multiply but would have to measure). bq. We'd probably need to add a timestamp multiplier or timestamp shift option to table or column family. That way we can grandfather in old tables and in all cases scale the TTL value accordingly. All or nothing I'd say Just say NO to more options (smile). Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114482#comment-14114482 ] Gary Helmling commented on HBASE-8927: -- Regarding precision, FWIW in the case of Tephra, we multiply the current timestamp in milliseconds by 100 and add a counter (reset every millisecond). 1 billion operations per seconds is probably more than we need, so if we need to support more that 292 years worth of timestamped values, we could probably adjust to a lower counter range. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114506#comment-14114506 ] stack commented on HBASE-8927: -- [~ghelmling] Not a left-shift but a multiply? Above speculate left-shifting two bytes. You'd shift more Gary? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14108946#comment-14108946 ] Nicolas Liochon commented on HBASE-8927: It's part of the ts. Typically used when the user application sets the ts. With a composition of: ts + uniqueClientId + counter all the operations are ordered. In the client application it's not very difficult to have a unique id per client process and then to maintain a counter. I expect that playing with transactions leads to these kind of needs. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105225#comment-14105225 ] Nicolas Liochon commented on HBASE-8927: So I'm adding there the comment I wrote in a duplicate jira: Am I the only one who would appreciate some extra bits, to have a timestamps + a counter? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106152#comment-14106152 ] stack commented on HBASE-8927: -- bq. So I'm adding there the comment I wrote in a duplicate jira: Am I the only one who would appreciate some extra bits, to have a timestamps + a counter? Say more. How would the counter be used? It'd be part of the ts or a different facility altogether? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104230#comment-14104230 ] stack commented on HBASE-8927: -- Just read through this issue again. [~sershe] stuff is good pushback above. We are careful now allocating sequenceid assigning edits an order. Should we at the same time allocate the Cell ts or at least modify the LSBs on the ts to add in some derivative of the sequenceid? (Would only work if we've not put the edit into memstore yet). [~cuijianwei] Didn't you suggest a few lines of code assigning a more granular ts (IIRC)? Inside a synchronize you assigned the ts, left-shifted, and filled in lower bits with something? I can't find it just now... Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104238#comment-14104238 ] stack commented on HBASE-8927: -- Sorry [~cuijianwei] Its [~heliangliang] over in https://issues.apache.org/jira/browse/HBASE-2256?focusedCommentId=13825164page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13825164 Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104406#comment-14104406
]
Hadoop QA commented on HBASE-8927:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591798/8927.txt
against trunk revision .
ATTACHMENT ID: 12591798
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 402
new or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10507//console
This message is automatically generated.
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Labels: phoenix
Attachments: 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104950#comment-14104950 ] He Liangliang commented on HBASE-8927: -- [~stack] Yes, another benefit of left shifting + synchronized counter is it does not suffer from system time drift Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Labels: phoenix Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711385#comment-13711385 ] stack commented on HBASE-8927: -- So, left-shifting the timestamping opens up the two least significant bytes. We could make these two bytes be a counter inside the millisecond. We could give each edit that comes in during this millisecond its own unique counter. We should never ever after have clashes (hard to do 64k ops inside a millisecond). No-clashing edits would help the distributed log replay case. Here is max long: http://en.wikipedia.org/wiki/9223372036854775807 which in hex is 0x7FFF. If we left shift currentTimeMillis, we can run it up to Tue Oct 16 19:45:55 PDT 6429 before it'd rollover. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708257#comment-13708257 ] Enis Soztutar commented on HBASE-8927: -- bq. This was just to provide an example. Goal was to show that other applications are already using nano instead on milli, Cassandra's issue is about changing to nano for tracking latencies, vs. in this case, this is about using nano as a globally pseudo-consistent wall clock. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708429#comment-13708429 ] Jean-Marc Spaggiari commented on HBASE-8927: Hi [~enis], as I just replied to [~saint@gmail.com], you're right. This was just to provide an example. They have nanoTime() all over the place, not just on this patch. But they also still have some currentMs() calls... Might be interesting to ask them ;) Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708733#comment-13708733 ] Sergey Shelukhin commented on HBASE-8927: - As I said can we please look at reasons to do this, regardless of what this clock does... there's not a single good reason to use nano in most places, with the exception of column timestamp precision, for which a more complex and safer scheme can be made as suggested by some comments above, and measuring time taken for short (normally expected to take single seconds or less) operations. Now, we could do it everywhere anyway just because why not, if it were free of other consequences, but in this case we want to ignore explicitly missing guarantees on the interface and rely on internals of particular JVM(s) and OS(es). If some JVM, OS, or future changes to JVM code change the implementation (still staying within the interface guarantees), there will be a fail of epic proportions. In my mind there isn't even a reason to look at internals and ponder, simply because we don't stand to gain anything from nano in most places. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708861#comment-13708861 ] stack commented on HBASE-8927: -- bq. ...with the exception of column timestamp precision, for which a more complex and safer scheme can be made as suggested by some comments above, and measuring time taken for short (normally expected to take single seconds or less) operations. Seems like good enough reasons to me for doing nano time. I took a look at jdk8 and the story is no different there in its description of system.nanotime. It adds a java.time package with classes like Instant w/ its nanosecond resolution http://download.java.net/jdk8/docs/api/java/time/Instant.html but it doesn't look too amenable given its long of seconds and then nanos inside a second (apart from it being jdk8) Could do something like Lars suggests above w/ left shift filling in bottom few bytes w/ an incrementing number; it'd be a bit of a pain to implement but would be nice avoiding clashes. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709178#comment-13709178 ] Andrew Purtell commented on HBASE-8927: --- bq. Could do something like Lars suggests above w/ left shift filling in bottom few bytes w/ an incrementing number; it'd be a bit of a pain to implement but would be nice avoiding clashes. That is the middle ground as I read the comments above. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709184#comment-13709184 ] stack commented on HBASE-8927: -- At a minimum, I think we should just left shift millis so we open up the lower bytes for external transactions, etc. to use, if we do nothing else on this issue. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709210#comment-13709210 ] Sergey Shelukhin commented on HBASE-8927: - bq. Seems like good enough reasons to me for doing nano time. Yes, for these particular pieces of code, not everywhere... Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708042#comment-13708042
]
Ted Yu commented on HBASE-8927:
---
I noticed that the size of patch for CASSANDRA-733 is relatively small.
It contains changes in the following form:
{code}
-long startTime = System.currentTimeMillis();
+long startTime = System.nanoTime();
...
-writeStats.add(System.currentTimeMillis() - startTime);
+writeStats.addNano(System.nanoTime() - startTime);
{code}
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708053#comment-13708053 ] Jean-Marc Spaggiari commented on HBASE-8927: Exact. They simply changed currentTimeMillis() for nanoTime(). And I think that's the point of St.Ack's patch too. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708062#comment-13708062 ] Jean-Marc Spaggiari commented on HBASE-8927: Just to add: jmspaggiari@t:~/cassandra/cassandra$ grep -R System.nanoTime() * | wc 112 643 13913 Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708154#comment-13708154 ] stack commented on HBASE-8927: -- [~jmspaggi] That patch is about something else, tracking latencies in nanos, not kv timestamping. For this issue to progress, I need to see what happens in linux multithreaded app on smp to see if CLOCK_MONOTONIC means what I think it means. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708167#comment-13708167 ] Jean-Marc Spaggiari commented on HBASE-8927: This was just to provide an example. Goal was to show that other applications are already using nano instead on milli, so we might be able to do that too. Regarding Mononotic, documentation says: CLOCK_MONOTONIC Clock that cannot be set and represents monotonic time since some unspecified starting point. cannot be set... Seems that even NTPs can't modify that. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707916#comment-13707916 ] Jean-Marc Spaggiari commented on HBASE-8927: I'm not getting the points against this change. We will not be the first one to move from Milli to Nano, no? https://issues.apache.org/jira/browse/CASSANDRA-733 The issues we might have we nano, we already have them with milli? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705562#comment-13705562
]
Hadoop QA commented on HBASE-8927:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591798/8927.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 402
new or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6307//console
This message is automatically generated.
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705568#comment-13705568 ] Jieshan Bean commented on HBASE-8927: - I think System.nanoTime() can not be used as TimeStamp. It can't ensure the accuracy. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705574#comment-13705574 ] Lars Hofhansl commented on HBASE-8927: -- nanoTime cannot be used as absolute timestamps, it can only be used to compare times over a relatively small interval. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705605#comment-13705605 ] Enis Soztutar commented on HBASE-8927: -- Agreed what Lars said. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705619#comment-13705619 ] Lars Hofhansl commented on HBASE-8927: -- Maybe we can use millies and fill the lower bits from nano time. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706031#comment-13706031
]
stack commented on HBASE-8927:
--
Yeah on what javadoc says but src is quoted in this thread:
http://stackoverflow.com/questions/510462/is-system-nanotime-completely-useless
{code}
jlong os::javaTimeMillis() {
timeval time;
int status = gettimeofday(time, NULL);
assert(status != -1, linux error);
return jlong(time.tv_sec) * 1000 + jlong(time.tv_usec / 1000);
}
jlong os::javaTimeNanos() {
if (Linux::supports_monotonic_clock()) {
struct timespec tp;
int status = Linux::clock_gettime(CLOCK_MONOTONIC, tp);
assert(status == 0, gettime error);
jlong result = jlong(tp.tv_sec) * (1000 * 1000 * 1000) + jlong(tp.tv_nsec);
return result;
} else {
timeval time;
int status = gettimeofday(time, NULL);
assert(status != -1, linux error);
jlong usecs = jlong(time.tv_sec) * (1000 * 1000) + jlong(time.tv_usec);
return 1000 * usecs;
}
}
{code}
The above looks pretty good to me.
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706117#comment-13706117 ] Enis Soztutar commented on HBASE-8927: -- bq. Maybe we can use millies and fill the lower bits from nano time. I did exactly that in HBASE-6833. The problem is that each CPU thread will observe it's own nanosecond increments. Thus, concurrent updates coming at the same time won't have any guarantees for having monotonically increasing ts's because they are handled by different hardware threads. We can add some synchronization there (see my patch there), but it is not clear whether using ns will gain us if we did that. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706136#comment-13706136 ] stack commented on HBASE-8927: -- We could left shift millis in the long and then keep incrementing sequence number within a millisecond? (more expensive than call to nano). On CLOCK_MONOTONIC from http://linux.die.net/man/3/clock_gettime, it says Clock that cannot be set and represents monotonic time since some unspecified starting point. which is off-putting but when I print it out it is same as millis. Did you see it going backward or out of order on SMP Enis (in spite of CLOCK_MONOTONIC)? Were you on windows? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706172#comment-13706172 ] Sergey Shelukhin commented on HBASE-8927: - What problem are we trying to solve? The fix with synchronized increments/etc. is not cheap and the only place we /really/ need it as timestamps, it seems. Just checking Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706212#comment-13706212 ] stack commented on HBASE-8927: -- + Less cordinate collisions (where coordinates are row+cf+qualifier+type+ts). As slow as we are we can do a bunch of ops inside a ms. If nanotime, can establish some order regards events that arrive inside the same ms. + We have a 64bit ts already but we don't use all the bytes; that seems a little silly. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706242#comment-13706242 ] Enis Soztutar commented on HBASE-8927: -- bq. Did you see it going backward or out of order on SMP Enis That was on windows, but I believe it should be the same in linux as well. The hardware clocks themselves do not go back, but if two updates come with seq_num X and seq_num X+1, then X+1 might get a smaller ns because the hardware thread might observe a different clock than the other thread. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706274#comment-13706274 ] Elliott Clark commented on HBASE-8927: -- bq.The hardware clocks themselves do not go back, but if two updates come with seq_num X and seq_num X+1, then X+1 might get a smaller ns because the hardware thread might observe a different clock than the other thread. That can happen now if the mutations are not on the same row: * [Thread 1] Mutation A comes in to thread * [Thread 1] Wal edit A gets seq id. * [Thread 1] Thread Context Switches * [Thread 2] Mutation B comes in * [Thread 2] Wal edit B gets seq id. * [Thread 2] Mutation B gets timestamp. * [Thread 1] Wakes up * [Thread 1] Mutation A gets time stamp. A.id B.id A.ts B.ts Locks ensure this isn't an issue on the same row. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706325#comment-13706325 ] Enis Soztutar commented on HBASE-8927: -- bq. That can happen now if the mutations are not on the same row: That is true, but I've never seen that happen in reality. Versus in ns case, it will happen very frequently. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706444#comment-13706444 ] Sergey Shelukhin commented on HBASE-8927: - [~stack] agree on first, but for general use (like e.g. measuring ttl expiration, how long things took for normal ops, and such) having a complex ts generation seems sillier ;) I'd do nano on case by case basis for non-timestamps. [~enis] hardware clocks do go back on faulty motherboards... also Windows does adjust the clock backwards from the time server if it drifts forwards. Linux ntpd can also do that unless explicitly disabled. [~eclark] but does it matter across rows? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706455#comment-13706455 ] stack commented on HBASE-8927: -- bq. ...a complex ts generation seems sillier It is no more complex than what we currently have? We'd do System.nanoTime instead of System.currentTimeMillis. That is all? Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706466#comment-13706466 ] Elliott Clark commented on HBASE-8927: -- bq.Elliott Clark but does it matter across rows? We make acid guarantees around rows. Not around different mutations on different rows. So yes. Showing that the clock always goes forward (minus errors) on a row means that we are only discussing changing behavior or timestamps that we gave no promises for. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[
https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706503#comment-13706503
]
Sergey Shelukhin commented on HBASE-8927:
-
[~stack] nanoTime is not the time... it cannot be used between machines (i.e.
for ttl and such):
{quote}
This method can only be used to measure elapsed time and is not related to any
other notion of system or wall-clock time. The value returned represents
nanoseconds since some fixed but arbitrary time (perhaps in the future, so
values may be negative). This method provides nanosecond precision, but not
necessarily nanosecond accuracy. No guarantees are made about how frequently
values change. Differences in successive calls that span greater than
approximately 292 years (263 nanoseconds) will not accurately compute elapsed
time due to numerical overflow. {quote}
Use nano time instead of mili time everywhere
-
Key: HBASE-8927
URL: https://issues.apache.org/jira/browse/HBASE-8927
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 8927.txt
Less collisions and we are paying the price of a long anyways so might as
well fill it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706530#comment-13706530 ] stack commented on HBASE-8927: -- Yeah. I've read that whiney-pants CYA bit of years-old javadoc. The implementation though looks like it should give us the behavior we want (and x-machines). I'd say next up is trying this on a few SMP machines. Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8927) Use nano time instead of mili time everywhere
[ https://issues.apache.org/jira/browse/HBASE-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706541#comment-13706541 ] Sergey Shelukhin commented on HBASE-8927: - Depending on implementation against explicit interface guarantee (or lack thereof) is not a good design decision imho Use nano time instead of mili time everywhere - Key: HBASE-8927 URL: https://issues.apache.org/jira/browse/HBASE-8927 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8927.txt Less collisions and we are paying the price of a long anyways so might as well fill it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
