[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462152#comment-13462152 ] Lars Hofhansl commented on HBASE-6833: -- @Ram: HBASE-6784. That's a badly written test. I wrote it :) I will fix it. No test should make any (implicit or explicit assumptions about the progression of time). [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.94.3, 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows Attachments: NanoSecondPrecisionEnvironmentEdge.java HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458439#comment-13458439 ] Elliott Clark commented on HBASE-6833: -- Would taking a currentTimeMillis and the nanoTime at class instantiation allow us to create an always increasing wall clock time. we would have a startNano and a startMilli return startMili + toMilli(startNano - System.nanoTime()); That has the potential to drift a little bit from wall time but should still be pretty close. And if we really need to stay close to wall time we can have a repeating timer that resets startMilli and startNano. [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458759#comment-13458759 ] Lars Hofhansl commented on HBASE-6833: -- Probably. I do not know how long nano time would be valid for comparison. (Presumably the problem is eventual wrap around, but I am just assuming). I'll ask some of the JVM guys we recently hired. [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458930#comment-13458930 ] ramkrishna.s.vasudevan commented on HBASE-6833: --- I recently raised an issue HBASE-6784 again a testcase failure just because the Puts got the same timestamp and the test was failing randomly. Based on the discussion over here i can provide a patch for that HBASe-6784. [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458949#comment-13458949 ] Enis Soztutar commented on HBASE-6833: -- Wow, it seems that great minds think alike :) I had actually done a ms + ns environment edge for experimentation. Let me attach it. [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458371#comment-13458371 ] stack commented on HBASE-6833: -- This one is a bit of a bummer. As you suggest elsewhere, in tests, could user EnvironmentEdge thing that updates each time its called but what happens when you run in production? You are going to get odd results. This does bring to the fore how much we presume a forward moving clock (Those spanner time servers with their aerials for GPS and atomic clock cards start to make a bit more sense now) [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458382#comment-13458382 ] Elliott Clark commented on HBASE-6833: -- Windows can have a more accurate normal system clock however it causes a lot of cpu load so is not always done: See: http://msdn.microsoft.com/en-us/library/dd757624%28VS.85%29.aspx Calling that works but it's frowned upon unless you really need timers that are 10 ms resolution. https://blogs.oracle.com/dholmes/entry/inside_the_hotspot_vm_clocks Seems to suggest that using the System.nanoTime will give the correct time as determined by QueryPerformanceFrequency. So our timing can use nanoTime converted to ms. This will only work on modernish hardware and modernish windows. However I think that's probably good enough. That still leaves the timers. Are there any cases where our timers must be 1ms accurate ? If it is then the above link seems to suggest that setting a timer that's not a multiple of 10ms will cause java to set the clock timer to 1 ms (though on older hardware I seem to remember that the api is ignored). [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458385#comment-13458385 ] stack commented on HBASE-6833: -- So a WindowsEnvironmentEdge backed by nano time? [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458392#comment-13458392 ] Elliott Clark commented on HBASE-6833: -- Yeah. nanoTime seems like it would work. Xp and earlier are the ones I remember having 15ms times; I don't remember the server versions as well since I never had to deal with them. So I'm kind of interested in what environment Enis is seeing 10ms+ [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6833) [WINDOWS] Java Milisecond precision on Windows
[ https://issues.apache.org/jira/browse/HBASE-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458419#comment-13458419 ] Lars Hofhansl commented on HBASE-6833: -- We need to keep in mind then that nanotime does not reflect any absolute time and is only supposed to be used for time-differences. As long we make sure it'll only ever used for that, we should be OK. I.e. we could not take nanotime, and use that for the persistent KV timestamps. (Going the documentation and what I have read elsewhere) [WINDOWS] Java Milisecond precision on Windows -- Key: HBASE-6833 URL: https://issues.apache.org/jira/browse/HBASE-6833 Project: HBase Issue Type: Sub-task Affects Versions: 0.96.0, 0.94.3 Reporter: Enis Soztutar Assignee: Enis Soztutar Labels: windows HBase relies on the system clock obtained by System.currentTimeMilis() to supply the version, if it is not provided by the client in Put's and Delete's. At HBASE-6832 and HBASE-6826, we discovered that on some platforms, the milis clock might not be updated every milisecond, which might cause unexpected behavior. We also did some preliminary analysis there. In this issue, we can further discuss and inspect whether it is a problem for main target platforms and if so what can be done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
