[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-9902: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk,0.96 and 0.94. Thanks for patch Kashif. Thanks for review Stack and Ted. Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2-0.94.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Attachment: HBASE-9902_v2-0.94.patch Patch for 0.94 version Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2-0.94.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Attachment: (was: HBASE-9902_v2-0.94.patch) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Attachment: HBASE-9902_v2-0.94.patch Patch for 0.94 Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2-0.94.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Fix Version/s: 0.94.14 Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Status: Open (was: Patch Available) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Attachment: HBASE-9902_v2.patch Junit TC modified for different hostport for all regionservers test Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Status: Patch Available (was: Open) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1, 0.94.14 Attachments: HBASE-9902.patch, HBASE-9902_v2.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Fix Version/s: (was: 0.96.0) 0.96.1 Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Release Note: Clock skew detection to be made absolute value comparison. Any time difference between master or region, high or low must prevent the region server startup Status: Patch Available (was: Open) Clock skew detection to be made absolute value comparison. Any time difference between master or region, high or low must prevent the region server startup Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9902: --- Assignee: Kashif J S Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Assignee: Kashif J S Fix For: 0.98.0, 0.96.1 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Fix Version/s: 0.96.0 0.98.0 Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Fix For: 0.98.0, 0.96.0 When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9902) Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time
[ https://issues.apache.org/jira/browse/HBASE-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kashif J S updated HBASE-9902: -- Attachment: HBASE-9902.patch Patch for absolute value for clock skew detection. For 0.98.0 and 0.96.0 versions Region Server is starting normally even if clock skew is more than default 30 seconds(or any configured). - Regionserver node time is greater than master node time Key: HBASE-9902 URL: https://issues.apache.org/jira/browse/HBASE-9902 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.11 Reporter: Kashif J S Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9902.patch When Region server's time is ahead of Master's time and the difference is more than hbase.master.maxclockskew value, region server startup is not failing with ClockOutOfSyncException. This causes some abnormal behavior as detected by our Tests. ServerManager.java#checkClockSkew long skew = System.currentTimeMillis() - serverCurrentTime; if (skew maxSkew) { String message = Server + serverName + has been + rejected; Reported time is too far out of sync with master. + Time difference of + skew + ms max allowed of + maxSkew + ms; LOG.warn(message); throw new ClockOutOfSyncException(message); } Above line results in negative value when Master's time is lesser than region server time and if (skew maxSkew) check fails to find the skew in this case. Please Note: This was tested in hbase 0.94.11 version and the trunk also currently has the same logic. The fix for the same would be to make the skew positive value first as below: long skew = System.currentTimeMillis() - serverCurrentTime; skew = (skew 0 ? -skew : skew); if (skew maxSkew) {. -- This message was sent by Atlassian JIRA (v6.1#6144)