[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2018-04-03 Thread maobaolong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423806#comment-16423806
 ] 

maobaolong commented on HDFS-9319:
--

+1 This patch make sense. It look like the author [~jingzhao] does not update 
the new patch for a long time? Should other contributor take this JIRA.

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Major
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2018-03-22 Thread genericqa (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409945#comment-16409945
 ] 

genericqa commented on HDFS-9319:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  7s{color} 
| {color:red} HDFS-9319 does not apply to trunk. Rebase required? Wrong Branch? 
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-9319 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12769144/HDFS-9319.000.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23627/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Major
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2018-03-22 Thread Liu Junhong (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409922#comment-16409922
 ] 

Liu Junhong commented on HDFS-9319:
---

[~jingzhao] Is there any more plan for  this issue?

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Major
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2016-01-21 Thread Aaron T. Myers (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111789#comment-15111789
 ] 

Aaron T. Myers commented on HDFS-9319:
--

[~jingzhao] - do you plan on committing this soon? I see that [~cnauroth] +1'ed 
it a few months back.

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2016-01-21 Thread Jing Zhao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111796#comment-15111796
 ] 

Jing Zhao commented on HDFS-9319:
-

Yeah, originally I planned to first make DatanodeInfo thread safe then maybe 
DatanodeDescriptor. In this way we can break some global lock protection during 
block report process into finer granularity. Then later I find that the work on 
{{DatanodeDescriptor}} will take a lot more effort. Currently I'm not sure if 
to make {{DatanodeInfo}}/{{DatanodeDescriptor}} thread safe is the best way or 
necessary to achieve the final goal of HDFS-8966. Will spend more time 
exploring this.

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2016-01-21 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111906#comment-15111906
 ] 

Hadoop QA commented on HDFS-9319:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
23s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 30s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 27s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 16s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 27s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 17s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 23s 
{color} | {color:red} hadoop-hdfs-project: patch generated 24 new + 103 
unchanged - 34 fixed = 127 total (was 137) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 20s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 9s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s 
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK 
v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 14s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s 
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK 
v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 43s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
20s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 53s {color} 
| {color:black} {color} |
\\
\\

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2015-12-07 Thread Mingliang Liu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046242#comment-15046242
 ] 

Mingliang Liu commented on HDFS-9319:
-

+1 (non-binding)

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2015-11-03 Thread Jing Zhao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988148#comment-14988148
 ] 

Jing Zhao commented on HDFS-9319:
-

Thanks for the review, Chris! Totally agree we should also correctly handle 
data fields like {{volumeFailures}} and {{volumeFailureSummary}} in 
DatanodeDescriptor. But in this jira my original plan is to clean up 
DatanodeInfo, but handle DatanodeDescriptor in a separate jira which may still 
need some more work. So can we leave the changes you suggested there?

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9319) Make DatanodeInfo thread safe

2015-10-29 Thread Chris Nauroth (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981603#comment-14981603
 ] 

Chris Nauroth commented on HDFS-9319:
-

Hi [~jingzhao].  The patch looks good overall.  There is a small inconsistency 
in that {{volumeFailures}} and {{volumeFailureSummary}} are updated outside the 
lock, whereas all of the other heartbeat information is captured in 
{{DatanodeState}} and updated atomically inside the lock.  This is probably not 
a big deal in practice, but do you think it makes sense to expand the lock just 
a little to cover those 2 pieces of data?  This would guarantee that all state 
reflects the outcome of the last processed heartbeat.  For example:

{code}
synchronized(this) {
  this.volumeFailures = volFailures;
  this.volumeFailureSummary = volumeFailureSummary;
  // Update total metrics for the node.
  updateState(totalCapacity, totalDfsUsed, totalRemaining, 
totalBlockPoolUsed,
  cacheCapacity, cacheUsed, lastUpdate, lastUpdateMonotonic, 
xceiverCount);
}
{code}

Let me know your thoughts.  Thanks!

> Make DatanodeInfo thread safe
> -
>
> Key: HDFS-9319
> URL: https://issues.apache.org/jira/browse/HDFS-9319
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-9319.000.patch
>
>
> This jira plans to make DatanodeInfo's internal states independent of 
> external locks. Note because DatanodeInfo extends DatanodeID, we still need 
> to change DatanodeID as a follow-on work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)