[jira] [Commented] (HDDS-1365) Fix error handling in KeyValueContainerCheck
[ https://issues.apache.org/jira/browse/HDDS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808398#comment-16808398 ] Hudson commented on HDDS-1365: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16332 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16332/]) HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by (yqlin: rev f96fb05a2b889f3acdfae60d8f64c755ff48b8c1) * (edit) hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerCheck.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java > Fix error handling in KeyValueContainerCheck > > > Key: HDDS-1365 > URL: https://issues.apache.org/jira/browse/HDDS-1365 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Supratim Deka >Priority: Major > Fix For: 0.5.0 > > Attachments: HDDS-1365.000.patch > > > Error handling and propagation in KeyValueContainerCheck needs to be based on > throwing IOException instead of passing an error code to the calling function. > HDDS-1163 implemented the basic framework using a mix of error code return > and exception handling. There is added complexity because exceptions deep > inside the call chain are being caught and translated to error code return > values. The goal is to change all error handling in this class to use > Exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1365) Fix error handling in KeyValueContainerCheck
[ https://issues.apache.org/jira/browse/HDDS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808391#comment-16808391 ] Yiqun Lin commented on HDDS-1365: - Hi [~sdeka], bq. When that happens, we can introduce a specific new exception for every such specialised category with its own handler logic. I agree on this, maybe we can define a new exception similar to StorageContainerException. We can temporarily commit this change. +1. > Fix error handling in KeyValueContainerCheck > > > Key: HDDS-1365 > URL: https://issues.apache.org/jira/browse/HDDS-1365 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Supratim Deka >Priority: Major > Attachments: HDDS-1365.000.patch > > > Error handling and propagation in KeyValueContainerCheck needs to be based on > throwing IOException instead of passing an error code to the calling function. > HDDS-1163 implemented the basic framework using a mix of error code return > and exception handling. There is added complexity because exceptions deep > inside the call chain are being caught and translated to error code return > values. The goal is to change all error handling in this class to use > Exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1365) Fix error handling in KeyValueContainerCheck
[ https://issues.apache.org/jira/browse/HDDS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807897#comment-16807897 ] Supratim Deka commented on HDDS-1365: - hello [~linyiqun], I did consider losing the error code. Not every corruption will require specific handling. At best there will be a few categories. When that happens, we can introduce a specific new exception for every such specialised category with its own handler logic. Basically, moving away from the error code approach - using exceptions does make the code cleaner and less clunky. And it should be ok to defer this incremental work until actually required. Yes? > Fix error handling in KeyValueContainerCheck > > > Key: HDDS-1365 > URL: https://issues.apache.org/jira/browse/HDDS-1365 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Components: Ozone Datanode >Reporter: Supratim Deka >Assignee: Supratim Deka >Priority: Major > Attachments: HDDS-1365.000.patch > > > Error handling and propagation in KeyValueContainerCheck needs to be based on > throwing IOException instead of passing an error code to the calling function. > HDDS-1163 implemented the basic framework using a mix of error code return > and exception handling. There is added complexity because exceptions deep > inside the call chain are being caught and translated to error code return > values. The goal is to change all error handling in this class to use > Exceptions. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1365) Fix error handling in KeyValueContainerCheck
[
https://issues.apache.org/jira/browse/HDDS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807836#comment-16807836
]
Yiqun Lin commented on HDDS-1365:
-
Hi [~sdeka], the error code can be kept since we can do the corresponding
handle logic (e.g. notify SCM corrupted containers)according to error code
value in the future like following:
{code:java}
KvCheckError error;
switch(error) {
case FILE_LOAD:
// Handler logic1..
break;
case METADATA_PATH_ACCESS
// Handler logic2..
break;
case CHUNKS_PATH_ACCESS
// Handler logic3..
break;
...
default
break;
}
{code}
Based on the current change, the exception is almost swallowed. We only log the
exception message, how can we do the handle logic for different error cases?
Other refactor looks good to me. One nit:
{{assertTrue(corruption == false);}} can be simplified to
{{assertFalse(corruption);}}
> Fix error handling in KeyValueContainerCheck
>
>
> Key: HDDS-1365
> URL: https://issues.apache.org/jira/browse/HDDS-1365
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Supratim Deka
>Assignee: Supratim Deka
>Priority: Major
> Attachments: HDDS-1365.000.patch
>
>
> Error handling and propagation in KeyValueContainerCheck needs to be based on
> throwing IOException instead of passing an error code to the calling function.
> HDDS-1163 implemented the basic framework using a mix of error code return
> and exception handling. There is added complexity because exceptions deep
> inside the call chain are being caught and translated to error code return
> values. The goal is to change all error handling in this class to use
> Exceptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1365) Fix error handling in KeyValueContainerCheck
[
https://issues.apache.org/jira/browse/HDDS-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807444#comment-16807444
]
Hadoop QA commented on HDDS-1365:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 45s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
32s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
10s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 30s{color} | {color:orange} hadoop-hdds: The patch generated 2 new + 0
unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 2s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 35s{color}
| {color:red} hadoop-hdds in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 14s{color}
| {color:red} hadoop-ozone in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.container.common.TestDatanodeStateMachine |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HDDS-Build/2621/artifact/out/Dockerfile
|
| JIRA Issue | HDDS-1365 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12964521/HDDS-1365.000.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux d2f5d41268b9 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HDDS-Build/ozone.sh |
| git revision | trunk / 2f75283 |
| maven |
