[
https://issues.apache.org/jira/browse/HDDS-1735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877382#comment-16877382
]
Anu Engineer edited comment on HDDS-1735 at 7/3/19 12:42 AM:
-------------------------------------------------------------
{code:java}
./hadoop-common-project/hadoop-auth/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-common/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-nfs/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-minikdc/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-kms/dev-support/findbugsExcludeFile.xml
./hadoop-hdds/common/target/findbugsExcludeFile.xml
./hadoop-hdds/common/dev-support/findbugsExcludeFile.xml
./hadoop-hdds/container-service/target/findbugsExcludeFile.xml
./hadoop-hdds/container-service/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-httpfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-client/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-nfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-rbf/dev-support/findbugsExcludeFile.xml{code}
what are these but suppressions ? Allow me to show you some more examples:
Here is some code from one these files:
{code:java}
<Match>
<Class
name="org.apache.hadoop.io.erasurecode.rawcoder.AbstractNativeRawEncoder" />
<Field name="nativeCoder" />
<Bug pattern="UUF_UNUSED_FIELD" />
</Match>
<Match>
<Class
name="org.apache.hadoop.io.erasurecode.rawcoder.AbstractNativeRawDecoder" />
<Field name="nativeCoder" />
<Bug pattern="UUF_UNUSED_FIELD" />
</Match>
<!--
Further SaslException should be ignored during cleanup and
original exception should be re-thrown.
-->
<Match>
<Class name="org.apache.hadoop.security.SaslRpcClient" />
<Bug pattern="DE_MIGHT_IGNORE" />
</Match>
<!--
Ignore Cross Scripting Vulnerabilities
-->
<Match>
<Package name="~org.apache.hadoop.mapred.*" />
<Bug code="XSS" />
</Match>
<Match>
<Class name="org.apache.hadoop.mapred.taskdetails_jsp" />
<Bug code="HRS" />
</Match>
{code}
This is code written by someone who made a judgement call, they saw the warning
message from findbugs and since they are experts on this domain made a
judgement call that they know better than a tool. Unless you have evidence to
prove that these lines are wrong; it would be better not to judge these
developers. They contribute work, time and talent to our cause; and you are
saying that grepping few lines over the code allows you to make a judgement
call and deem that this is inferior code? I have shown your claim that Hadoop
does not use suppressions – and the only one was contributed by [~elek] is
incorrect. The above lines prove that suppression is rampant in Hadoop code
base. Much more than Ozone, if you care to look.
What I don't understand is why have such blatant disregard and disrespect for
the community members? if you find a bug, file it as JIRA, please stop making
blanket statements and over-arching pronouncements on code quality. Let us be
specific; if you see an issue; we welcome it; and thank you for your making our
software better; the nature of software development is such that we will have
issues. But let us do it with some consideration; and not say things like I
grepped and found suppressions; therefore your code is bad. if you don't
understand that code; feel free to ask us; perhaps we are wrong; and you have
found a bug; perhaps we have a perfectly valid reason to do it that way.
Once more; I stand by my earlier statement; unless you are willing to measure
and show me numbers – I am going to presume that you have really no issues to
be solved.
Thank you for the conversation.
was (Author: anu):
{code:java}
./hadoop-common-project/hadoop-auth/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-common/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-nfs/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-minikdc/dev-support/findbugsExcludeFile.xml
./hadoop-common-project/hadoop-kms/dev-support/findbugsExcludeFile.xml
./hadoop-hdds/common/target/findbugsExcludeFile.xml
./hadoop-hdds/common/dev-support/findbugsExcludeFile.xml
./hadoop-hdds/container-service/target/findbugsExcludeFile.xml
./hadoop-hdds/container-service/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-httpfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-client/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-nfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml
./hadoop-hdfs-project/hadoop-hdfs-rbf/dev-support/findbugsExcludeFile.xml{code}
what are these but suppressions ? Allow me to show you some more examples:
Here is some code from one these files:
{code:java}
<Match>
<Class
name="org.apache.hadoop.io.erasurecode.rawcoder.AbstractNativeRawEncoder" />
<Field name="nativeCoder" />
<Bug pattern="UUF_UNUSED_FIELD" />
</Match>
<Match>
<Class
name="org.apache.hadoop.io.erasurecode.rawcoder.AbstractNativeRawDecoder" />
<Field name="nativeCoder" />
<Bug pattern="UUF_UNUSED_FIELD" />
</Match>
<!--
Further SaslException should be ignored during cleanup and
original exception should be re-thrown.
-->
<Match>
<Class name="org.apache.hadoop.security.SaslRpcClient" />
<Bug pattern="DE_MIGHT_IGNORE" />
</Match>
<!--
Ignore Cross Scripting Vulnerabilities
-->
<Match>
<Package name="~org.apache.hadoop.mapred.*" />
<Bug code="XSS" />
</Match>
<Match>
<Class name="org.apache.hadoop.mapred.taskdetails_jsp" />
<Bug code="HRS" />
</Match>
{code}
This is code written by someone who made a judgement call, they saw the warning
message from findbugs and since they are experts on this domain made a
judgement call that they know better than a tool. Unless you have evidence to
prove that these lines are wrong; it would be better not to judge these
developers. They contribute work, time and talent to our cause; and you are
saying that grepping few lines over the code allows you to make a judgement
call and deem that this is inferior code? I have shown your claim that Hadoop
does not use suppressions – and the only one was contributed by [~elek] is
incorrect. The above lines prove that suppression is rampant in Hadoop code
base. Much more than Ozone, if you care to look.
What I don't understand is why have such blatant disregard and disrespect for
the community members? if you find a bug, file it as JIRA, please stop making
blanket statements and over-arching pronouncements on code quality. Let us be
specific; if you see an issue; we welcome it; and thank you for your making our
software better; the nature of software development is such that we will have
issues. But let us do it with some consideration; and not say things like I
grepped and found suppressions; therefore your code is bad. if you don't
understand that code; feel free to ask us; perhaps we are wrong; and you have
found a bug; perhaps we have a perfectly valid reason to do it that way.
Once more; I stand by my earlier statement; unless you are willing to measure
and show me numbers – I am going to presume that you have really no really
issues to be solved.
Thank you for the conversation.
> Create separate unit and integration test executor dev-support script
> ---------------------------------------------------------------------
>
> Key: HDDS-1735
> URL: https://issues.apache.org/jira/browse/HDDS-1735
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Labels: pull-request-available
> Attachments: Screen Shot 2019-07-02 at 3.25.33 PM.png
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> hadoop-ozone/dev-support/checks directory contains multiple helper script to
> execute different type of testing (findbugs, rat, unit, build).
> They easily define how tests should be executed, with the following contract:
> * The problems should be printed out to the console
> * in case of test failure a non zero exit code should be used
>
> The tests are working well (in fact I have some experiments with executing
> these scripts on k8s and argo where all the shell scripts are executed
> parallel) but we need some update:
> 1. Most important: the unit tests and integration tests can be separated.
> Integration tests are more flaky and it's better to have a way to run only
> the normal unit tests
> 2. As HDDS-1115 introduced a pom.ozone.xml it's better to use them instead
> of the magical "am pl hadoop-ozone-dist" trick--
> 3. To make it possible to run blockade test in containers we should use - T
> flag with docker-compose
> 4. checkstyle violations are printed out to the console
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
