[
https://issues.apache.org/jira/browse/HDDS-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773255#comment-16773255
]
Allen Wittenauer commented on HDDS-891:
---------------------------------------
1.
bq. If I understood well, you suggest to double quote the
$DOCKER_INTERACTIVE_RUN variable in the docker run line.
It's not even that. To do what you are wanting to do should really just be an
extra flag to set the extra options rather than a full-blown 'set it from the
outside' variable.
bq. But please let me know if I am wrong.
Let me do one better and give you an example.
DOCKER_INTERACTIVE_RUN opens the door for users to set command line options to
docker. Most notably, -c and -v and a few others that share one particular
characteristic: they reference the file system. As soon as shell code hits the
file system, it is no longer safe to assume space delimited options. In other
words, -c /My Cool Filesystem/Docker Files/config.json or -v /c_drive/Program
Files/Data:/data may be something a user wants to do, but the script now breaks
because of the IFS assumptions.
This bug is exactly why shellcheck is correctly flagging it as busted code.
2.
bq. running docker based acceptance tests,
While it is not well tested, it should be doable with 0.9.0+ with --dockerind
mode. If external volumes are required to be mounted, things might get wonky
though. Just be aware that users and other ASF build server patrons get annoyed
when jobs take too long during peak hours. precommit response should be quick
checks with better, post-commit checks happening at full build time. If they
can't be triggered from maven test, then a custom test needs to be defined,
either in the personality (recommended) or in the --user-plugins directory (not
recommended, mainly because people will forget to set this option when they run
test-patch interactively).
bq. run ALL the hdds/ozone unit tests all the time, not just for the changed
projects
See above. Full tests are run as part of the nightlies due to time constraints.
bq. check ALL the findbugs/checkstyle issues not just the new ones
For findbugs, that's the --findbugs-strict-precheck option which AFAIK most/all
of the Hadoop jobs have enabled. It will fail the patch if there are
pre-existing findbugs issues. Adding a similar option to checkstyle wouldn't be
hard, but a reminder that this info is also presented in the nightlies. Also,
if the source tree is already clean, then new checkstyle failures should
technically be 'all' already.
Experience has shown, though, that users tend to blow right past precheck
failures and commit code anyway. [Hell, many PMC members ignore errors that
their own patches generated, blaming the Jenkins nodes when it's pretty clear
that their Java code has e.g., javadoc errors.]
3.
bq. I am convinced to run the more strict tests in addition to the existing
yetus tests.
It sounds like everything you want is either already there or is fairly trivial
to implement.
bq. Please let me know If I can do something to get Yetus results for the PR-s.
I think [[email protected]] just needs to edit the user/pw for the
hadoop-multibranch job credentials and HADOOP-16035 committed. I do
practically zero Java these days, so it may not be 100% and probably needs a
few more tweaks after it is implemented. (A definite flaw with Jenkins'
multibranch pipelines.)
> Create customized yetus personality for ozone
> ---------------------------------------------
>
> Key: HDDS-891
> URL: https://issues.apache.org/jira/browse/HDDS-891
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
>
> Ozone pre commit builds (such as
> https://builds.apache.org/job/PreCommit-HDDS-Build/) use the official hadoop
> personality from the yetus personality.
> Yetus personalities are bash scripts which contain personalization for
> specific builds.
> The hadoop personality tries to identify which project should be built and
> use partial build to build only the required subprojects because the full
> build is very time consuming.
> But in Ozone:
> 1.) The build + unit tests are very fast
> 2.) We don't need all the checks (for example the hadoop specific shading
> test)
> 3.) We prefer to do a full build and full unit test for hadoop-ozone and
> hadoop-hdds subrojects (for example the hadoop-ozone integration test always
> should be executed as it contains many generic unit test)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
