[
https://issues.apache.org/jira/browse/SUBMARINE-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830865#comment-16830865
]
Szilard Nemeth commented on SUBMARINE-52:
-----------------------------------------
Hi [~wangda]!
Let me summarize the jenkins issues that we are having here:
1. We are frequently getting errors like this (not with the latest build, but I
saw it for a previous patch), see the explanation Adam gave on another jira:
https://issues.apache.org/jira/browse/SUBMARINE-54?focusedCommentId=16825206&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16825206
This should be fixed soon by someone who has access to Jenkins as without this,
we can't update / merge any patch with confidence.
2. I don't really get the trunk mvnsite compilation errors as I don't have
compilation errors on trunk or either on my patched branch:
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/81/artifact/out/branch-mvnsite-hadoop-submarine_hadoop-submarine-yarnservice-runtime.txt
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/81/artifact/out/branch-mvnsite-hadoop-submarine_hadoop-submarine-tony-runtime.txt
3. Same goes with the findbugs result, these are red due to those compilation
errors mentioned in 2.
4. hadolint: Do we really want to fix these? I based my Dockerfile on PyTorch's
dockerfile plus added some bits I found from the TF docker file.
See the errors here:
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/81/artifact/out/diff-patch-hadolint.txt
5. whitespace:
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/81/artifact/out/whitespace-eol.txt
These are coming from the cifar 10 estimator python script I found on github. I
think we don't need to touch the script and fix these.
6. pylint:
https://builds.apache.org/job/PreCommit-SUBMARINE-Build/81/artifact/out/patch-pylint-stderr.txt
I don't really get these errors.
7. Checkstyle:
a.) I usually don't care about the HiddenField errors so I will do the same
here.
b.) Same goes with VisibilityModifier for protected fields: I don't feel it
will improve the code in any way to fix these.
c.) The rest of the checkstyle issues have been fixed with my latest patch.
So all in all, I think we are good.
What's your opinion?
Thanks!
> Generate Service spec + launch script for single-node PyTorch learning job
> --------------------------------------------------------------------------
>
> Key: SUBMARINE-52
> URL: https://issues.apache.org/jira/browse/SUBMARINE-52
> Project: Hadoop Submarine
> Issue Type: Sub-task
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: SUBMARINE-52-2.001.patch, SUBMARINE-52.001.patch,
> SUBMARINE-52.002.patch, SUBMARINE-52.003.patch, SUBMARINE-52.004.patch,
> SUBMARINE-52.005.patch, SUBMARINE-52.006.patch, SUBMARINE-52.007.patch,
> SUBMARINE-52.008.patch
>
>
> Similar to what we have for Tensorflow in
> {{org.apache.hadoop.yarn.submarine.runtimes.yarnservice.YarnServiceJobSubmitter}},
> we need a code that generates Service spec file (json) for PyTorch.
> We also need to take care of the separation of CLI/YAML arguments of TF /
> PyTorch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
