Hi Arnau,
Thank you very much for reporting this issue and taking the effort of
figuring out what the cause is.
Basically the issue is that the .append does not return anything, which
makes 'stats' equal to 'None'.
The problem is fixed with the changes made in
https://github.com/hpcugent/easybuild-framework/pull/1948, and a test
was added to make sure this aspect does not get broken again.
regards,
Kenneth
On 05/10/16 12:26, Arnau wrote:
Hi,
I've found that the problem occurs when the easyconfig file has some
previous build stats information in it. If the easyconfig is "clean"
(does not have any previous build stats infomration) EB is able to add
buildstats and push into the git respository.
/DISCLAIMER: I'm not a python guy so, please, doubt about what I'm
going to say: /
I think the problem is in filerepo.py. Stats has some value but after :
if previous:
stats = previous.append(stats)
stats gets emptied.
Looking at EB 2.8.1 (sorry, I told you that is was working in 2.8.2
but it was not true, it was working in 2.8.1) there's some code
difference:
89,95c101,105
< if not previous:
< statsprefix = "\n# Build statistics\nbuildstats = ["
< statssuffix = "]\n"
< else:
< # statstemplate = "\nbuildstats.append(%s)\n"
< statsprefix = "\nbuildstats.append("
< statssuffix = ")\n"
---
> statscomment = "\n# Build statistics\n"
> statsprefix = prefix
> statssuffix = "]\n"
> if previous:
> stats = previous.append(stats)
So, I've tried with the old code and it works again:
102,109c102,105
< if not previous:
< statsprefix = "\n# Build statistics\nbuildstats = ["
< statssuffix = "]\n"
< else:
< # statstemplate = "\nbuildstats.append(%s)\n"
< statsprefix = "\nbuildstats.append("
< statssuffix = ")\n"
<
---
> statsprefix = prefix
> statssuffix = "]\n"
> if previous:
> stats = previous.append(stats)
Kenneth, what do you think? did you test the new code? is it working
for you?
Best,
Arnau
2016-10-04 16:42 GMT+02:00 Arnau <[email protected]
<mailto:[email protected]>>:
Hi,
I think I got the error message:
== 2016-10-04 16:35:44,557 gitrepo.py:104 DEBUG rep name is
easyconfigs
== 2016-10-04 16:35:44,558 gitrepo.py:112 DEBUG connectiong to git
repo in /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs
== 2016-10-04 16:35:45,452 gitrepo.py:120 DEBUG pulled succesfully
to Already up-to-date. in /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs
== 2016-10-04 16:35:45,453 filetools.py:950 DEBUG Not creating
existing path /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs/FusionCatcher
== 2016-10-04 16:35:45,627 build_log.py:163 ERROR Can only pretty
print build stats in dictionary form, not of type <type
'NoneType'> (at easybuild/framework/easyconfig/tools.py:387 in
stats_to_str)
== 2016-10-04 16:35:45,628 easyblock.py:2439 WARNING Unable to
commit easyconfig to repository: "Can only pretty print build
stats in dictionary form, not of type <type 'NoneType'>"
it's in the temporary log file.
I'll look into EB code to try to understand what the problem is.
Thanks!
Arnau
2016-10-04 14:20 GMT+02:00 Kenneth Hoste <[email protected]
<mailto:[email protected]>>:
Hi Arnau,
Can you share the full debug log of an installation?
If something went wrong when copying to the git repo, you
should see a couple of (non-fatal) warnings pop up near the
end (shortly before the "COMPLETED: Installation ended
successfully" log message).
Let's first see if the EasyBuild log contains anything useful
before we try and debug GitPython (I'm not sure how to do that).
regards,
Kenneth
On 04/10/16 14:07, Arnau wrote:
Hi Kenneth,
thanks for your answer.
All our easyconfig files are saved in an internal git
repository. After each successfully build the easyconfig file
was being pushed to the git repository with its build
information. That's what is not working now. After new
successful build no git push is triggered.
No special commands. i.e:
$ eb -d --robot . easyconfig
$ eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment
variable, F: configuration file)
#
buildpath (E) = /dev/shm
installpath (E) = /XX/easybuild.el7.x86_64
modules-tool (E) = Lmod
optarch (E) = GENERIC
packagepath (E) = /XX/easybuild.el7.x86_64/packages
prefix (E) = /XX/easybuild.el7.x86_64
repository (E) = GitRepository
repositorypath (E) = [email protected]/easyconfigs.git
<mailto:[email protected]/easyconfigs.git>
robot-paths (D) =
/XX/easybuild.el7.x86_64/software/EasyBuild/2.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-2.9.0-py2.7.egg/easybuild/easyconfigs
sourcepath (E) = /XX/easybuild/sources
I've also tried passing the repository options in the command
line.
$ eb --check-github
== temporary log file in case of crash
/tmp/eb-MDnvHJ/easybuild-_xllBs.log
Checking status of GitHub integration...
Making sure we're online... OK
* GitHub user... (none available) => FAIL
* GitHub token... (no token found) => FAIL
* git command... OK ("git version 2.7.4; ")
* GitPython module... OK (GitPython version 1.0.1)
* push access to None/easybuild-easyconfigs repo @ GitHub...
FAIL (no GitHub user specified)
* creating gists... OK
* location to Git working dirs... not found (suboptimal)
One or more checks FAILed, GitHub configuration not fully
complete!
See
http://easybuild.readthedocs.org/en/latest/Integration_with_GitHub.html#configuration
<http://easybuild.readthedocs.org/en/latest/Integration_with_GitHub.html#configuration>
for help.
Status of GitHub integration:
* --from-pr: OK
* --new-pr: not supported
* --review-pr: OK
* --update-pr: not supported
* --upload-test-report: not supported
> It's normal that you're not seeing any 'git' commands pop up
in the debug log, this is handled entirely by GitPython.
any way to debug the GitPython process?
Thanks,
Arnau
2016-10-04 13:28 GMT+02:00 Kenneth Hoste
<[email protected] <mailto:[email protected]>>:
Hi Arnau,
On 04/10/16 12:25, Arnau wrote:
Hi,
while running EB 2.8.2 on SL6.7 we had some Git
Repository configuration that was working nicely:
repository=GitRepository
[email protected]/e
<mailto:[email protected]/e>asyconfigs.git
Now we have another instance running RH7 and EB2.9
and the same Git Repository configuration has stooped
working.
A part from new EB and new OS the GitPython package
has changed from 0.3.2-0.6.RC1.el6 to
1.0.1-5.el7 (both coming from EPEL). I've tried to
install an old version of GitPython
(/GitPython-0.3.2-0.7.RC1.fc21.noarch) but the
problem persists
I'm running in debug mode and I don't see any
refernce to any git command, only few refernces to
"name: github_account, config: None".
Anyone faced similar issue? How may I debug this problem?
Can you specify more precisely what you are trying
(commands executed, output of 'eb --show-config'), and
what is not working anymore?
What output are you getting from 'eb --check-github'
(although that's probably mostly irrelevant to your problem)?
It's normal that you're not seeing any 'git' commands pop
up in the debug log, this is handled entirely by GitPython.
regards,
Kenneth
