Hi,
I've found that the problem occurs when the easyconfig file has some
previous build stats information in it. If the easyconfig is "clean" (does
not have any previous build stats infomration) EB is able to add buildstats
and push into the git respository.
*DISCLAIMER: I'm not a python guy so, please, doubt about what I'm going to
say: *
I think the problem is in filerepo.py. Stats has some value but after :
if previous:
stats = previous.append(stats)
stats gets emptied.
Looking at EB 2.8.1 (sorry, I told you that is was working in 2.8.2 but it
was not true, it was working in 2.8.1) there's some code difference:
89,95c101,105
< if not previous:
< statsprefix = "\n# Build statistics\nbuildstats = ["
< statssuffix = "]\n"
< else:
< # statstemplate = "\nbuildstats.append(%s)\n"
< statsprefix = "\nbuildstats.append("
< statssuffix = ")\n"
---
> statscomment = "\n# Build statistics\n"
> statsprefix = prefix
> statssuffix = "]\n"
> if previous:
> stats = previous.append(stats)
So, I've tried with the old code and it works again:
102,109c102,105
< if not previous:
< statsprefix = "\n# Build statistics\nbuildstats = ["
< statssuffix = "]\n"
< else:
< # statstemplate = "\nbuildstats.append(%s)\n"
< statsprefix = "\nbuildstats.append("
< statssuffix = ")\n"
<
---
> statsprefix = prefix
> statssuffix = "]\n"
> if previous:
> stats = previous.append(stats)
Kenneth, what do you think? did you test the new code? is it working for
you?
Best,
Arnau
2016-10-04 16:42 GMT+02:00 Arnau <[email protected]>:
> Hi,
>
> I think I got the error message:
>
> == 2016-10-04 16:35:44,557 gitrepo.py:104 DEBUG rep name is easyconfigs
> == 2016-10-04 16:35:44,558 gitrepo.py:112 DEBUG connectiong to git repo in
> /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs
> == 2016-10-04 16:35:45,452 gitrepo.py:120 DEBUG pulled succesfully to
> Already up-to-date. in /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs
> == 2016-10-04 16:35:45,453 filetools.py:950 DEBUG Not creating existing
> path /tmp/eb-wa6QMO/git-wc-Ptx6bk/easyconfigs/FusionCatcher
> == 2016-10-04 16:35:45,627 build_log.py:163 ERROR Can only pretty print
> build stats in dictionary form, not of type <type 'NoneType'> (at
> easybuild/framework/easyconfig/tools.py:387 in stats_to_str)
> == 2016-10-04 16:35:45,628 easyblock.py:2439 WARNING Unable to commit
> easyconfig to repository: "Can only pretty print build stats in dictionary
> form, not of type <type 'NoneType'>"
>
> it's in the temporary log file.
>
> I'll look into EB code to try to understand what the problem is.
>
> Thanks!
> Arnau
>
>
>
> 2016-10-04 14:20 GMT+02:00 Kenneth Hoste <[email protected]>:
>
>> Hi Arnau,
>>
>> Can you share the full debug log of an installation?
>>
>> If something went wrong when copying to the git repo, you should see a
>> couple of (non-fatal) warnings pop up near the end (shortly before the
>> "COMPLETED: Installation ended successfully" log message).
>>
>> Let's first see if the EasyBuild log contains anything useful before we
>> try and debug GitPython (I'm not sure how to do that).
>>
>>
>> regards,
>>
>> Kenneth
>>
>>
>> On 04/10/16 14:07, Arnau wrote:
>>
>> Hi Kenneth,
>>
>> thanks for your answer.
>>
>> All our easyconfig files are saved in an internal git repository. After
>> each successfully build the easyconfig file was being pushed to the git
>> repository with its build information. That's what is not working now.
>> After new successful build no git push is triggered.
>>
>>
>> No special commands. i.e:
>>
>> $ eb -d --robot . easyconfig
>>
>>
>> $ eb --show-config
>> #
>> # Current EasyBuild configuration
>> # (C: command line argument, D: default value, E: environment variable,
>> F: configuration file)
>> #
>> buildpath (E) = /dev/shm
>> installpath (E) = /XX/easybuild.el7.x86_64
>> modules-tool (E) = Lmod
>> optarch (E) = GENERIC
>> packagepath (E) = /XX/easybuild.el7.x86_64/packages
>> prefix (E) = /XX/easybuild.el7.x86_64
>> repository (E) = GitRepository
>> repositorypath (E) = [email protected]/easyconfigs.git
>> robot-paths (D) = /XX/easybuild.el7.x86_64/softw
>> are/EasyBuild/2.9.0/lib/python2.7/site-packages/easybuild_
>> easyconfigs-2.9.0-py2.7.egg/easybuild/easyconfigs
>> sourcepath (E) = /XX/easybuild/sources
>>
>>
>> I've also tried passing the repository options in the command line.
>>
>>
>> $ eb --check-github
>> == temporary log file in case of crash /tmp/eb-MDnvHJ/easybuild-_xllB
>> s.log
>>
>> Checking status of GitHub integration...
>>
>> Making sure we're online... OK
>>
>> * GitHub user... (none available) => FAIL
>> * GitHub token... (no token found) => FAIL
>> * git command... OK ("git version 2.7.4; ")
>> * GitPython module... OK (GitPython version 1.0.1)
>> * push access to None/easybuild-easyconfigs repo @ GitHub... FAIL (no
>> GitHub user specified)
>> * creating gists... OK
>> * location to Git working dirs... not found (suboptimal)
>>
>> One or more checks FAILed, GitHub configuration not fully complete!
>> See http://easybuild.readthedocs.org/en/latest/Integration_with_
>> GitHub.html#configuration for help.
>>
>> Status of GitHub integration:
>> * --from-pr: OK
>> * --new-pr: not supported
>> * --review-pr: OK
>> * --update-pr: not supported
>> * --upload-test-report: not supported
>>
>>
>> > It's normal that you're not seeing any 'git' commands pop up in the
>> debug log, this is handled entirely by GitPython.
>>
>> any way to debug the GitPython process?
>>
>> Thanks,
>> Arnau
>>
>>
>> 2016-10-04 13:28 GMT+02:00 Kenneth Hoste <[email protected]>:
>>
>>> Hi Arnau,
>>>
>>> On 04/10/16 12:25, Arnau wrote:
>>>
>>>> Hi,
>>>>
>>>> while running EB 2.8.2 on SL6.7 we had some Git Repository
>>>> configuration that was working nicely:
>>>>
>>>> repository=GitRepository
>>>> [email protected]/easyconfigs.git
>>>>
>>>> Now we have another instance running RH7 and EB2.9 and the same Git
>>>> Repository configuration has stooped working.
>>>>
>>>> A part from new EB and new OS the GitPython package has changed from
>>>> 0.3.2-0.6.RC1.el6 to
>>>> 1.0.1-5.el7 (both coming from EPEL). I've tried to install an old
>>>> version of GitPython (/GitPython-0.3.2-0.7.RC1.fc21.noarch) but the
>>>> problem persists
>>>>
>>>> I'm running in debug mode and I don't see any refernce to any git
>>>> command, only few refernces to "name: github_account, config: None".
>>>>
>>>> Anyone faced similar issue? How may I debug this problem?
>>>>
>>>
>>>
>>> Can you specify more precisely what you are trying (commands executed,
>>> output of 'eb --show-config'), and what is not working anymore?
>>> What output are you getting from 'eb --check-github' (although that's
>>> probably mostly irrelevant to your problem)?
>>>
>>> It's normal that you're not seeing any 'git' commands pop up in the
>>> debug log, this is handled entirely by GitPython.
>>>
>>>
>>> regards,
>>>
>>> Kenneth
>>>
>>
>>
>>
>
