[boinc_dev] BOINC has a SciCrunch "Research Resource IDentifier" (RRID)

[Steffen Möller]

Is is

BOINC - Berkeley Open Infrastructure for Network Computing, RRID:SCR_015896

These RRIDs have become pretty much accepted as a help to orchestrate 
the zoo of tools in computational biology. So, whenever a finding that 
was produced with the help of BOINC is announced, be it online or in 
print, these 15 characters for RRID:SCR_015896 would not be 
inappropriate to surface.


Best,

Steffen


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Linux build issues

[Steffen Möller]

On 16.11.17 22:53, Kathryn Marks wrote:
> Installing the ssl lib worked.  Thanks.
>
>
> BTW:  Just doing this for the pure fun of it (yes I'm strange)
>
>
> That said, I'm stuck again.
>
> checking for LIBNOTIFY... no
> configure: error: Package requirements (libnotify) were not met:
>
> No package 'libnotify' found
Dependencies for Debian should also apply for Rapbian. You find them here:

https://anonscm.debian.org/cgit/pkg-boinc/boinc.git/tree/debian/control

which includes a dependency to ||

|libnotify-dev Enjoy! |

||

>> Why are you trying to build from source? It should already be available
>> for Raspian.
>>
>> Cheers,
>>
>> Laurence
>>
>>
>> On 16/11/17 20:28, Kathryn Marks wrote:
>>
>>> Hi all
>>>
>>> I'm having a difficult time building the client on Raspian.  The OS is
>>> fully updated everyday.
>>>
>>> I've solved previous errors, but I'm stuck on this.
>>>
>>>
>>> Package openssl was not found in the pkg-config search path.
>>> Perhaps you should add the directory containing `openssl.pc'
>>> to the PKG_CONFIG_PATH environment variable
>>> No package 'openssl' found
>>> checking for openssl... no
>>> configure: error:
>>> --
>>>Cannot find openssl libraries.
>>>
>>>Please install openssl or specify installation directory with
>>>--with-ssl=(dir).
>>> --
>>>
>>>
>>> I've tried to install SSL through the package manager (I'm, not fluent in
>>> apt-get).  I found a old post on the boinc message boards that someone
>>> solved the problem by installing libculibcurl4-openssl-devrl4-openssl-dev
>>> but that didn't help me.
>>>
>>> My linux fu is weak right now, so I'm not sure how to find ssl (I did do
>>> find ssl but that didn't help).  I also don't remember much about
>>> environmental variables.
>>>
>>> Any guidance would be appreciated.
>>>
>>>
>> ___
>> boinc_dev mailing list
>> boinc_dev@ssl.berkeley.edu
>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>> To unsubscribe, visit the above URL and
>> (near bottom of page) enter your email address.
>>
>
>
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] release 7.8.3?

[Steffen Möller]

Hi Laurence,

On 22.10.17 23:04, Laurence wrote:

> The BOINC client v7.8.3 is available for testing on Linux
>
> For Debian/Ubuntu
>
> sudo apt-add-repository ppa:costamagnagianfranco/boinc
> sudo apt-get update
> sudo apt-get install boinc-manager

kind of :)

For the regular user of Debian-derived distros, "sudo apt-get install
boinc" is the way to go. No extras.

boinc-manager is the GUI for controlling local or remote clients.
boinc-client is the client with the boinccmd command line.

to install both in one go there is the aforementioned third package
(consisting only of dependencies to boinc-manager and boinc-client) with
the name "boinc".

Both Debian and Ubuntu have BOINC packages shipping as regular parts of
their distribution since version 5.2.15 in March 2006. This is some
freaking 12 years, see http://snapshot.debian.org/package/boinc/.   Mint
also takes it from there. The PPA (personal package archive) of
Gianfranco is a neat extra service for those aiming at running the very
latest very early but do not run the latest version of the distribution
(where does this guy get all the energy from? Ubuntu Artful is on 7.8.3,
Zesty still on 7.6.33, but later backports to the earlier versions of
Ubuntu are to be expected). The same Gianfranco (and sometimes also I)
do for the backports.debian.org. It is all in perfect shape as seen on
https://tracker.debian.org/pkg/boinc .  As TarotApprentice had pointed
out, this needs an extra flag to apt-get like -t stretch-backports to be
installed instead of the sufficiently recent of Debian stable.

The number of Debian installations are mostly invariant at
https://qa.debian.org/popcon.php?package=boinc .  The Ubuntu PopCon is a
bit more tedious to read - 45204 installations are reported from those
who installed the popularity-contest package. No idea about the other
Debian derivatives.


>
> For Fedora/RedHat/CentOS
>
> sudo dnf config-manager --set-enable updates-testing
> sudo dnf install boinc-manager
>
> Test reports can be sent to the alpha testing project and bugs can be
> reported to the Fedora or Debian bug trackers respectively.
>
OpenSuSE has https://software.opensuse.org/package/boinc-client but is
still at 7.2.42.   For arch I found
https://wiki.archlinux.org/index.php/BOINC and
https://www.archlinux.org/packages/?q=boinc but they are still at 7.6.33
. I have no contact to either of those maintainers but would only be
worried for the also RPM-based SuSE community.

I very much agree that a distribution-agnostic self-extracting static
version makes perfect sense until those distributions are also covered.
However, I would rather work on finding volunteer packagers for those
platforms than to work on the .sea for those who do not want to compile
the very latest versions themselves. archlinux likely only needs a
friendly email informing the maintainers that the 7.8.3 can now be trusted.

Cheers,

Steffen


>
> On 16/10/17 00:49, David Anderson wrote:
>> 7.8.3 has 90% test coverage and it looks good.
>> Any objections to releasing it?
>> -- David
>> ___
>> boinc_dev mailing list
>> boinc_dev@ssl.berkeley.edu
>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>> To unsubscribe, visit the above URL and
>> (near bottom of page) enter your email address.
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] release 7.8.3?

[Steffen Möller]

Hello,

On 17.10.17 00:15, David Anderson wrote:
> Our policy for a long time was:
>
> - provide a .sea release (including manager) for current Ubuntu/x86,
>   with no effort to make it run anywhere else.
> - provide a client-only (no manager) release built on an old Linux
> system,
>   everything static, for maximum compatibility.
>   We have a "compatability VM" in which to build this.
Close to nothing on Ubuntu is statically linked. And for BOINC there is
not need to be statically linked, either - when shipping as as part of the
distribution, this is.
> At some point (maybe when Rom left) we stopped doing this.
> I suggest that we start again.
> Maybe change to 64 bit.

Of course you can just go and do that. And for the "competition is
always good"
mantra or as a reference for bug reports I am also embracing that. A
complementary strategy could be to just embrace all those volunteer
packagers out there and call them a part of BOINC.

I have not talked back to Gianfranco about it, but I have some confidence
that he does not mind to move the Debian/Ubuntu-package build instructions
from git.debian.org to github for easier accessibility, if that is of
any help.

Best,

Steffen


>
> On 10/16/2017 7:46 AM, Steffen Möller wrote:
>> On 16.10.17 15:07, Laurence wrote:
>>> Hi Jord,
>>>
>>> On 16/10/17 12:24, Jord van der Elst wrote:
>>>> On Mon, Oct 16, 2017 at 12:06 PM, Richard Haselgrove
>>>> <r.haselgr...@btopenworld.com <mailto:r.haselgr...@btopenworld.com>>
>>>> wrote:
>>>>
>>>>  Berkeley has outsourced the distribution of Linux clients to the
>>>>  package maintainers for individual distributions, for several
>>>>  years now.
>>>>
>>>>
>>>> Laurence is the release manager for Linux, I suspect he knows about
>>>> all of that. :-)
>>> No I didn't, so thanks.  Only stepped forward after the September
>>> workshop.
>>>> But even if the distro package maintainers release these versions,
>>>> they also do so first for testing and can then still ask people to
>>>> report their results on the Alpha project. Their source code is still
>>>> coming from github as well.
>>> Even if the package maintainers build and release, as you pointed out
>>> the versioned upstream code still comes from the project. I have just
>>> discovered that Gianfranco is doing daily builds (every 4h) and
>>> creating packages for Debian from the git master. This is a nice step
>>> towards CI.
>> And he also builds the complete packages together with the server side
>> components afterwards that go to the experimental section of Debian -
>> see the boinc-server-maker package
>> https://packages.qa.debian.org/b/boinc/news/20171005T094923Z.html.  It
>> would help a lot if the server bits in master are always be at a stage
>> that it could be released.
>>
>> In my mind this goes as far as that for automated testing we could have
>> dummy project set up in an automated fashion and do few workunits on
>> those.
>>
>> Cheers,
>>
>> Steffen
>>
>> ___
>> boinc_dev mailing list
>> boinc_dev@ssl.berkeley.edu
>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>> To unsubscribe, visit the above URL and
>> (near bottom of page) enter your email address.
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] release 7.8.3?

[Steffen Möller]

On 16.10.17 23:07, Laurence Field wrote:
> Hi Steffen,
>
> On 16/10/17 16:46, Steffen Möller wrote:
>>
>> And he also builds the complete packages together with the server side
>> components afterwards that go to the experimental section of Debian -
>> see the boinc-server-maker package
>> https://packages.qa.debian.org/b/boinc/news/20171005T094923Z.html.  It
>> would help a lot if the server bits in master are always be at a stage
>> that it could be released.
> Thanks, I was already aware of the server in the experimental section
> and completely agree that master should always be 'deployable'.
> Recently I built a server package for CentOS7 and asked for some
> feedback on this very list. There was none.

That server-side package is somewhat of importance to me - please kindly
(re)send me a pointer to your packaging instructions and I certainly
will have comments - I mean, I have tons of comments on what I had once
come up with :)

> At the BOINC workshop in Paris this was discussed and the consensus
> was to try using Docker for the server releases.

Right. Would work. And it would make some sense to capsule it all away,
too. I just hope you get it into Docker/Singularity as a package, not
from the sources.

The boinc-server-maker package ships make-project, including the example
apps, so there is no need to perform the compilation, and the Docker
image is based on Debian (old-stable) anyway. Takes far less than five
minutes, too :)

> The approach was stimulated by this presentation from Marius. We at
> LHC@home plan to investigate this for our project at some point in the
> future.
>
> https://indico.cern.ch/event/648533/contributions/2710464/attachments/1519821/2373725/boincserverdocker.pdf
>

Except that your are in Scientific Linux land, so I presume.


>> In my mind this goes as far as that for automated testing we could have
>> dummy project set up in an automated fashion and do few workunits on
>> those.
>>
> It seems from the presentation that we are not that far from this goal.

Good.

The server side is not only about getting the project going. It is also
about clarifying the exact include paths and generally helping projects
with updates/upgrades of the BOINC-parts of their infrastructure. And
then it does not matter if you run on bare metal or with some sort of
virtualisation. You want to know about what version of the BOINC server
was updated with what other version and that it has worked. Linux
distributions are pretty good at this. As a neat side effect, the
example projects get compiled for the many architectures of Debian,
which you can collect and distribute to clients.

Most important to me is that the BOINC server as a package in a regular
Linux distribution brings (data parallel) HPC to every home, to every
scientific institute. I tend to think of it as a social achievement.
Having containers/a VM in between is desirable for security purposes,
but it also weakens the point a bit.

Sorry for being somewhat "emotional" here,

Steffen


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] release 7.8.3?

[Steffen Möller]

On 16.10.17 15:07, Laurence wrote:
> Hi Jord,
>
> On 16/10/17 12:24, Jord van der Elst wrote:
>> On Mon, Oct 16, 2017 at 12:06 PM, Richard Haselgrove
>> >
>> wrote:
>>
>>     Berkeley has outsourced the distribution of Linux clients to the
>>     package maintainers for individual distributions, for several
>>     years now.
>>
>>
>> Laurence is the release manager for Linux, I suspect he knows about
>> all of that. :-)
> No I didn't, so thanks.  Only stepped forward after the September
> workshop.
>> But even if the distro package maintainers release these versions,
>> they also do so first for testing and can then still ask people to
>> report their results on the Alpha project. Their source code is still
>> coming from github as well.
> Even if the package maintainers build and release, as you pointed out
> the versioned upstream code still comes from the project. I have just
> discovered that Gianfranco is doing daily builds (every 4h) and
> creating packages for Debian from the git master. This is a nice step
> towards CI.

And he also builds the complete packages together with the server side
components afterwards that go to the experimental section of Debian -
see the boinc-server-maker package
https://packages.qa.debian.org/b/boinc/news/20171005T094923Z.html.  It
would help a lot if the server bits in master are always be at a stage
that it could be released.

In my mind this goes as far as that for automated testing we could have
dummy project set up in an automated fashion and do few workunits on those.

Cheers,

Steffen

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

[boinc_dev] Separation of BOINC client and server source tree

[Steffen Möller]

Hello,

We cannot distribute the BOINC server package in Debian since it is not
allowed to have code redundancy between packages of the distribution,
and be it for the source packages, so we cannot have the server and
client releases in parallel. The server side is on me, so this is
somewhat fine since I am happily busy on many fronts, but it is also
unfortunate as I think many good things could be achieved with a range
of short time projects. In particular I see it bringing BOINC closer to
wet lab environments who then provide proof of a prediction, which leads
to more BOINC-applying publications and respective promotion. Today,
with every client release the BOINC server package in the experimental
section must also be updated to avoid it being removed from the
distribution to avoid it looking outdated, but this then is from the
client source tree, not from the server one, so it is wrong.

When you now meet to rethink about how to organise the BOINC source tree
( refer to as set C of files) and have releases, please also have a
thought about how to have all the client needs (set A) separated from
what the server needs (set B) but the client does not (set B \ A). We
have the separation in A and B/A for the Debian BOINC server build
instructions (see
https://anonscm.debian.org/cgit/pkg-boinc/boinc.git/tree/debian).

There may be good reasons to keep both sides of the BOINC project in a
single source tree that I fail to see. Another way to get to something
consistent to distribute would then be to have regular releases of the
master branch, which is then have to be stable both for the BOINC client
and for the BOINC server.

Best,

Steffen

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Software development and branches, was Re: [boinc_projects] keywords

[Steffen Möller]

On 07.08.17 17:51, Oliver Bock wrote:
> On 07.08.2017 16:21, Steffen Möller wrote:
>> So, please find a way to stop this so very outdated discussion.
> This discussion is meant to help reaching a consensus on how to do
> things in the future - the very things you asked for. Only when one
> established the how one can establish the who.
I apologize for the confusion I seem to have caused. I am with you,
as you know.  Here a somewhat more frank rewrite of what I had
meant to say:

People on this thread take one of two positions:
 a) have it the way we always had it in BOINC, mostly in ignorance of
what git could do for BOINC
 b) have it the same way that git is used in any other larger Open
Source project
More discussions in this thread do more harm than good in my experience,
hence my urgent request to stop it. Come to an agreement, which must be
b) for BOINC to survive,  or fork and have it like b). But, please, by
all means, stop discussing where there is not much to discuss. Nobody
will voluntarily change behaviors because of an email thread. One needs
to experience first hand how a different development model works to
learn from that.

>
>> Heck, the
>> git-creator-Linus' Linux kernel just gives a very nice model of how it
>> can be done once it gets any complicated
> Right, and it has been part of this discussion.
>
>> No need to outsmart them.
> Outsmart? The kernel folks? As you know, we haven't even established the
> bare bones yet. Where do we try to outsmart anyone?
Missunderstanding. "you" as in a) (the current git routine in BOINC) are
trying
to do something different than the git-routine of b) for no particular
reason.

I hope to have clarified my position and that it is perceived as a
constructive contribution.

Steffen




___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Software development and branches, was Re: [boinc_projects] keywords

[Steffen Möller]

Hello,

On 07.08.17 21:40, David Anderson wrote:
> Steffen:
> I agree - git is here to serve us, not vice versa :-)
;) I just removed a story on why people introduce
software like SAP in a company that has not seen
proper controlling before. Too much text.

> I've proposed creating server release branches,
> similar to the existing client release branches.
> Hopefully this will satisfy everyone's needs.
No. It won't it is very much off the problem. Offer
separate repositories ( as in github.com/BOINC/client
and github.com/BOINC/server, not branches, and you
make people happier. Of course the two repositories
have zero code redundancy.

I understand that you do not want to update a server as
often as you want to update a client. Separate release
schemes seem like a natural consequence. Separate branches
for maintenance are not a natural consequence. Releases
are points in time. Releases are not branches but tags.
So, the fear I have read on this thread is that your coworkers
do not know from where to branch their development and
that there is nothing reliable to merge that development
back to.

Steffen - will be in Budapest at the time of the workshop. Good luck!


>
> On 8/7/2017 7:21 AM, Steffen Möller wrote:
>> Well, git makes one's "senses reeling", at least when one has started a
>> career with CVS. Linus had helped us escape, and he possibly gets more
>> respect from me for that than for his kernel work. For me, it was the
>> git repository for BOINC _packaging_ that had forced me into it, long
>> before BOINC transitioned to git. I am still thankful for it.
>>
>> So, please find a way to stop this so very outdated discussion. When you
>> get together now, just vote for the individual or set of individuals who
>> shall be in charge of what is the new_master branch from which releases
>> are to be branched off.  And you can have the same or other sets of
>> individuals in charge of any of these 7.8.x or 7.10.x branches. And you
>> can have individuals in charge for various features of BOINC. Heck, the
>> git-creator-Linus' Linux kernel just gives a very nice model of how it
>> can be done once it gets any complicated, just, BOINC is comparatively
>> simple. No need to outsmart them.
>>
>> Best,
>>
>> Steffen
>>
>>
>> On 07.08.17 12:20, Jord van der Elst wrote:
>>
>>> Thanks for that last paragraph, Oliver. You put into words what has
>>> been
>>> running through my mind since Friday.
>>>
>>>
>>> -- Jord van der Elst.
>>>
>>> On Mon, Aug 7, 2017 at 9:55 AM, Oliver Bock <oliver.b...@aei.mpg.de>
>>> wrote:
>>>
>>>> On 06/08/17 22:40 , David Anderson wrote:
>>>>> Testing a feature in isolation is not the same as testing the system.
>>>> True.
>>>>
>>>>> No one is advocating committing untested or buggy code into master.
>>>> Yet it happens most of the time, mostly because *development*
>>>> happens in
>>>> master. And even if one sees a seldom feature branch for development
>>>> it's more often than not merged to master after incomplete feature
>>>> testing (e.g. not even a full build on all platforms). In any case
>>>> master is broken.
>>>>
>>>>> However, feature testing doesn't mean that master is stable.
>>>> Right, but it should be as stable as possible which requires a
>>>> continuous improvement effort. Why is it broken and what can you do to
>>>> actively avoid it next time?
>>>>
>>>>> For that, we need to do system-level testing in a separate release
>>>> branch.
>>>>
>>>> You're right that system-level (a.k.a. integration) testing should
>>>> take
>>>> place in a *specific* branch. However that branch should be master in
>>>> our opinion. As Laurence pointed out: release branches are to
>>>> stabilize
>>>> and fix releases.
>>>>
>>>> I agree with Bernd: can it be that your resistance to use master for
>>>> integration just stems from the fact that you don't like developing in
>>>> feature branches because you're still used to CVS or SVN, and in your
>>>> mind branching and merging still is a pain?
>>>>
>>>>> This is what we've done for years with the client software.
>>>> Thing is, it's probably time to reconsider your view on BOINC. That
>>>> "we"
>>>> means 2-3 developers running their "own" project. BOINC is different
>>>> now, at least it officially wants to be

Re: [boinc_dev] Software development and branches, was Re: [boinc_projects] keywords

[Steffen Möller]

Well, git makes one's "senses reeling", at least when one has started a
career with CVS. Linus had helped us escape, and he possibly gets more
respect from me for that than for his kernel work. For me, it was the
git repository for BOINC _packaging_ that had forced me into it, long
before BOINC transitioned to git. I am still thankful for it.

So, please find a way to stop this so very outdated discussion. When you
get together now, just vote for the individual or set of individuals who
shall be in charge of what is the new_master branch from which releases
are to be branched off.  And you can have the same or other sets of
individuals in charge of any of these 7.8.x or 7.10.x branches. And you
can have individuals in charge for various features of BOINC. Heck, the
git-creator-Linus' Linux kernel just gives a very nice model of how it
can be done once it gets any complicated, just, BOINC is comparatively
simple. No need to outsmart them.

Best,

Steffen


On 07.08.17 12:20, Jord van der Elst wrote:

> Thanks for that last paragraph, Oliver. You put into words what has been
> running through my mind since Friday.
>
>
> -- Jord van der Elst.
>
> On Mon, Aug 7, 2017 at 9:55 AM, Oliver Bock  wrote:
>
>> On 06/08/17 22:40 , David Anderson wrote:
>>> Testing a feature in isolation is not the same as testing the system.
>> True.
>>
>>> No one is advocating committing untested or buggy code into master.
>> Yet it happens most of the time, mostly because *development* happens in
>> master. And even if one sees a seldom feature branch for development
>> it's more often than not merged to master after incomplete feature
>> testing (e.g. not even a full build on all platforms). In any case
>> master is broken.
>>
>>> However, feature testing doesn't mean that master is stable.
>> Right, but it should be as stable as possible which requires a
>> continuous improvement effort. Why is it broken and what can you do to
>> actively avoid it next time?
>>
>>> For that, we need to do system-level testing in a separate release
>> branch.
>>
>> You're right that system-level (a.k.a. integration) testing should take
>> place in a *specific* branch. However that branch should be master in
>> our opinion. As Laurence pointed out: release branches are to stabilize
>> and fix releases.
>>
>> I agree with Bernd: can it be that your resistance to use master for
>> integration just stems from the fact that you don't like developing in
>> feature branches because you're still used to CVS or SVN, and in your
>> mind branching and merging still is a pain?
>>
>>> This is what we've done for years with the client software.
>> Thing is, it's probably time to reconsider your view on BOINC. That "we"
>> means 2-3 developers running their "own" project. BOINC is different
>> now, at least it officially wants to be. You said BOINC is now a
>> community project. If you really mean it, then please listen to the
>> community. From what I can tell, the community is in agreement on how
>> things should be done nowadays. Why are you opposing *all* of us? Also,
>> you haven't yet given any concrete arguments/reasons why the model we
>> propose *really* wouldn't work. So far, you stated personal
>> impressions/facts that were often misinformed or in fact unrelated to
>> what we discussed. All of this could be resolved constructively, it
>> would just take some open mindedness. Please consider this: when you're
>> thinking "why is everyone but me headed in the wrong direction?", it's
>> probably about time to reconsider you own course.
>>
>> Best,
>> Oliver
>>
>>
>>
>> ___
>> boinc_dev mailing list
>> boinc_dev@ssl.berkeley.edu
>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>> To unsubscribe, visit the above URL and
>> (near bottom of page) enter your email address.
>>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Steffen Möller]

On 22.07.17 10:13, David Anderson wrote:
>
> On 7/21/2017 1:26 AM, David Wallom wrote:
>> the responsibility for functions to different community groups. As
>> such it will be essential that we move to a multi branch development
>> methodology in some form of public repository. I use this in a large
>> number of my current international projects and it works well.
> Interesting idea.  Do you know if Github has these sorts of features?
I have seen that with the Rosetta code base - you have multiple
groups/lead researchers  that offer stable "theme-master" branches from
which others then branch off and then send pull requests to their lead.

Steffen

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Steffen Möller]

Don't know how much it is worth, but as a part-time contributor to
_many_ Open Source projects (and once having worked with a computer
science degree as a software engineer for a software company that does
software environments for software development) I would like to express
my wholehearted support for Oliver's position.

Quick summary: Developments in branches that branch from master and are
merged back to master once the development has completed. Strategic
adjustments have an influence on the developments that are started but
have no direct effect on the master branch. Of course you can always
fork or name master something else. But you always have something stable
to branch your new developments from.

Steffen


On 21.07.17 09:49, Oliver Bock wrote:
> Hi David,
>
> On 21/07/17 0:50 , David Anderson wrote:
>> This discussion comes down to two contrasting models for software
>> development:
> Sorry, no, that's not the point.
>
>> 1) The "waterfall model":
>> 2) The "agile model":
> I know both models very well, professionally and scientifically.
>
>> Requirements and design can change on the fly.
>> New features are divided into smaller self-contained pieces,
>> which are completed and merged with master frequently. 
> Yes, totally right.
>
> (master serving as integration branch in this case)
>
>> Development need not be done in a separate branch.
> Where does this conclusion come from? To the contrary: agile
> development, due to its velocity, needs separate feature/topic/fix
> branches because many developers need to work on the same codebase at
> the same time, without interfering with each other. One needs
> coordination (by workflow). Also, developers need a stable base to start
> off from, not something that's broken most of the times. How do you want
> to achieve that when everyone develops (let alone tests) in the same
> branch? How do you want to do integration tests without affecting
> development or release maintenance?
>
> Please ask yourself why tools like git, global agile development and
> particular SCM workflows like GitFlow more or less arrived and spread
> massively at the same time. Mere correlation or actual causality...?
>
>> Oliver prefers the waterfall model,
> The opposite is true. I'm all for agile development like Scrum and I'm
> even evangelizing to do so wherever I go (and see fit). Why have I tried
> for years to convince BOINC to move to git and make use of Continuous
> Integration for instance? Because I prefer the waterfall model?
>
>> so strongly that he won't work with people who use agile.
> This all leaves me kind of speechless. I can't even fathom how one can
> reach a conclusion like that, based on what I've described. In
> particular since SCM (that's all we talked about in this thread) is just
> one part of implementing a process model instance, it doesn't define it.
>
> There are obviously severe misunderstandings at play here and I'm
> honestly starting to doubt they can be overcome - unless other projects
> step up and chime in as well.
>
>> I prefer the agile model for several reasons:
> All those reasons/motivations (first sentence of each given bullet) are
> absolutely fine. Unfortunately the conclusions/consequences reached are
> wrong. What I and others proposed *will help you* with what you're doing
> and trying to achieve as it's at the heart of agile development. So far
> we've just failed to show you how.
>
> Apart from that there are other developers, projects and contributors
> (existing and prospective) who depend on how BOINC's SCM is done. I
> think we agree that their needs need to be taken into account as well,
> even if you have the impression that their needs are at odds with yours
> - which might not even be the case.
>
>> Also, let's not confuse the choice of model with how we do stable branches.
> Precisely!
>
>
> HTH,
> Oliver
>
>
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] master broken

[Steffen Möller]

On 12/05/2017 12:16, Christian Beer wrote:
> On 12.05.2017 12:05, Vitalii Koshura wrote:
>> Hello Gianfranco,
>>
>> We have Travis CI for continuous builds and it is not failed.
>> Could you please take a look why Travis built it ok and you got errors?
> My guess is that there is a compiler option that warns about format
> errors which is treated as an error because of another option and both
> are not activated in our default build but they are on the debian build.
> I thought it's a compiler vresion issue but my gcc 6 doesn't even warn
> me about this. The travis build uses gcc 4 I think.

I had seen it in past versions of the BOINC client and the one or other
app - I recall some history on using "printf" without a format. That is
fine for string literals like 'printf("huhu")' but you cannot just do
"char *c="huhu"; printf(c);". Or, to be more precise, you apparently can
but you should not, really, it should be printf("%s",c), instead. The
compiler says just that and Debian rebuilds enforce compiler settings
that do not allow it to pass. Debian refers to this as "hardening",
where there is more on  https://wiki.debian.org/Hardening  . You may
want to discuss adding those flags for your travis builds, too.

Cheers,

Steffen

>> 2017-05-12 12:20 GMT+03:00 Gianfranco Costamagna <
>> costamagnagianfra...@yahoo.it>:
>>
>>> Hello David,
>>>
>>> boinc_client-app.o `test -f 'app.cpp' || echo './'`app.cpp
>>> /usr/bin/g++ -DHAVE_CONFIG_H -I. -I..  -I.. -I../api -I../db -I../lib
>>> -I../lib/mac -I../sched -I../tools -I../vda -pthread -Wdate-time
>>> -D_FORTIFY_SOURCE=2  -Wall -Wextra -Wshadow -Wredundant-decls
>>> -Wdisabled-optimization -Wpointer-arith -Wstrict-aliasing -Wcast-align
>>> -I/usr/include -I/usr/include/openssl  -g -O3 -fdebug-prefix-map=/<<
>>> BUILDDIR>>/boinc-7.7.0+dfsg~git20170511+r23716~r6~ubuntu16.10.1=. -fPIE
>>> -fstack-protector-strong -Wformat -Werror=format-security -Wall -O3
>>> -funroll-loops -fforce-addr -ffast-math -flto -Wall -c -o
>>> boinc_client-app_config.o `test -f 'app_config.cpp' || echo
>>> './'`app_config.cpp
>>> app_config.cpp: In function ‘void 
>>> print_msgs(std::vector>> app_config.cpp:128:58: error: format not a string literal and no format
>>> arguments [-Werror=format-security]
>>> msg_printf_notice(p, false, NULL, msgs[i].c_str());
>>> ^
>>> cc1plus: some warnings being treated as errors
>>>
>>>
>>> +static void print_msgs(vector msgs, PROJECT* p) {
>>> +for (unsigned int i=0; i>> +msg_printf_notice(p, false, NULL, msgs[i].c_str());
>>> +}
>>>
>>> you probably need to use
>>> msg_printf_notice(p, false, NULL, "%s", msgs[i].c_str());
>>>
>>> (note: I didn't test the above)
>>>
>>> cheers,
>>>
>>> G.
>>> ___
>>> boinc_dev mailing list
>>> boinc_dev@ssl.berkeley.edu
>>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>>> To unsubscribe, visit the above URL and
>>> (near bottom of page) enter your email address.
>> ___
>> boinc_dev mailing list
>> boinc_dev@ssl.berkeley.edu
>> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
>> To unsubscribe, visit the above URL and
>> (near bottom of page) enter your email address.
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Suspected cpid hash collisions for newly installed clients on Linux

[Steffen Möller]

ffect also leaves oddities on the server, like this from my first
> experience of this issue
>
> http://www.primegrid.com/show_host_detail.php?hostid=512618
>
> as you can see the computer has a different creation and last contact time,
> so you might think it had contacted the server at least twice. But by the
> server's own count, it has done so zero times. Maybe you can see how that
> makes sense (apart from it being a tunnelling effect of your quantum
> computing module ;)
>
>
> I am now told on the PG forum that "Linux sometimes fails to pick up the
> MAC address".
>
> ALSO, I have seen this among my collection of 11 desktop machines, 2 of
> which are identical apart from MAC address, and 1 is a NFS server, and 8
> are diskless loading their OS from the server using PXE and root=/dev/nfs.
>
> The server runs LinuxMInt 18.1, The other desktop machines run a minimal
> Debian command line OS, netinstall plus ssh plus boinc-client. These are
> cloned, but the boinc directories are re-initialised each time to contain
> only the four config files in /etc/boinc-client and softlinks to them from
> /var/lib/boinc, plus a minimal account_www.primegird.xml that provides my
> weak auth code. In particular, there is no contamination of the 
> value as the file that holds that value is not cloned.
>
> Running the diskless machines one at a time works fine, but it does seem
> random whether it picks up thew history of its own hardware, or of one of
> the other machines. I am not sur about this yet, I am still collecting data.
>
> In any case, my preference would be to start each freshly booted machine as
> a new machine on the PG server, allowing me to merge them manually (I have
> been around long enough to remember when that was the norm after a
> re-install, and personally I preferred that).
>
> If  I do not turn a machine on for months on end, when it is powered up I
> want it to be at the default location, not wherever that physical hardware
> was located last time it was used. I do understand the this old behaviour
> changed because of specific requests from users who had their own reasons
> for wanting hardware continuity.
>
> I am fairly sure there is at least one bug here, possbily a different bug
> in the two scenarios.
>
> I find it interesting that so far, in ten months, there has not yet been a
> case of confusion between a laptop and a desktop -- at some poiunt the
> different hardware becomes sufficiently different to avoid ambiguity.
>
> So, FIRST I believe there is a bug that means that cpid is sometimes
> independent of the MAC address.
>
> SECOND, I am requesting a user-selectble option that allows a user, at
> instll time, to choose to switch hardware-continuity on or off.
>
> I believe the second could be achieved by asking a question in the
> post-install trigger, and if the user wanted hardware continuity off, the
> script would create a cpid based on a freshly generated uuid. There could
> even be a three way option: hardware based, based on hostname, or based on
> a fresh-every install cpid (the latter not beng a hash of anything on the
> system but a random based uuid with the punctuation stripped out).
>
> This option would not be offered where the post install trigger found a
> pre-existing stare file with a pre-existing cpid.
>
> As a work-around, that same option would solve the issues created if there
> is, in fact, a bug in the client-generated cpid routine.
>
> Unless you guys can suggest a good reason why not, I intend to make this
> change to the .deb on my own system, and see what happens. I have spent too
> much time clearing up the messes that false allegations of "abandonment"
> make -- by which I mean when tasks on a different set of hardware get
> marked as abandoned.
>
> If you know of a reason why this idea is unwise, as a home project, please
> let me know in the next few days.
>
> I am also offering this to you as something you may (or may not) want to
> roll out more generally.
>
> I also do not know about making a bug report. This effect comes and goes,
> and I can think I have an effective way to avoid it (as with buying new USB
> dongles) then that can fall in a heap. I cannot (yet) produce a definitive
> recipe to reliably demonstrate this effect, so up to know I have not filed
> a bug report. Do you think I should?
>
>
>
> I would value your thoughts on any of the above.
>
> And if it does turn out to be provoked by me in a way I have not yet
> thought of, I would be glad to know that, too.
>
> Is there anything else you need to know from me at this time?
>
> River~~
>
>
> On 22 January 2017 at 15:17, Steffen Möller <steffen_moel...@gmx.de> wrote:
>
&

Re: [boinc_dev] providing BOINC through the Apple App Store

[Steffen Möller]

Homebrew has a

$ brew search boinc
Caskroom/cask/boinc

and it might make some good sense to interact with those folks a bit
more, also for local compilations of scientific apps. But how many will
find that by chance who would not go and install from the website. No idea.

Steffen


On 23/05/16 10:00, David Anderson wrote:
> It would be great to have BOINC in the App Store.
> However, there are items in Apple's "Review Guidelines"
> that would seem to preclude BOINC:
>
> 2.15 Apps must be self-contained, single application installation
> bundles, and cannot install code or resources in shared locations
> 2.16 Apps that download or install additional code or resources to add
> functionality or change their primary purpose will be rejected
> 2.17 Apps that download other standalone Apps will be rejected
>
> See
> https://developer.apple.com/app-store/review/guidelines/mac/#functionality
>
> It's possible that Apple would relax these rules in our case.
> If anyone wants to initiate a dialog with them, please go ahead.
>
> -- David
>
> On 5/23/2016 12:46 AM, pola...@comcast.net wrote:
>> hello,
>> thank you for your wonderful software. I want to ask if it is
>> possible to provide this software through the Apple App store? The
>> reason that I ask is because OS X provides an option for security to
>> download software only through the App Store to prevent malware from
>> being installed into the OS X operating system. I have read that some
>> opensource software has been compromised by malware such as the
>> Transmission bittorrent client. I presume that Apple does some
>> additional security checks before releasing apps through the App
>> Store. Are there copyright or patent issues or financial reasons that
>> prevent you from doing this? thanks for your time and your efforts
>> developing this software.
>> joe
>> The attachment to this e-mail as well as the content of the email
>> itself may be privileged, proprietary or confidential. Do not open
>> it. It is intended only for the e-mail recipient noted above. If you
>> are not the
>> intended recipient or a person responsible for delivering this
>> transmission to the intended recipient, you may not disclose, copy or
>> distribute this transmission or take any action in reliance on it.
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

[boinc_dev] Astonishing effect of Link Time Optimisation on BOINC client

[Steffen Möller]

Dear all,

some of you may be aware of the BOINC client shipping with Debian,
Ubuntu, Mint and the other .deb-based Linux distributions [1]. While
updating to 7.6.32 we have revised our invocation of the Link Time
Optimisation (LTO) [2]. This had the effect of a notable reduction
of the disk-space occupied by the (stripped) binaries:

previous revised change
 7.6.31   7.6.32   
1292528   932032  -28%  /usr/bin/boinc
  3490430864  -16%  /usr/bin/boinccmd
3657424  2292936  -37%  /usr/bin/boincmgr
  26464264640%  /usr/lib/x86_64-linux-gnu/libboinc_crypt.so.7.6.32
 665960   6659600%  /usr/lib/x86_64-linux-gnu/libboinc.so.7.6.32
 467392   4673920%  /usr/lib/x86_64-linux-gnu/libboinc_zip.so.7.6.32

We cannot assess the effect on BOINC. But we reckon that the effort to
quickly bring BOINC functionality into action and then hide it again is
likely proportional to the reduction of the code base. LTO is also know
to help the computation - not surprisingly so since less code needs to be
processed for the same functionality. Completely ignorant of the complexity
of the BOINC benchmark, we just went and checked *boinccmd --run_benchmarks*
with the following results:

v7.6.31 (prior to revision)

09-May-2016 03:39:53 [---]3973 floating point MIPS (Whetstone) per CPU
09-May-2016 03:39:53 [---]17322 integer MIPS (Dhrystone) per CPU
09-May-2016 03:42:22 [---]4087 floating point MIPS (Whetstone) per CPU
09-May-2016 03:42:22 [---]17774 integer MIPS (Dhrystone) per CPU
09-May-2016 03:43:17 [---]4069 floating point MIPS (Whetstone) per CPU
09-May-2016 03:43:17 [---]17988 integer MIPS (Dhrystone) per CPU

v7.6.32 (after revision)
09-May-2016 03:45:18 [---]3961 floating point MIPS (Whetstone) per CPU
09-May-2016 03:45:18 [---]78155 integer MIPS (Dhrystone) per CPU
09-May-2016 03:46:35 [---]4057 floating point MIPS (Whetstone) per CPU
09-May-2016 03:46:35 [---]76289 integer MIPS (Dhrystone) per CPU
09-May-2016 03:48:42 [---]3962 floating point MIPS (Whetstone) per CPU
09-May-2016 03:48:42 [---]76374 integer MIPS (Dhrystone) per CPU

We tried two different computers with equivalent quadruplication of the
integer performance only. The change is the same as for v7.6.32 without
the LTO corrections, which vividly demonstrates how sensitive the LTO
compiler flags are. The -flto flags we had first added one or two years
ago.

Expected was to observe a 20% speedup that is frequently reported for
LTO. Without further investigation, the 300% we consider to be somehow
explainable - with hindsight - such that the application could be optimised
in a way that it is also faster for regular compilation. But why bother
if LTO has become so usable. And helpful. And controllable by merely 
looking at what it does to the disk footprint.

This email has two messages:
 * If using the Debian client already, please update to 7.6.32 that is now
   in Debian unstable and comes to you any time soon. It reduces the memory
   footprint, which helps the scientific app, a bit, and because of an
   astonishing effect on the benchmark, this also helps your RAC. (This is
   until BOINC developers adopt LTO also for their publicly offered builds.)
 * It hopes to be an eye opener. The scientific applications are the ones
   that should all adopt LTO. It is working, also for static binaries. But
   expect a 20% speedup, not as much as with the client's benchmarks. 

Keep crunching

Gianfranco and Steffen


[1] https://wiki.debian.org/BOINC
[2] https://gcc.gnu.org/onlinedocs/gccint/LTO.html#LTO

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Boinc and shared dynamic libraries

[Steffen Möller]

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Problems with crypt_prog

[Steffen Möller]

Hi Crit/Randolph,

I cannot recall to have run into this issue myself. This is how it looks for 
us, following
(and improving) the Debian Wiki page:

moeller@twin1a:~ $ dpkg -L boinc-server-maker | grep crypt
/usr/lib/boinc-server-maker/lib/crypt_prog
moeller@twin1a:~ $ dpkg -l boinc-server-maker
ii  boinc-server-maker   7.2.28+dfsg-1exp1 amd64 BOINC server 
applications and data files

Are you running the same version? It should reside in debian/experimental or 
snapshot.debian.org,
so we always have something current when we want it updated for some security 
reason.

From above location crypt_prog is copied to the project root's
subdirectory 'bin', as in 
/var/lib/boinc-server-uppercase-test/test/bin/crypt_prog

Is locate crypt_prog helping to locate the binary? What did you execute when
the error surfaced?

Cheers,

Steffen


 Gesendet: Dienstag, 18. Februar 2014 um 09:57 Uhr
 Von: David Anderson da...@ssl.berkeley.edu
 An: boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] Problems with crypt_prog

 Did you do configure/make?
 That builds crypt_prog (and many other programs that you'll need).
 -- David
 
 On 18-Feb-2014 12:52 AM, Crit Crit wrote:
  I encountered an error already posted to this mail, but i didn't find an 
  answer,
  solution or even reply.
 
  Original post:
 
  Randolph Patterson light.proton at gmail.com
  Mon May 9 10:05:30 PDT 2011
 
  Hello, thanx for reading
 
  Im actually trying to configure a BOINC server, by reading many guides such
  as the one at the official wiki, and the guide at the debian wiki
 
  But im getting troubles with making the proyect, im getting this error here
  *
  *
  *Setting up server files: generating keys*
  *sh: /home/boinc-server/boinc/lib/crypt_prog: not found*
  *FATAL ERROR: Command failed: /home/boinc-server/boinc/lib/crypt_prog
  -genkey 1024 /var/www/boinc/keys/upload_private
  /var/www/boinc/keys/upload_public /dev/null*
 
  Any kind of help will be greatly appreciated.
 
  In my case i'm running on Ubuntu 13.04 on VM. I'm getting:
 
  *sh: /home/boinc-server/boinc/lib/crypt_prog: not found*
 
  *FATAL ERROR: Command failed: /home/boinc-server/boinc/lib/crypt_prog
  -genkey 1024
 
  I heard a thing, that it might be caused by missing prerequirements, but I 
  am
  unable to track what packed i missed. I used this lists: 1)
  http://boinc.berkeley.edu/trac/wiki/SoftwarePrereqsUnix and 2)
  http://boinc.berkeley.edu/trac/wiki/ServerIntro (Debian cookbook part).
 
  .
  ___
  boinc_dev mailing list
  boinc_dev@ssl.berkeley.edu
  http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
  To unsubscribe, visit the above URL and
  (near bottom of page) enter your email address.
 ___
 boinc_dev mailing list
 boinc_dev@ssl.berkeley.edu
 http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
 To unsubscribe, visit the above URL and
 (near bottom of page) enter your email address.
 
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] GLIBC 2.15 required for BOINC Manager?

[Steffen Möller]

Hello,

Gesendet: Mittwoch, 29. Januar 2014 um 19:41 Uhr
Von: Jon Sonntag j...@thesonntags.com

My only hesitation with backports is the warning on the home page which
states: Backports cannot be tested as extensively as Debian stable, and
backports are provided on an as-is basis, with risk of incompatibilities
with other components in Debian stable. Use with care!

It should read: The decision to provide an a package through backports
is with the maintainer of that package. Other than with the stable release,
the package had little opportunity to experience peer review. We trust
our maintainers, but mistakes and unexpected incompatibilities may happen.
In case of difficulties, installations can be reverted to the previous
version, just avoid too many updates from backports at the same time
and report your experience. 
 
So, I went ahead and installed Ubuntu 12.04 on a couple virtual machines
to do Linux OpenCL and CUDA development. IMO, it makes no sense for
Debian 7 to become unstable by using semi-tested backports.

Previous packages I had tested on a server of ours that intentionally
escaped the updates. I could not test cuda on it, though. To render the
backports fully tested, I would have needed help. But if it builds, why
shouldn't it work.

Cheers,

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] GLIBC 2.15 required for BOINC Manager?

[Steffen Möller]

Debian stable ships 7.0.27, backported is 7.0.65.
There is a deadline for a stable point release upcoming
next week for which there could be an update, if
we really want this to happen. The version distributed
via backport.debian.org I could update any time.

Steffen

 Gesendet: Dienstag, 28. Januar 2014 um 21:08 Uhr
 Von: Rom Walton r...@romwnet.org
 An: Steffen Möller steffen_moel...@gmx.de, Jon Sonntag 
 j...@thesonntags.com
 Cc: BOINC Developers Mailing List @berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: RE: [boinc_dev] GLIBC 2.15 required for BOINC Manager?

 At this point, our builds are built against Ubuntu Precise Pangolin(12.04) 
 and use whatever version of libc it does.
 
 I figured I would have to setup a new build machine to support wxWidgets 3.0 
 for the BOINC 7.3/7.4.
 
 Does Debian stable include a recent version of BOINC?
 
 - Rom
 
 -Original Message-
 From: boinc_dev [mailto:boinc_dev-boun...@ssl.berkeley.edu] On Behalf Of 
 Steffen Möller
 Sent: Sunday, January 26, 2014 5:36 AM
 To: Jon Sonntag
 Cc: BOINC Developers Mailing List @berkeley.edu
 Subject: Re: [boinc_dev] GLIBC 2.15 required for BOINC Manager?
 
 Hi Jon,
 
  Debian 7.3 (stable) uses glibc v2.13, Debian Jessie (testing)  uses 
  v2.17, and Debian Sid (unstable) uses v2.18 so it looks like they've 
  changed their minds.  However, at present, the only way to run the 
  current BOINC client on the current Debian version is to use either the 
  testing or unstable
  sources.
 
 If I got this right, then you are saying that the binaries provided for Linux 
 by Berkeley are not usable for the stable Debian release, right?
 Hm. I had accepted the message from past postings to some BOINC mailing list 
 that Debian's BOINC packages we are providing is not usable for everyone, and 
 my latest upload to backports.debian.org is (admittedly) a while ago.
 But from my side I would not mind if the Berkeley developers left the support 
 for Debian stable to the Debian people - just, I personally do not run stable 
 and it is tedious for me to test it all, so I am not too keen on frequent 
 updates.
 
  That, or download and compile your own BOINC client.  I don't think 
  most end users can or are willing to do that. I don't care too much if 
  the latest BOINC version doesn't work since I usually wait until a few 
  others identify all the issues with the current version so I can 
  decide whether to upgrade. That is, unless BOINC keeps nagging me to 
  use a version that I know won't run every single time I launch the 
  BOINC manager.  That gets to be really annoying.)
 
 Hm. This is exactly the spirit that backports.debian.org should help everyone 
 with. 
  
  Yeah, I know Debian isn't as popular as it once was.  In fact, since 
  only RHEL, OpenSUSE, and Ubuntu are supported by nVidia and AMD's GPU 
  toolkits, why not switch the official BOINC server VM's (and 
  instruction on the wiki
  pages) to one of those?  Then the same VM image can be used to build 
  and test client apps as well as run the server.
 
 The NVidia packages shipping with Debian work nicely. Concerning support, 
 with Valve's StreamOS distribution deriving directly from Debian, I have some 
 good confidence that a continued bet on Debian is a good one. 
 
  I'd be happy to help with
 
 Same here.
 
 Cheers,
 
 Steffen
 ___
 boinc_dev mailing list
 boinc_dev@ssl.berkeley.edu
 http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
 To unsubscribe, visit the above URL and
 (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] GLIBC 2.15 required for BOINC Manager?

[Steffen Möller]

Hi Jon,

 Debian 7.3 (stable) uses glibc v2.13, Debian Jessie (testing)  uses v2.17,
 and Debian Sid (unstable) uses v2.18 so it looks like they've changed their
 minds.  However, at present, the only way to run the current BOINC client
 on the current Debian version is to use either the testing or unstable
 sources.

If I got this right, then you are saying that the binaries provided for
Linux by Berkeley are not usable for the stable Debian release, right?
Hm. I had accepted the message from past postings to some BOINC mailing list
that Debian's BOINC packages we are providing is not usable for everyone,
and my latest upload to backports.debian.org is (admittedly) a while ago.
But from my side I would not mind if the Berkeley developers left the
support for Debian stable to the Debian people - just, I personally do not
run stable and it is tedious for me to test it all, so I am not too keen
on frequent updates.

 That, or download and compile your own BOINC client.  I don't
 think most end users can or are willing to do that. I don't care too much
 if the latest BOINC version doesn't work since I usually wait until a few
 others identify all the issues with the current version so I can decide
 whether to upgrade. That is, unless BOINC keeps nagging me to use a version
 that I know won't run every single time I launch the BOINC manager.  That
 gets to be really annoying.)

Hm. This is exactly the spirit that backports.debian.org should help everyone
with. 
 
 Yeah, I know Debian isn't as popular as it once was.  In fact, since only
 RHEL, OpenSUSE, and Ubuntu are supported by nVidia and AMD's GPU toolkits,
 why not switch the official BOINC server VM's (and instruction on the wiki
 pages) to one of those?  Then the same VM image can be used to build and
 test client apps as well as run the server. 

The NVidia packages shipping with Debian work nicely. Concerning support,
with Valve's StreamOS distribution deriving directly from Debian, I have some
good confidence that a continued bet on Debian is a good one. 

 I'd be happy to help with 

Same here.

Cheers,

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] addressing some clang warnings

[Steffen Möller]

 Gesendet: Dienstag, 09. Juli 2013 um 11:47 Uhr
 Von: Gianfranco Costamagna costamagnagianfra...@yahoo.it
 An: Alyssa Milburn amilb...@math.leidenuniv.nl
 Cc: steffen_moel...@gmx.de steffen_moel...@gmx.de, 
 boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] addressing some clang warnings

 - Messaggio originale -

  Da: Alyssa Milburn amilb...@math.leidenuniv.nl
  A: Gianfranco Costamagna costamagnagianfra...@yahoo.it
  Cc: steffen_moel...@gmx.de steffen_moel...@gmx.de; 
  boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
  Inviato: Martedì 9 Luglio 2013 11:43
  Oggetto: Re: [boinc_dev] addressing some clang warnings
 
  On Tue, Jul 09, 2013 at 08:25:24AM +0100, Gianfranco Costamagna wrote:
     -        fscanf(f, %2x, n);
     +        fs = fscanf(f, %2x, n);
          key-data[i] = n;
     +    if (EOF == fs) return ERR_NULL;
   
Shouldn't these check that fscanf's result is 1
(e.g. change these to 'if (1 != fs)'?)
 
   I partially agree.
   The most appropriate solution should be
   if(1 != fs || EOF != fs)
 
 You are right,
 I mean
 if(1 == fs || EOF == fs)

 sorry, I shouldn't send mail in the morning :p

And we should bail out before n is assigned to data[i],
and if so, data[i] should get (or already have) some not
assigned, yet value.

Many thanks for the extra eyeballs.

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] boinc libstdc++ linking

[Steffen Möller]

 Gesendet: Montag, 08. Juli 2013 um 21:46 Uhr
 Von: David Anderson da...@ssl.berkeley.edu

 The programs in samples/ are examples of how to build
 applications that will run on as many Linux machines as possible,
 including those that have old or missing libstdc++.
 That's why they statically link libstdc++.a,
 and why they have custom Makefiles.

We are experimenting (in Debian experimental) with the boinc-server-maker
package. The samples/* had been built in the past and went into a
package of their own, i.e. boinc-app-examples. As a regular Debian package,
also the boinc-app-examples (samples/*) are auto-built on all the ~11 different
Linux platforms Debian supports. The idea behind it all was to allow
anybody interested in BOINC to have a sample-project configured that allows
multi-platform contributions by not much more than a wget from the Debian
repository.

I have not yet made my mind up about libstdc++. At least for Debian and
Ubuntu, /usr/bin/{boinc,boinccmd} are dynamically linked against libstdc++,
and as such, the beast should already be installed when the app is run.
I do not know about the official BOINC distribution. For the sample apps,
such failures could even be considered educational.
 
 They're not intended to build out of the box,
 or to be built by the latest compilers.
 They're not built by a top-level configure/make.

I completely understand that there are too busy to care too much corners
in the BOINC code. I have complete confidence that you would accept patches
to bring back compatibility with latest compilers. But, could you sketch for
us to some integration with the - not top-level, but maybe within the
boinc-server realm - build process that you would be prepared to accept?

Cheers,

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] addressing some clang warnings

[Steffen Möller]

 Gesendet: Dienstag, 09. Juli 2013 um 00:36 Uhr
 Von: David Anderson da...@ssl.berkeley.edu
 An: Gianfranco Costamagna costamagnagianfra...@yahoo.it
 Cc: boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] addressing some clang warnings

 Are these changes all thoroughly tested?
 -- David

By eye they look fine in the sense that they should not disturb
today's regular operation. The patches did their best in embedding
themselves to the respective error handling philosophy of the
surrounding code and just added a (void) if it was felt that
an error did not want to be detected at all.

I just skimmed through it again, the one here

 145 --- a/api/graphics2_unix.cpp
 146 +++ b/api/graphics2_unix.cpp
 147 @@ -191,7 +191,9 @@
 148  FILE *f = boinc_fopen(gfx_info, r);
 149  if (f) {
 150  // ToDo: change this to XML parsing
 151 -fscanf(f, %d %d %d %d\n, xpos, ypos, width, height);
 152 +if (! fscanf(f, %d %d %d %d\n, xpos, ypos, width, height)) {
 153 +   fprintf(stderr,Coud not parse parameters for xpos, ypos, 
width, height from glx_info file.\n);
 154 +   }
 155  fclose(f);
 156  }
 157  

is not strict enough, which again is better than not checking at all. It should 
check if 
the return value of fscanf is != 4 (if I have counted right).

The patch already ships with the latest Debian/Ubuntu package and I am not 
aware of any
disturbance to be reported because of it.

Cheers,

Steffen

 
 On 08-Jul-2013 8:51 AM, Gianfranco Costamagna wrote:
  Hi David et all,
 
  The following patch address many clang warnings.
 
 
  Other people can see the patch here
  http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob;f=debian/patches/more_clang_warnings.patch;h=56bb29543205bdb06e295f5bdc4d3e914dfb96dc;hb=HEAD
 
   The patch is from Steffen Moeller.
 
  thanks
 
  Gianfranco
 
 ___
 boinc_dev mailing list
 boinc_dev@ssl.berkeley.edu
 http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
 To unsubscribe, visit the above URL and
 (near bottom of page) enter your email address.
 
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] boinc libstdc++ linking

[Steffen Möller]

Hello,

 Gesendet: Montag, 08. Juli 2013 um 12:53 Uhr
 Von: Alyssa Milburn amilb...@math.leidenuniv.nl
 An: Gianfranco Costamagna costamagnagianfra...@yahoo.it
 Cc: boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] boinc libstdc++ linking

 On Mon, Jul 08, 2013 at 11:44:35AM +0100, Gianfranco Costamagna wrote:
  Ok Alyssa I understand your point, so something like
  if
     don't have any libstdc++ at all
  then
     link it statically
  else
     link dinamically
 
  could be good enough?

 You build the apps on the server, and distribute only the binaries to
 the clients (usually).

 The apps are meant to be run on the *clients*, and you don't know what
 libstdc++ is on the clients.

 So, you have to build the apps so that they're compatible with as many
 distributions/versions as possible, which usually means static linking.
 But.. I thought that this is a problem for libc anyway, which is why
 the instructions say to use an old VM for building apps:

 http://boinc.berkeley.edu/trac/wiki/CompileAppLinux

You can link statically, including the libc or the libstdc++. Just, what
you redistribute gets larger.

 So I think it's almost impossible to build compatible apps as a Debian
 or Ubuntu package, unless I misunderstand something. It's an annoying
 problem.

Redistributing binaries with a Debian/Ubuntu package? This is what
Debian/Ubuntu is all about. You just need to provide a package for every
version of the distribution ... well ... at least this is what Debian/Ubuntu
do. The packages with apps for BOINC follow the naming scheme  boinc-app-X
and the one I find to work very nicely is boinc-app-seti. We take it a bit
to the extreme and even link dynamically to as much as we can, which includes
the two BOINC libraries.

The binaries are prepared by build demons for all distributions (just like
what the Ubuntu already has for everyone with its PPA (and Debian soon gets
and has for packages in backports.debian.org). I would expect it to be possible
to cherry-pick the binary autobuilt for an older Ubuntu version, say, Lucid,
and use those binaries for a Linux-wide redistribution - if you just linked
statically - just do it and then reupload a dynamic version.

Steffen


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

 Gesendet: Montag, 17. Juni 2013 um 21:29 Uhr
 Von: David Anderson da...@ssl.berkeley.edu
 An: boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] Can we do shared memory with no disk usage?

 In situations were BOINC is causing unexpectedly large
 ( 1 GB/hour) disk I/O,
 we need to figure out the source of the I/O.
 -- David

The situation goes away with the boinc-app-seti package's binary instead of the
official SETI clients, which is not configured for the graphics. I confirmed 
this
on two machines, one of which beging the laptop that had so severe problems 
before.

Example for WCG's clean energy (1 client) together with SETI-no-graphics (2 
clients):

iotop:

 6046 idle boinc   0.00 B/s 1300.60 K/s  0.00 %  0.00 % 
../../projects/www.worldcommunitygrid.org/wcgrid_cep2_qchem~4.C28H14S5Se.18.3.bp86.svp.n.opt
 4038401046153527445 8411 0
 6043 idle boinc   0.00 B/s  408.16 B/s  0.00 %  0.00 % 
../../projects/www.worldcommunitygrid.org/wcgrid_cep2_6.40_~-exec 
wcgrid_cep2_qchem_prod_linux.x86 -float 1 -stop 43200

a  1MB/s write by the WCG, the two non-graphical SETI are just silent. The 
same I observed with the no-graphics Milkyway@Home of Debian. I then tried POEM 
and Yoyo@Home, which was seen in iotop, albeit performing equally and always  
a tolerable 1KB/s rate.

I'll go and try geeting the original situation back with a self-compiled 
graphics-enabled SETI over the course of next week.

Cheers,

Steffen


 On 17-Jun-2013 11:14 AM, Steffen Möller wrote:
  Hello,
 
  I presume nobody wants to have the user play too much with any folders. I
  liked the all on a ramdisk idea, but BOINC occupies more than 10GB on the
  machine(s) in question, and it will similarly ask for that much (too much
  with today's common 16 or 24 GB setups when adding the memory for the apps)
  also for other setups.
 
  The checkpointing interval I understood (asked around) to be ignored by the
  client apps of a quite a few projects. Mine is set to some 7200 seconds (two
  hours) and the IO does not decrease. It would not be my prime target for
  optimisation.
 
  Could it be the graphics? We once observed that SETI without configuration
  for graphics is a few %ages faster than the same client with the same
  compilation options but graphics-savvy, even though no graphical client was
  run. If much IO was used for that, this might be an explanation. I cannot
  test this at the moment.
 
  Cheers,
 
  Steffen
 
 
  Gesendet: Montag, 17. Juni 2013 um 17:20 Uhr Von: Eric J Korpela
  korp...@ssl.berkeley.edu An: Steffen Möller steffen_moel...@gmx.de Cc:
  boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu Betreff: Re:
  [boinc_dev] Can we do shared memory with no disk usage?
 
  We considered using memory mapped files for the checkpoint state information
  in SETI@home, but decided that it was virtually impossible to guarantee
  synchronization on exit.  And, of course, the problem with using a ram disk
  for checkpoints that the checkpoint data disappears if power is lost.
 
 
  But if you want checkpoints to not occur, there is a preference for that...
  Tasks checkpoint to disk at most every:   60 seconds
 
  You can set it to 8 hours.  But you'll also want to choose suspend to
  memory.  Of course the output of the app (which depending on the app might
  be more frequent than checkpoints) will still spin up the disks.
 
  If you really want to operate from a ram disk, just copy your project
  directory to /dev/shm and link it into /var/lib/boinc/projects.   You'll
  still need to back it up occasionally for those times when the power goes
  out. ___ boinc_dev mailing list
  boinc_dev@ssl.berkeley.edu
  http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe,
  visit the above URL and (near bottom of page) enter your email address.
 
 ___
 boinc_dev mailing list
 boinc_dev@ssl.berkeley.edu
 http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
 To unsubscribe, visit the above URL and
 (near bottom of page) enter your email address.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

Hello,

 Gesendet: Mittwoch, 19. Juni 2013 um 21:39 Uhr
 Von: Nicolás Alvarez nicolas.alva...@gmail.com

 2013/6/18 Heinz-Bernd Eggenstein heinz-bernd.eggenst...@aei.mpg.de:
  Hi all,
 
  Interesting. I guess it would be useful to run BOINC on a dedicated
  partition (e.g. ext hard disk/ USB stick)  to isolate BOINC's contribution
  to the stats.

 Or drop iostat and use iotop instead, which gives realtime I/O stats
 per *thread*.

Many thanks for the suggestion. Done it. It is not the boinc client but the
scientific applications that make up for the IO. But that was not so much
disputed. The remaining question is about how much of that IO is due to the
 * file based shared memory
 * status updates
 * checkpointing
 * the respective app's intrinsic purpose

To help the interpretation of iotop, please feel reminded that the
average persecond rate in KB/s for an average 2GB/s is
 (2*1024*1024*1024)/(60*60*1024)
[1] 582.5422

Averaged over 10 seconds, I get typically values between 50 and 150KB/s

Total DISK READ :   0.00 B/s | Total DISK WRITE :  55.31 K/s
Actual DISK READ:   0.00 B/s | Actual DISK WRITE:  68.83 K/s
  TID  PRIO  USER DISK READ  DISK WRITE  SWAPIN IOCOMMAND
 2337 be/4 root0.00 B/s0.00 B/s  0.00 %  0.06 % [flush-8:0]
 4095 idle boinc   0.00 B/s7.56 K/s  0.00 %  0.05 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3111937
 4068 idle boinc   0.00 B/s4.77 K/s  0.00 %  0.03 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3160216
 4675 idle boinc   0.00 B/s9.15 K/s  0.00 %  0.02 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3030191
 3589 idle boinc   0.00 B/s9.95 K/s  0.00 %  0.01 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3194037
 4401 idle boinc   0.00 B/s4.38 K/s  0.00 %  0.01 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3075156
 4132 idle boinc   0.00 B/s6.76 K/s  0.00 %  0.00 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3764396
 4133 idle boinc   0.00 B/s  407.43 B/s  0.00 %  0.00 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3764396
18744 idle boinc   0.00 B/s 1222.28 B/s  0.00 %  0.00 % boinc 
--check_all_logins --redirectio --dir /var/lib/boinc-client
 1022 idle boinc   0.00 B/s6.37 K/s  0.00 %  0.00 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3231298
 3869 idle boinc   0.00 B/s4.77 K/s  0.00 %  0.00 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3180570
 4096 idle boinc   0.00 B/s0.00 B/s  0.00 %  0.00 % 
../../projects/boinc.bakerlab.org_rosetta/minirosetta_3~watchdog -run::rng 
mt19937 -constant_seed -jran 3111937

and while run over 10 minutes, iostat gives

$ date; iostat -h -m; sleep 600; date; iostat -h -m

Thu Jun 20 12:04:04 CEST 2013

Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda   3.27 0.02 0.35  29254 668009

Thu Jun 20 12:14:04 CEST 2013

Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda   3.27 0.02 0.36  29254 668359

So, over 10 minutes, we still have a 35 MB/(60s) = 597KB/s and the ever 
reported 2 GB/h.

One of my first actions after reading the initial iostat output (not shown) was 
to have the ext4 formatted root partition to receive the extra option 
commit=60. The noatime was already set. All data presented here were taken 
after that change.

The file mapping of shared memory seems to be generally accepted as a possible 
IO trap, as seen e.g. here 
http://serverfault.com/questions/361032/high-disk-i-o-when-cache-is-used .

It seems a bit like the scientific applications have some bits here and there 
to tune their IO behaviour. Running Milkyway 0.18 for instance, I see no IO for 
the clients at all while they went from 5% to 27%.

I have locally implemented a patch for (presumed) diskless shared memory. You 
can have a look over here
http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob;f=debian/patches/mmap_mem_only.patch;hb=HEAD
Again: it is not applied in Debian's regular BOINC packages but needs to be 
commented in in debian/patches/series.

I'll report if it works and about respective effects on SETI (may required a 
recompilation) next week.

Best,

Steffen

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom 

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

 Gesendet: Montag, 17. Juni 2013 um 21:29 Uhr
 Von: David Anderson da...@ssl.berkeley.edu
 An: boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] Can we do shared memory with no disk usage?

 In situations were BOINC is causing unexpectedly large
 ( 1 GB/hour) disk I/O,
 we need to figure out the source of the I/O.
 -- David


My little centrino laptop had SETI (official client) run over night.

$ iostat -h -m
Linux 3.8-1-rt-amd64 (Toshiba)  06/18/2013  _x86_64_(2 CPU)
...
Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
 33.80 0.17 0.39   3040   6859
$ date
Tue Jun 18 00:30:06 CEST 2013
$ iostat -h -m
...
Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
 65.32 0.07 0.98   3387  45737
$ date
Tue Jun 18 08:36:50 CEST 2013

Looking only at the last column, in those 8 hours this were

 (45737-6859)/1024
[1] 37.9668

GB and consequently

 (45737-6859)/1024/8
[1] 4.74585

GB/h


another machine, running about 10 clients in parallel (all Rosetta, one WCG),
had a bit less of IO ... here iostat was run twice with 1h interim sleep

twin1a:~ $ date ; iostat -m -h
Mon Jun 17 23:34:12 CEST 2013
Linux 3.2.0-3-amd64 (twin1a)06/17/2013  _x86_64_(24 CPU)

Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
  3.08 0.02 0.30  29091 504228

$ sleep 3600 ; date ; iostat -m -h
Tue Jun 18 00:34:26 CEST 2013
..
Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
  3.09 0.02 0.30  29091 506276


So this is about 2 GB per hour written and nothing read 

And yet another machine, same hardware, mostly einstein with 1 Rosetta and 1 WCG

twin1b:~ $ date; iostat -h -m ; sleep 3600 ; date ; iostat -h -m
Mon Jun 17 23:58:16 CEST 2013
...
Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
  1.67 0.00 0.12  758482377118


Tue Jun 18 00:58:16 CEST 2013

Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn
sda
  1.67 0.00 0.12  758482379914


Which means another 2 GB per hour.

The machines were not running anything else but BOINC, the laptop had firefox 
open in the background.
No BOINC graphical clients run anywhere.

iostat comes with the sysstat package, for anyone out there to try.

The laptop only has 1G of mem, which may lead to some swapping and account for 
some of the IO. Still,
to me it looks mostly like some process writing a lot of status information 
that is not read by anyone.
The missing reads for the big machines I might want to explain by large IO 
buffers ... there certainly
have been a couple of uploads that should have caused some read, otherwise.

Cheers,

Steffen

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

 Gesendet: Montag, 17. Juni 2013 um 08:46 Uhr
 Von: David Anderson da...@ssl.berkeley.edu

 Manager/client communication uses TCP - no disk I/O.
 So the possible sources of large disk I/O are:

 1) checkpoint or output file generation by apps
 2) writing of client_state.xml (or maybe other files)
 by the BOINC client

The checkpointing and the client_state.xml for my 24/7
machines I would very much like to just switch off or
have updated only every hour or have their update initiated
only upon request by the boinc client.

 3) client/app communication via memory-mapped files.
 According to my calculations,
 this should generate less than 1 MB/Hour per task.
 Note: we use memory-mapped files (mmap) instead of
 pure shared memory segments (shmget)
 because Mac OS X has a system-wide limit of 32 shared-mem segments,
 and some Linux systems have limits.
 Maybe there's a way to configure memory-mapped files
 to not write back to the disk file, but I can't find one.

It should be the shm_open with mmap together, i.e. just substituting the
call to open that BOINC currently performs with shm_open.
From http://pubs.opengroup.org/onlinepubs/009695399/functions/shm_open.html

fd = shm_open(/myregion, O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
rptr = mmap(NULL, sizeof(struct region),
   PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

Should be disk-free, with /myregion only having a symbolic meaning.
Here
http://stackoverflow.com/questions/4836863/shared-memory-or-mmap-linux-c-c-ipc
I also found a reference to an interesting IPv6 to localhost with multicast 
idea,
No idea if that is applicable for BOINC. The trend is more towards 
open_shm+mmap.

Many thanks for the swift reply

Steffen


 Can someone investigate which of these is the source of the large I/O?

 -- David

 On 16-Jun-2013 11:03 AM, Steffen Möller wrote:
  Hello,
 
  iostat gives rather drastic values for the amount of data that is written 
  to disk by
  BOINC and/or its applications.  Some good fellow once crafted a but report 
  about it
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=636075
  and my own reply was not overly helpful at the time. To reduce the IO 
  overhead is
  certainly helpful. Reduced latency is one thing, but with an outreach to 
  the mobile
  world there are many SSD / flash device users who care so much about write 
  endurance.
 
  It kept bugging me, and quite a while back it came to me that this may not 
  be
  because of the application's work but instead be mere overhead of a file 
  based
  communication between the app and the boinc-client / boinc-manager. I just 
  never
  got around chasing this up, also having read so often about communication 
  done via
  shared memory, which should not need much IO, and if so, then not with disk 
  devices
  but something like tmpfs. Let's see how it is done, I just thought.
 
   From what I found, there are two functions to create shared memory in 
  BOINC, both in
  lib/shmem.cpp. One is through
 create_shmem
  which internally uses shmget and should be just fine. The other is
 create_shmem_mmap
  which internally uses mmap - which can be memory-only or not. The early 
  mmap allowed the
  memory only (anonymous) sharing only for forks of the same process. For 
  anything else
  one needs to pass a regular file descriptor to communicate to mmap from/to 
  where to get/write
  the data. Newer years brought the function shm_open 
  (http://linux.die.net/man/3/shm_open),
  which creates an entry in /dev/shm if I got this right, and allows 
  forwarding this fd for
  a complete in memory-only communication with a (pseudo-)file-mapping mmap.
 
  In today's BOINC code, mmap is called with a file descriptor created with 
  open (no
  typo, it is from boinc/lib/std_fixes.h), which itself calles open64 as 
  defined in
  /usr/include/fcntl.h (?) and expects to create a regular file.
 
  I admit not to know too much about the consequences of a memory-only 
  communication for BOINC.
  It is not more than a couple of megs every second indicating the status of 
  the applications,
  right? So not too much memory would be taken. With checkpointing performed 
  independently,
  this could then work just fine. Is there something I did not see? I am 
  otherwise much tempted
  to address it and see how far I get. Some extra thinking is required to 
  maintain a
  compatibility between the BOINC client and older statically linked 
  applications. With Debian
  we have the applications dynamically linking to the same BOINC library as 
  the BOINC client.
  Promising enough? Please direct me.
 
  Cheers,
 
  Steffen
  ___
  boinc_dev mailing list
  boinc_dev@ssl.berkeley.edu
  http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
  To unsubscribe, visit the above URL and
  (near bottom of page) enter your email address.
 
 ___
 boinc_dev mailing list

[boinc_dev] opendir() failures saga continues

[Steffen Möller]

Dear all,

It took three days of rather intense compute and the opendir() error 
resurfaced. I (now :-) ) agree with David that my past patches would not have 
any effect ... that it is the older statically linked clients causing the 
problem. The application run was mostly rosetta, this time. Nobody else seeing 
this? I get it in irregular intervals on all three BOINC-long-running machines. 
The three days it took now are exceptionally short.

My next attempt will be to get the problem to surface with a SETI rebuilt 
against our libboinc.

Cheers,

Steffen

$ tail /var/lib/boinc-client/stderrdae.txt
dir_open: Could not open directory 'slots/0' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/7' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/2' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/1' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/4' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/8' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/9' from '/var/lib/boinc-client'.
dir_open: Could not open directory 'slots/10' from '/var/lib/boinc-client'.
md5_file: can't open 
projects/boinc.bakerlab.org_rosetta/arm_6_13_highAlaWt_radius_11.5_rise_7.80_omega_0.6.pdb_abinitio_SAVE_ALL_OUT_86915_71_0_0
md5_file: Too many open files


$ manual_Magic /var/lib/boinc-client/stdoutdae.txt

13-Jun-2013 15:40:22 [---] Config: GUI RPCs allowed from:
13-Jun-2013 15:40:22 [---] Version change (7.0.65 - 7.1.10)
[...]

16-Jun-2013 15:02:41 [rosetta@home] Started download of boincTB3_data.zip
16-Jun-2013 15:02:44 [World Community Grid] Finished upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_1
16-Jun-2013 15:02:44 [World Community Grid] Finished upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_2
16-Jun-2013 15:02:44 [World Community Grid] Started upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_3
16-Jun-2013 15:02:44 [World Community Grid] Started upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_4
16-Jun-2013 15:02:45 [World Community Grid] Finished upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_3
16-Jun-2013 15:02:50 [rosetta@home] Finished download of boincTB3_data.zip
16-Jun-2013 15:02:50 [rosetta@home] Starting task 
boincTB3_abinitio_SAVE_ALL_OUT_86705_8779_0 using minirosetta version 346 in 
slot 9
16-Jun-2013 15:02:54 [World Community Grid] Finished upload of 
E213941_003_A.34.C28H15NOS3Se.108.0.set1d06_0_4
16-Jun-2013 16:53:35 [rosetta@home] Can't get task disk usage: opendir() failed
[many lines with the same information not shown]
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:03:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:13:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 [rosetta@home] Can't get task disk usage: opendir() failed
16-Jun-2013 18:23:40 

Re: [boinc_dev] Can we do shared memory with no disk usage?

[Steffen Möller]

Hello,
 
I presume nobody wants to have the user play too much with any folders. I liked 
the all on a ramdisk idea, but BOINC occupies more than 10GB on the 
machine(s) in question, and it will similarly ask for that much (too much with 
today's common 16 or 24 GB setups when adding the memory for the apps) also for 
other setups.

The checkpointing interval I understood (asked around) to be ignored by the 
client apps of a quite a few projects. Mine is set to some 7200 seconds (two 
hours) and the IO does not decrease. It would not be my prime target for 
optimisation.

Could it be the graphics? We once observed that SETI without configuration for 
graphics is a few %ages faster than the same client with the same compilation 
options but graphics-savvy, even though no graphical client was run. If much IO 
was used for that, this might be an explanation. I cannot test this at the 
moment.

Cheers,

Steffen


Gesendet: Montag, 17. Juni 2013 um 17:20 Uhr
Von: Eric J Korpela korp...@ssl.berkeley.edu
An: Steffen Möller steffen_moel...@gmx.de
Cc: boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
Betreff: Re: [boinc_dev] Can we do shared memory with no disk usage?

We considered using memory mapped files for the checkpoint state information in 
SETI@home, but decided that it was virtually impossible to guarantee 
synchronization on exit.  And, of course, the problem with using a ram disk for 
checkpoints that the checkpoint data disappears if power is lost.

 
But if you want checkpoints to not occur, there is a preference for that...
Tasks checkpoint to disk at most every:   60 seconds 
 
You can set it to 8 hours.  But you'll also want to choose suspend to memory. 
 Of course the output of the app (which depending on the app might be more 
frequent than checkpoints) will still spin up the disks.
 
If you really want to operate from a ram disk, just copy your project directory 
to /dev/shm and link it into /var/lib/boinc/projects.   You'll still need to 
back it up occasionally for those times when the power goes out.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] prepared to accept patches for llvm, i.e. clang, compatibility?

[Steffen Möller]

Hi Charlie,

Have many thanks! I can now also confirm BOINC to build with clang. A couple of 
warnings are left, but those also appear with the gcc.

Cheers,

Steffen

 Gesendet: Donnerstag, 30. Mai 2013 um 03:28 Uhr
 Von: Charlie Fenton charl...@ssl.berkeley.edu
 An: Steffen Möller steffen_moel...@gmx.de
 Cc: boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] prepared to accept patches for llvm, i.e. clang, 
 compatibility?

 Hi Steffen,

 For what it's worth, I have been building BOINC on the Mac using the LLVM 
 GCC 4.2 compiler for quite some time with no problems.  (This is one of two 
 compilers provided as part of Xcode 4.4.1, the other is Apple LLVM Compiler 
 4.0.)

 According to Apple's documentation:
  In Xcode, the LLVM compiler uses the Clang front end (a C-based languages 
  project on LLVM.org) to parse source code and turn it into an interim 
  format. Then the LLVM code generation layer (back end) turns that interim 
  format into final machine code. Xcode also includes the LLVM GCC compiler, 
  which uses the GCC compiler front end for maximum compatibility, and the 
  LLVM back end, which takes advantage of LLVM's advanced code generator.


 Cheers,
 --Charlie

 On May 29, 2013, at 1:02 AM, Steffen Möller wrote:

  Hello,
 
  The compiler LLVM (http://en.wikipedia.org/wiki/Clang) makes continuous 
  progress, both technically and for its acceptance in the community, and is 
  faster than gcc/g++ in some cases. They have a gcc-compatibility frontent 
  called clang, which after a funcountable number of patches/fun I 
  could run on BOINC already about a year or two ago. Also shipping are tools 
  for code analysis which may be of general interest - to the development of 
  BOINC but possibly even more so for the developers of the scientific 
  applications.
 
  There is an effort by Debian to recompile the whole archive with clang, 
  http://clang.debian.net/ , completely independent of this request. It gives 
  an overview of issues typically found. From what I recall, for BOINC there 
  was nothing that IMHO could not possibly be tolerated, even when there is 
  no gain in functionality, and nothing that would change the API. I would 
  volunteer to redo that past effort, if there are some voices from the 
  community, especially David's, to appreciate it. My driving force is less 
  on the benefits for the BOINC client than about the extra confidence for 
  the application developers to just go and try that alternative to g++ that 
  they have heard so much about but just not got around to run.
 
  Cheers,
 
  Steffen
  ___
  boinc_dev mailing list
  boinc_dev@ssl.berkeley.edu
  http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
  To unsubscribe, visit the above URL and
  (near bottom of page) enter your email address.
 


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

[boinc_dev] prepared to accept patches for llvm, i.e. clang, compatibility?

[Steffen Möller]

Hello,

The compiler LLVM (http://en.wikipedia.org/wiki/Clang) makes continuous 
progress, both technically and for its acceptance in the community, and is 
faster than gcc/g++ in some cases. They have a gcc-compatibility frontent 
called clang, which after a funcountable number of patches/fun I could 
run on BOINC already about a year or two ago. Also shipping are tools for code 
analysis which may be of general interest - to the development of BOINC but 
possibly even more so for the developers of the scientific applications.

There is an effort by Debian to recompile the whole archive with clang, 
http://clang.debian.net/ , completely independent of this request. It gives an 
overview of issues typically found. From what I recall, for BOINC there was 
nothing that IMHO could not possibly be tolerated, even when there is no gain 
in functionality, and nothing that would change the API. I would volunteer to 
redo that past effort, if there are some voices from the community, especially 
David's, to appreciate it. My driving force is less on the benefits for the 
BOINC client than about the extra confidence for the application developers to 
just go and try that alternative to g++ that they have heard so much about but 
just not got around to run.

Cheers,

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] could the git comment be more descriptive or better empty ?

[Steffen Möller]

Hi Toralf,

 
 Yeah - I was just grumbling about a commit with the only message
 
   Include cmath instead of math.h various places
 
 nothing else - and of course gitk shows for a dozen of files such a
 change. Every ?!%%$$ user can see the diff using gitk - so important
 IMO is just a short notice why cmath.h rules over math.h

http://stackoverflow.com/questions/8734230/math-interface-vs-cmath-in-c

Did you run in to any problems because of that change? A discussion about
cmath being better or worse than math.h for BOINC I would rather want
to see here than with the commit. 

Cheers,

Steffen
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

[boinc_dev] [patch] .po files with issues

[Steffen Möller]

Hello,

Debian performs a routine check on the .po files in all its packages. The one 
of BOINC 
http://i18n.debian.org/l10n-pkg-status/b/boinc.html
indicates a few issues. I then ran
msgfmt -c -o /dev/null po file
as indicated and for de.po prepared the attached patch.

Cheers,

Steffen

Index: boinc_debian/html/languages/translations/de.po
===
--- boinc_debian.orig/html/languages/translations/de.po
+++ boinc_debian/html/languages/translations/de.po
@@ -1066,8 +1066,8 @@
 
 #: ../inc/prefs.inc:157
 #, php-format
-msgid % of the processors
-msgstr % von den Prozessoren
+msgid %% of the processors
+msgstr %% von den Prozessoren
 
 #: ../inc/prefs.inc:161
 msgid Use at most %1 Can be used to reduce CPU heat %2
@@ -1075,8 +1075,8 @@
 
 #: ../inc/prefs.inc:166
 #, php-format
-msgid % of CPU time
-msgstr % der Prozessorzeit
+msgid %% of CPU time
+msgstr %% der Prozessorzeit
 
 #: ../inc/prefs.inc:174 ../inc/prefs.inc:188
 msgid Disk: use at most
@@ -1095,8 +1095,8 @@
 #: ../inc/prefs.inc:190 ../inc/prefs.inc:200 ../inc/prefs.inc:205
 #: ../inc/prefs.inc:210
 #, php-format
-msgid % of total
-msgstr % von Gesamt
+msgid %% of total
+msgstr %% von Gesamt
 
 #: ../inc/prefs.inc:193
 msgid Tasks checkpoint to disk at most every
@@ -5279,9 +5279,10 @@
 our message boards are moderated.\n
 Message board postings are subject to the following posting rules:\n
 msgstr 
+\n
 Um eine angenehme Diskussion und den bestmöglichen Informationsfluss zu 
-gewährleisten wird dieses Forum moderiert. Forenbeiträge müssen den 
-folgenden Regeln entsprechen:
+gewährleisten, wird dieses Forum moderiert. Forenbeiträge müssen den 
+folgenden Regeln entsprechen:\n
 
 #: ../user/moderation.php:30
 msgid 
Index: boinc_debian/html/languages/translations/ru.po
===
--- boinc_debian.orig/html/languages/translations/ru.po
+++ boinc_debian/html/languages/translations/ru.po
@@ -386,6 +386,7 @@
 li Не должно быть оскорбительных комментариев, затрагивающих расу, 
 религию,\n
  национальность, пол, класс или сексуальность.\n
+
 
 #: ../inc/forum.inc:734
 msgid Rules:
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [patch] mystery solved? Re: BOINC having too many open files - failure in opendir()

[Steffen Möller]

Hello,

 Gesendet: Sonntag, 19. Mai 2013 um 05:34 Uhr
 Von: David Anderson da...@ssl.berkeley.edu
 An: boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] [patch] mystery solved? Aw: Re: BOINC having too 
 many open files - failure in opendir()

 As far as I can tell, none of these changes would fix
 a file descriptor leak in the client.

The MFILE class today I read as

class A {
 FILE *f;
 A::A(string fname) {
   f=fopen(fname);
 }
 A::close{
   fclose(f);
 }
 A::~A() {
   // not closing the file pointer
 }
}

which leaks a file pointer with every creation of an object of class A that 
does not see a close(). For MFILE this is in lib/gui_rpc_client.cpp Please have 
a second look at it and kindly ignore those 'const' changes ... I did not 
notice adding them :o)   By grepping through the code I agree that all 
instances of the BOINC client that use MFILE with a filename do indeed perform 
the close.

I can emperically now confirm that the attached patch does not fix the issue. 
When now bringing the boinc client up again, the first thing the client does is 
to upload results. The machine is otherwise idle, no idea whom else to blame.

Steffen



 We need a system-call trace that shows open()s.

 It's also possible that the system is running out of file descriptors
 because of software other than BOINC.

 -- David

 On 18-May-2013 4:00 AM, Steffen Möller wrote:
  Dear all,
 
  I skimmed through all invocations of (boinc_)?fopen() in api/ and lib/,
  seeking the respective matching fclose(). What I found missing I placed here
  http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob;f=debian/patches/fopen_closing.patch;hb=HEAD
 
 
 as a patch. The trickiest and possibly the most important one is the omission 
 of
 a close in the destructor of the MFILE class.
 
  Cheers,
 
  Steffen
 
 
  Gesendet: Freitag, 17. Mai 2013 um 22:00 Uhr Von: Nicolás Alvarez
  nicolas.alva...@gmail.com An: boinc_dev@ssl.berkeley.edu
  boinc_dev@ssl.berkeley.edu Betreff: Re: [boinc_dev] BOINC having too many
  open files - failure in opendir()
 
  Get the list of open files (ls -l /proc/$(pidof boinc)/fd) when that
  happens. Does the client die after that last fopen() failure? Maybe you
  could write a script to log the open file list every few minutes.
 
  -- Nicolás
 
  2013/5/16 Steffen Möller steffen_moel...@gmx.de:
  Dear all,
 
  every few months I get an error like the one below (taken from the
  stdoutdae.txt) the report too many open files. This is see for about
  three years on several Linux machines, I only recall such with many cores
  (12 or 24), though, Opterons and Xeons alike. Is anything jumping at you
  where to look?
 
  Cheers,
 
  Steffen
 
  16-May-2013 16:58:33 [World Community Grid] Sending scheduler request: To
  fetch work. 16-May-2013 16:58:33 [World Community Grid] Requesting new
  tasks for CPU 16-May-2013 16:58:36 [World Community Grid] Scheduler
  request completed: got 0 new tasks 16-May-2013 16:58:36 [World Community
  Grid] No tasks sent 16-May-2013 16:58:36 [World Community Grid] No tasks
  are available for The Clean Energy Project - Phase 2 16-May-2013 16:58:36
  [World Community Grid] No tasks are available for the applications you
  have selected. 16-May-2013 16:58:42 [Einstein@Home] Sending scheduler
  request: To fetch work. 16-May-2013 16:58:42 [Einstein@Home] Reporting 4
  completed tasks 16-May-2013 16:58:42 [Einstein@Home] Requesting new tasks
  for CPU 16-May-2013 16:58:46 [Einstein@Home] Scheduler request completed:
  got 1 new tasks 16-May-2013 17:15:53 [Einstein@Home] Sending scheduler
  request: To fetch work. 16-May-2013 17:15:53 [Einstein@Home] Requesting
  new tasks for CPU 16-May-2013 17:15:56 [Einstein@Home] Scheduler request
  completed: got 1 new tasks 16-May-2013 17:30:11 [World Community Grid]
  Can't get task disk usage: opendir() failed 16-May-2013 17:30:11
  [Einstein@Home] Can't get task disk usage: opendir() failed 16-May-2013
  17:30:11 [Einstein@Home] Can't get task disk usage: opendir() failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir()
  failed 16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage:
  opendir() failed 16-May-2013 17:30:11 [Einstein@Home] Can't get task disk
  usage: opendir() failed 16-May-2013 17:30:11 [Einstein@Home] Can't get
  task disk usage: opendir() failed 16-May-2013 17:30:11 [Einstein@Home]
  Can't get task disk usage: opendir() failed 16-May-2013 17:30:11
  [Einstein@Home] Can't get task disk usage: opendir() failed 16-May-2013
  17:32:31 [Einstein@Home] read_stderr_file(): malloc() failed 16-May-2013
  17:32:31 [Einstein@Home] Computation for task
  LATeah0024U_80.0_500_-4.66e-10_1 finished 16-May-2013 17:32:31
  [Einstein@Home] md5_file failed for
  projects/einstein.phys.uwm.edu/einstein_S6BucketLVE_1.04_i686-pc-linux-gnu__SSE2:
  fopen() failed 16-May-2013 17:32:31 [---] Can't open
  client_state_next.xml: fopen() failed 16-May-2013 17:32:31 [---] Couldn't
  write state file

[boinc_dev] [patch] mystery solved? Aw: Re: BOINC having too many open files - failure in opendir()

[Steffen Möller]

Dear all,

I skimmed through all invocations of (boinc_)?fopen() in api/ and lib/, seeking 
the respective matching fclose().
What I found missing I placed here
http://anonscm.debian.org/gitweb/?p=pkg-boinc/boinc.git;a=blob;f=debian/patches/fopen_closing.patch;hb=HEAD
as a patch. The trickiest and possibly the most important one is the omission 
of a close in the destructor of the MFILE class.

Cheers,

Steffen


 Gesendet: Freitag, 17. Mai 2013 um 22:00 Uhr
 Von: Nicolás Alvarez nicolas.alva...@gmail.com
 An: boinc_dev@ssl.berkeley.edu boinc_dev@ssl.berkeley.edu
 Betreff: Re: [boinc_dev] BOINC having too many open files - failure in 
 opendir()

 Get the list of open files (ls -l /proc/$(pidof boinc)/fd) when that
 happens. Does the client die after that last fopen() failure? Maybe
 you could write a script to log the open file list every few minutes.

 --
 Nicolás

 2013/5/16 Steffen Möller steffen_moel...@gmx.de:
  Dear all,
 
  every few months I get an error like the one below (taken from the 
  stdoutdae.txt) the report too many open files. This is see for about three 
  years on several Linux machines, I only recall such with many cores (12 or 
  24), though, Opterons and Xeons alike. Is anything jumping at you where to 
  look?
 
  Cheers,
 
  Steffen
 
  16-May-2013 16:58:33 [World Community Grid] Sending scheduler request: To 
  fetch work.
  16-May-2013 16:58:33 [World Community Grid] Requesting new tasks for CPU
  16-May-2013 16:58:36 [World Community Grid] Scheduler request completed: 
  got 0 new tasks
  16-May-2013 16:58:36 [World Community Grid] No tasks sent
  16-May-2013 16:58:36 [World Community Grid] No tasks are available for The 
  Clean Energy Project - Phase 2
  16-May-2013 16:58:36 [World Community Grid] No tasks are available for the 
  applications you have selected.
  16-May-2013 16:58:42 [Einstein@Home] Sending scheduler request: To fetch 
  work.
  16-May-2013 16:58:42 [Einstein@Home] Reporting 4 completed tasks
  16-May-2013 16:58:42 [Einstein@Home] Requesting new tasks for CPU
  16-May-2013 16:58:46 [Einstein@Home] Scheduler request completed: got 1 new 
  tasks
  16-May-2013 17:15:53 [Einstein@Home] Sending scheduler request: To fetch 
  work.
  16-May-2013 17:15:53 [Einstein@Home] Requesting new tasks for CPU
  16-May-2013 17:15:56 [Einstein@Home] Scheduler request completed: got 1 new 
  tasks
  16-May-2013 17:30:11 [World Community Grid] Can't get task disk usage: 
  opendir() failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:30:11 [Einstein@Home] Can't get task disk usage: opendir() 
  failed
  16-May-2013 17:32:31 [Einstein@Home] read_stderr_file(): malloc() failed
  16-May-2013 17:32:31 [Einstein@Home] Computation for task 
  LATeah0024U_80.0_500_-4.66e-10_1 finished
  16-May-2013 17:32:31 [Einstein@Home] md5_file failed for 
  projects/einstein.phys.uwm.edu/einstein_S6BucketLVE_1.04_i686-pc-linux-gnu__SSE2:
   fopen() failed
  16-May-2013 17:32:31 [---] Can't open client_state_next.xml: fopen() failed
  16-May-2013 17:32:31 [---] Couldn't write state file: fopen() failed; 
  giving up
 ___
 boinc_dev mailing list
 boinc_dev@ssl.berkeley.edu
 http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
 To unsubscribe, visit the above URL and
 (near bottom of page) enter your email address.
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] I: Several security issues in ops html directory

[Steffen Möller]

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.