Re: [boinc_dev] Software development and branches, was Re: [boinc_projects] keywords

[James Wanless]

Hi All,
I can see (good) arguments both ways - though now that the pace of  
development is slowing, and the number of projects growing/projected  
to grow, I would tend to err on the side of the 'stable' master-branch  
model.
As regards a testing method, I could maybe suggest there is a specific  
shared, scratch, testing project, (with one fairly simple app), to  
which all the github developers would have root access. The default  
state of this at any one time would be the 'stable' master, but  
developers could 'book it out' to try out any developmetal changes  
they were doing. The _users_ of said project would have to understand  
that their credit/accounts could disappear at any time! (But maybe  
this is what Seti@Home beta/YAFU already do, intended to do, at least  
to some extent??)

J

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Christian Beer]

On 21.07.2017 06:21, David Anderson wrote:
> Stable branches for server and API: yes.
> (prerequisite to this is a way of thoroughly testing these components,
> which we don't currently have.
> How about if people start discussing this?)

I don't see having thorough tests as a prerequisite to having a stable
branch. What matters is that we are working toward making a branch
stable by finding ways to test the code. Testing the code is a technical
issue to be solved. Declaring what branches there should be is a
management issue. Both are independent from each other. We can decide
what branches to use for what purpose before we implement tests. In fact
I think it would be easier to make the decision about the branching
model up front and then gradually implement tests on the stable
branches. After some time we then declare the stable branch to be
stable. It does not need to be stable from the beginning.

I already made several attempts to discuss using testing frameworks in
BOINC. There was no resonance at the time. What I want BOINC to do is to
make an informed decision what framework to use before starting to
implement actual tests are even begin to implement a custom test
framework. What I also want to prevent from the start is that I'm the
only user of this test framework. If BOINC is going to get a test
framework their must be a decision to make sure that every new feature
needs to have accompanying tests (that are derived from a reference or
are the reference themselves). When fixing bugs it should also be made
mandatory to check if an automated test would have caught this bug and
then provide such a test alongside the bugfix. These simple rules, when
enforced, make sure that at least over time the code coverage of tests
will increase.

>
> Separate branch for TBD: no.
> TBD has its own completely separate repo.
> It requires some features in BOINC (such as keywords)
> which are of general utility beyond TDB.

The features that TBD needs in BOINC should go into a feature branch and
once they are tested they are merged into master. This is to ensure that
if a project decides to update their server code while this new feature
is developed they don't get a not-production ready feature built into
the scheduler when they checkout the master branch. This is, as I see
it, the case right now and this is what sparked my initial message.
There are other developers working on BOINC at the same time and we
somehow need to coordinate our work and how to integrate it into BOINC.
With git and github we have powerful tools available that help us in
this regard and that help us automate a lot of the stuff that needs
doing we just need to embrace them and use them.

Regards
Christian
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Bernd Machenschalk]

Hi David!

On 2017-07-21 00:50, David Anderson wrote:
This discussion comes down to two contrasting models for software 
development:


1) The "waterfall model":

...

2) The "agile model":
https://en.wikipedia.org/wiki/Agile_software_development
New features are divided into smaller self-contained pieces,
which are completed and merged with master frequently.
Requirements and design can change on the fly.
Development need not be done in a separate branch.


The "agile model" (as referenced by you) doesn't say anything about SCM 
or branches. However, collaborative agile development requires and 
implies doing basically all development (bugfixes, features, etc.) in 
branches (see e.g. https://www.atlassian.com/agile/branching) to isolate 
development, until a development is complete and merged back into a 
'master' branch (that may or may not be called like that).



Master is unstable in both models; releases are done from stable branches.


Where do you get that from?


2) I often work on several things at once.
If I do each one in a separate branch,
I inevitably commit code to the wrong branch,
and it's hard to untangle things later.


When working on many things in parallel, working in branches even helps 
me to stay structured and focused. There are tools to keep track of 
which branch you are currently on, and ways to move commits from one 
branch to another, if you accidentally committed to the wrong one.



3) I want my code reviewed and tested by others ASAP.
I have only Win7 and Linux computers.
If my code doesn't work on other platforms, I want to know this ASAP.
I want to commit to master in small-size, frequent units.


That doesn't require using a single branch ('master' or other). 
Continuous Integration (like Travis on github) can and does test 
(virtually) every branch on every configured platform. Everyone can 
check out any branch and test this, without being affected by any 
problems introduced by other development (in master) he doesn't care about.



4) It's worked extremely well so far.
BOINC has evolved rapidly and continuously over 15 years.
It's still very malleable and well-structured, making it easy to 
change and add.


Apparently it worked for you quite well. However the way BOINC has been 
developed in the past years is causing a lot of volunteer developers 
quite some headache and trouble, up to the point where they quit 
contributing. This is exactly why it needs to be improved.


Best,
Bernd

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[David Wallom]

Indeed we have drifted from the original keywords discussion already (

David

--
===
Professor David Wallom
Associate Professor
Oxford eResearch Centre
University of Oxford
7 Keble Road
Oxford
OX1 3QG
UK

Tel: 01865 610601
===


___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Oliver Bock]

On 21/07/17 10:26 , David Wallom wrote:
> Another thing I would like to introduce (at risk of a large back lash
> ;) is the type of open source license currently used for BOINC.

Please let's use a different thread and/or the workshop to discuss this.

Thanks :-)



smime.p7s
Description: S/MIME Cryptographic Signature
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[David Wallom]

Hi,

If we are to build a much larger community of developers from within the 
current larger users of BOINC then we are going to have to adopt best practice 
processes for collaborative software development. As such this for example is 
one of the things that the Workshop 2017 is to focus on. Unlike previous WS 
which were more generalized the first day is going to specifically have 
sessions for the discussion on how BOINC as a community moves forward, removing 
points of weakness due to the current small developer community and hopefully 
in the process distribute the responsibility for functions to different 
community groups. As such it will be essential that we move to a multi branch 
development methodology in some form of public repository. I use this in a 
large number of my current international projects and it works well.

 Another thing I would like to introduce (at risk of a large back lash ;) is 
the type of open source license currently used for BOINC. We have had 
approaches from non-academic parties who would like to use BOINC but who for 
legal reasons cannot without significant additional work due to its current 
license. Moving from LGPL would be beneficial.

David

--
===
Professor David Wallom
Associate Professor
Oxford eResearch Centre
University of Oxford
7 Keble Road
Oxford
OX1 3QG
UK

Tel: 01865 610601
===


On 21/07/2017, 08:58, "boinc_dev on behalf of Steffen Möller" 
 wrote:

Don't know how much it is worth, but as a part-time contributor to
_many_ Open Source projects (and once having worked with a computer
science degree as a software engineer for a software company that does
software environments for software development) I would like to express
my wholehearted support for Oliver's position.

Quick summary: Developments in branches that branch from master and are
merged back to master once the development has completed. Strategic
adjustments have an influence on the developments that are started but
have no direct effect on the master branch. Of course you can always
fork or name master something else. But you always have something stable
to branch your new developments from.

Steffen


On 21.07.17 09:49, Oliver Bock wrote:
> Hi David,
>
> On 21/07/17 0:50 , David Anderson wrote:
>> This discussion comes down to two contrasting models for software
>> development:
> Sorry, no, that's not the point.
>
>> 1) The "waterfall model":
>> 2) The "agile model":
> I know both models very well, professionally and scientifically.
>
>> Requirements and design can change on the fly.
>> New features are divided into smaller self-contained pieces,
>> which are completed and merged with master frequently. 
> Yes, totally right.
>
> (master serving as integration branch in this case)
>
>> Development need not be done in a separate branch.
> Where does this conclusion come from? To the contrary: agile
> development, due to its velocity, needs separate feature/topic/fix
> branches because many developers need to work on the same codebase at
> the same time, without interfering with each other. One needs
> coordination (by workflow). Also, developers need a stable base to start
> off from, not something that's broken most of the times. How do you want
> to achieve that when everyone develops (let alone tests) in the same
> branch? How do you want to do integration tests without affecting
> development or release maintenance?
>
> Please ask yourself why tools like git, global agile development and
> particular SCM workflows like GitFlow more or less arrived and spread
> massively at the same time. Mere correlation or actual causality...?
>
>> Oliver prefers the waterfall model,
> The opposite is true. I'm all for agile development like Scrum and I'm
> even evangelizing to do so wherever I go (and see fit). Why have I tried
> for years to convince BOINC to move to git and make use of Continuous
> Integration for instance? Because I prefer the waterfall model?
>
>> so strongly that he won't work with people who use agile.
> This all leaves me kind of speechless. I can't even fathom how one can
> reach a conclusion like that, based on what I've described. In
> particular since SCM (that's all we talked about in this thread) is just
> one part of implementing a process model instance, it doesn't define it.
>
> There are obviously severe misunderstandings at play here and I'm
> honestly starting to doubt they can be overcome - unless other projects
> step up and chime in as well.
>
>> I prefer the agile model for several reasons:
> All those reasons/motivations (first sentence of each given bullet) are
> absolutely fine. Unfortunately the conclusions/consequences reached are
> wrong. What I and others proposed *will help you* with what you're doing
> and trying to achieve as it's at the heart of agile development. So far
> we've just failed to show 

Re: [boinc_dev] [boinc_projects] keywords

[Steffen Möller]

Don't know how much it is worth, but as a part-time contributor to
_many_ Open Source projects (and once having worked with a computer
science degree as a software engineer for a software company that does
software environments for software development) I would like to express
my wholehearted support for Oliver's position.

Quick summary: Developments in branches that branch from master and are
merged back to master once the development has completed. Strategic
adjustments have an influence on the developments that are started but
have no direct effect on the master branch. Of course you can always
fork or name master something else. But you always have something stable
to branch your new developments from.

Steffen


On 21.07.17 09:49, Oliver Bock wrote:
> Hi David,
>
> On 21/07/17 0:50 , David Anderson wrote:
>> This discussion comes down to two contrasting models for software
>> development:
> Sorry, no, that's not the point.
>
>> 1) The "waterfall model":
>> 2) The "agile model":
> I know both models very well, professionally and scientifically.
>
>> Requirements and design can change on the fly.
>> New features are divided into smaller self-contained pieces,
>> which are completed and merged with master frequently. 
> Yes, totally right.
>
> (master serving as integration branch in this case)
>
>> Development need not be done in a separate branch.
> Where does this conclusion come from? To the contrary: agile
> development, due to its velocity, needs separate feature/topic/fix
> branches because many developers need to work on the same codebase at
> the same time, without interfering with each other. One needs
> coordination (by workflow). Also, developers need a stable base to start
> off from, not something that's broken most of the times. How do you want
> to achieve that when everyone develops (let alone tests) in the same
> branch? How do you want to do integration tests without affecting
> development or release maintenance?
>
> Please ask yourself why tools like git, global agile development and
> particular SCM workflows like GitFlow more or less arrived and spread
> massively at the same time. Mere correlation or actual causality...?
>
>> Oliver prefers the waterfall model,
> The opposite is true. I'm all for agile development like Scrum and I'm
> even evangelizing to do so wherever I go (and see fit). Why have I tried
> for years to convince BOINC to move to git and make use of Continuous
> Integration for instance? Because I prefer the waterfall model?
>
>> so strongly that he won't work with people who use agile.
> This all leaves me kind of speechless. I can't even fathom how one can
> reach a conclusion like that, based on what I've described. In
> particular since SCM (that's all we talked about in this thread) is just
> one part of implementing a process model instance, it doesn't define it.
>
> There are obviously severe misunderstandings at play here and I'm
> honestly starting to doubt they can be overcome - unless other projects
> step up and chime in as well.
>
>> I prefer the agile model for several reasons:
> All those reasons/motivations (first sentence of each given bullet) are
> absolutely fine. Unfortunately the conclusions/consequences reached are
> wrong. What I and others proposed *will help you* with what you're doing
> and trying to achieve as it's at the heart of agile development. So far
> we've just failed to show you how.
>
> Apart from that there are other developers, projects and contributors
> (existing and prospective) who depend on how BOINC's SCM is done. I
> think we agree that their needs need to be taken into account as well,
> even if you have the impression that their needs are at odds with yours
> - which might not even be the case.
>
>> Also, let's not confuse the choice of model with how we do stable branches.
> Precisely!
>
>
> HTH,
> Oliver
>
>
>
> ___
> boinc_dev mailing list
> boinc_dev@ssl.berkeley.edu
> https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
> To unsubscribe, visit the above URL and
> (near bottom of page) enter your email address.

___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.

Re: [boinc_dev] [boinc_projects] keywords

[Oliver Bock]

Hi David,

On 21/07/17 0:50 , David Anderson wrote:
> This discussion comes down to two contrasting models for software
> development:

Sorry, no, that's not the point.

> 1) The "waterfall model":
> 2) The "agile model":

I know both models very well, professionally and scientifically.

> Requirements and design can change on the fly.
> New features are divided into smaller self-contained pieces,
> which are completed and merged with master frequently. 

Yes, totally right.

(master serving as integration branch in this case)

> Development need not be done in a separate branch.

Where does this conclusion come from? To the contrary: agile
development, due to its velocity, needs separate feature/topic/fix
branches because many developers need to work on the same codebase at
the same time, without interfering with each other. One needs
coordination (by workflow). Also, developers need a stable base to start
off from, not something that's broken most of the times. How do you want
to achieve that when everyone develops (let alone tests) in the same
branch? How do you want to do integration tests without affecting
development or release maintenance?

Please ask yourself why tools like git, global agile development and
particular SCM workflows like GitFlow more or less arrived and spread
massively at the same time. Mere correlation or actual causality...?

> Oliver prefers the waterfall model,

The opposite is true. I'm all for agile development like Scrum and I'm
even evangelizing to do so wherever I go (and see fit). Why have I tried
for years to convince BOINC to move to git and make use of Continuous
Integration for instance? Because I prefer the waterfall model?

> so strongly that he won't work with people who use agile.

This all leaves me kind of speechless. I can't even fathom how one can
reach a conclusion like that, based on what I've described. In
particular since SCM (that's all we talked about in this thread) is just
one part of implementing a process model instance, it doesn't define it.

There are obviously severe misunderstandings at play here and I'm
honestly starting to doubt they can be overcome - unless other projects
step up and chime in as well.

> I prefer the agile model for several reasons:

All those reasons/motivations (first sentence of each given bullet) are
absolutely fine. Unfortunately the conclusions/consequences reached are
wrong. What I and others proposed *will help you* with what you're doing
and trying to achieve as it's at the heart of agile development. So far
we've just failed to show you how.

Apart from that there are other developers, projects and contributors
(existing and prospective) who depend on how BOINC's SCM is done. I
think we agree that their needs need to be taken into account as well,
even if you have the impression that their needs are at odds with yours
- which might not even be the case.

> Also, let's not confuse the choice of model with how we do stable branches.

Precisely!


HTH,
Oliver



smime.p7s
Description: S/MIME Cryptographic Signature
___
boinc_dev mailing list
boinc_dev@ssl.berkeley.edu
https://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev
To unsubscribe, visit the above URL and
(near bottom of page) enter your email address.