Re: Checking for problems before committing

[Joshua Root]

On 2016-3-11 08:41 , David Evans wrote:

On 3/10/16 12:29 PM, Ryan Schmidt wrote:


On Mar 10, 2016, at 12:48 AM, Mojca Miklavec wrote:


On 10 March 2016 at 05:48, Ryan Schmidt wrote:


Obviously nobody is going to commit something they believe is broken, but it 
does sometimes end up being the case for some subset of users. When it does, 
and we learn that it has happened, we try to fix it. If everybody had to test 
everything on a clean system on every version of OS X and every version of 
Xcode before committing, nobody would ever commit anything because nobody has 
the time and resources to do all that testing. We do have automated build 
machines that do build every commit on a clean system with no ports installed 
with several versions of OS X and the latest version of Xcode for those 
systems. That automated system did catch this webkit2-gtk build problem on El 
Capitan,



When I was testing wxWidgets, discovered a problem and submitted a
patch, I noticed what they are doing now (which is some light years
more advanced compared to what they did a few years back when most of
the tickets were stuck ignored at their Trac; similar to what happens
in many cases in MacPorts).

- user submits a patch
- the system checks whether the patch applies cleanly
- the system automatically builds wxWidgets on Windows in 6 different
ways (cygwin, mingw32 with msys, mingw, nmake with VS 14, nmake with
VS 9, msbuild), on Linux in five different ways (one is with clang
3.5), and on OS X 10.9
- I'm not sure whether wxWidgets does it as well, but it is also
possible to run tests as part of the build

The developers then only apply the patch if all of the checks mentioned pass.

The point is that this is all done *in advance* and avoids a lot of
problems. I would love to see something similar being done for patches
submitted to our Trac. Of course they would have to be submitted in a
different way and I'm aware that this is not really trivial to set up.
But it would be extremely helpful.


Yes, this would be helpful. I intend to look into doing something like this. 
However right now and for the next several weeks there are a lot of other more 
pressing issues I need to be working on for Mac OS Forge.


One approach I have thought about is this:

Establish a "testing" instance of the port repository and have maintainers 
commit to it.  The buildbots would then build
from changes to this repo and, upon a successful build, move (copy, merge, whatever) the 
port to the "live" repository.
The rsync server would publish from the "live" repository.


I've thought about this too. One issue I can see is that a successful 
port update can still break dependents (see openssl). So unless you 
rebuild all dependents on every commit, you still don't have any sort of 
guarantee that the ports in the live repo work at any given time.



This is not a detailed proposal but you get the idea and I think could be 
implemented without too much work. The plus is
that it would be relatively automated without too many false positives.  Exception cases 
are ports that "fail" building
but are considered "good."

   * ports that "fail" because they require non-default dependencies (+quartz 
is one class, gtk-osx-application for instance)
   * ports that "fail" because they are meant to (obsolete ports, graveyards)
   * you can probably think of more

Not a really original idea but at least it would be some filter between 
maintainer commits and what the user gets


Another approach would be to add a 'try' scheduler to our buildbot config.



- Josh
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Checking for problems before committing

[David Evans]

On 3/10/16 12:29 PM, Ryan Schmidt wrote:
> 
> On Mar 10, 2016, at 12:48 AM, Mojca Miklavec wrote:
> 
>> On 10 March 2016 at 05:48, Ryan Schmidt wrote:
>>>
>>> Obviously nobody is going to commit something they believe is broken, but 
>>> it does sometimes end up being the case for some subset of users. When it 
>>> does, and we learn that it has happened, we try to fix it. If everybody had 
>>> to test everything on a clean system on every version of OS X and every 
>>> version of Xcode before committing, nobody would ever commit anything 
>>> because nobody has the time and resources to do all that testing. We do 
>>> have automated build machines that do build every commit on a clean system 
>>> with no ports installed with several versions of OS X and the latest 
>>> version of Xcode for those systems. That automated system did catch this 
>>> webkit2-gtk build problem on El Capitan,
>>
>>
>> When I was testing wxWidgets, discovered a problem and submitted a
>> patch, I noticed what they are doing now (which is some light years
>> more advanced compared to what they did a few years back when most of
>> the tickets were stuck ignored at their Trac; similar to what happens
>> in many cases in MacPorts).
>>
>> - user submits a patch
>> - the system checks whether the patch applies cleanly
>> - the system automatically builds wxWidgets on Windows in 6 different
>> ways (cygwin, mingw32 with msys, mingw, nmake with VS 14, nmake with
>> VS 9, msbuild), on Linux in five different ways (one is with clang
>> 3.5), and on OS X 10.9
>> - I'm not sure whether wxWidgets does it as well, but it is also
>> possible to run tests as part of the build
>>
>> The developers then only apply the patch if all of the checks mentioned pass.
>>
>> The point is that this is all done *in advance* and avoids a lot of
>> problems. I would love to see something similar being done for patches
>> submitted to our Trac. Of course they would have to be submitted in a
>> different way and I'm aware that this is not really trivial to set up.
>> But it would be extremely helpful.
> 
> Yes, this would be helpful. I intend to look into doing something like this. 
> However right now and for the next several weeks there are a lot of other 
> more pressing issues I need to be working on for Mac OS Forge.

One approach I have thought about is this:

Establish a "testing" instance of the port repository and have maintainers 
commit to it.  The buildbots would then build
from changes to this repo and, upon a successful build, move (copy, merge, 
whatever) the port to the "live" repository.
The rsync server would publish from the "live" repository.

This is not a detailed proposal but you get the idea and I think could be 
implemented without too much work. The plus is
that it would be relatively automated without too many false positives.  
Exception cases are ports that "fail" building
but are considered "good."

  * ports that "fail" because they require non-default dependencies (+quartz is 
one class, gtk-osx-application for instance)
  * ports that "fail" because they are meant to (obsolete ports, graveyards)
  * you can probably think of more

Not a really original idea but at least it would be some filter between 
maintainer commits and what the user gets

Dave




___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Checking for problems before committing

[Ryan Schmidt]

On Mar 10, 2016, at 12:48 AM, Mojca Miklavec wrote:

> On 10 March 2016 at 05:48, Ryan Schmidt wrote:
>> 
>> Obviously nobody is going to commit something they believe is broken, but it 
>> does sometimes end up being the case for some subset of users. When it does, 
>> and we learn that it has happened, we try to fix it. If everybody had to 
>> test everything on a clean system on every version of OS X and every version 
>> of Xcode before committing, nobody would ever commit anything because nobody 
>> has the time and resources to do all that testing. We do have automated 
>> build machines that do build every commit on a clean system with no ports 
>> installed with several versions of OS X and the latest version of Xcode for 
>> those systems. That automated system did catch this webkit2-gtk build 
>> problem on El Capitan,
> 
> 
> When I was testing wxWidgets, discovered a problem and submitted a
> patch, I noticed what they are doing now (which is some light years
> more advanced compared to what they did a few years back when most of
> the tickets were stuck ignored at their Trac; similar to what happens
> in many cases in MacPorts).
> 
> - user submits a patch
> - the system checks whether the patch applies cleanly
> - the system automatically builds wxWidgets on Windows in 6 different
> ways (cygwin, mingw32 with msys, mingw, nmake with VS 14, nmake with
> VS 9, msbuild), on Linux in five different ways (one is with clang
> 3.5), and on OS X 10.9
> - I'm not sure whether wxWidgets does it as well, but it is also
> possible to run tests as part of the build
> 
> The developers then only apply the patch if all of the checks mentioned pass.
> 
> The point is that this is all done *in advance* and avoids a lot of
> problems. I would love to see something similar being done for patches
> submitted to our Trac. Of course they would have to be submitted in a
> different way and I'm aware that this is not really trivial to set up.
> But it would be extremely helpful.

Yes, this would be helpful. I intend to look into doing something like this. 
However right now and for the next several weeks there are a lot of other more 
pressing issues I need to be working on for Mac OS Forge.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Checking for problems before committing

[Brandon Allbery]

On Thu, Mar 10, 2016 at 1:48 AM, Mojca Miklavec  wrote:

>
> When I was testing wxWidgets, discovered a problem and submitted a
> patch, I noticed what they are doing now (which is some light years
> more advanced compared to what they did a few years back when most of
> the tickets were stuck ignored at their Trac; similar to what happens
> in many cases in MacPorts).


Much of this now comes "for free" with github these days; just create a
data file in the repo with config information and add some "canned" commit
triggers. Apple/MacOSForge has the infrastructure but not the connections
you'd need to automate it quickly and cleanly.

(No, moving to github would not help us. They can build on El Cap only ---
and that service often simply disappears since Travis CI only offers the
free github service when they have spare capacity, and unlike the other
platforms they must use dedicated Apple hardware for the build slaves --
this due to Apple licensing. So there's a limited number of OS X
buildslaves, and if they're all in use for the commercial Travis customers
the free service on OS X gets dropped.)

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Checking for problems before committing

[Bachsau]

> Am 10.03.2016 um 07:48 schrieb Mojca Miklavec :
> 
> The point is that this is all done *in advance* and avoids a lot of
> problems. I would love to see something similar being done for patches
> submitted to our Trac. Of course they would have to be submitted in a
> different way and I'm aware that this is not really trivial to set up.
> But it would be extremely helpful.

Well, I think that would be a great way to handle submissions. As MacPorts 
already has automated build systems able to check if a build succeeds, it might 
be not that hard to implement. I think its most important a port is always 
working on the latest OS version, which currently is El Capitan. However, when 
I opened this thread I wasn't aware there's a ticket system. Now posted a 
ticket with all the requested information for libbonoboui which also fails to 
build on my system with a similar error. Lets see what happens…

smime.p7s
Description: S/MIME cryptographic signature
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Checking for problems before committing

[Mojca Miklavec]

(Was: Why do so many builds fail? @ macports-users)

On 10 March 2016 at 05:48, Ryan Schmidt wrote:
>
> Obviously nobody is going to commit something they believe is broken, but it 
> does sometimes end up being the case for some subset of users. When it does, 
> and we learn that it has happened, we try to fix it. If everybody had to test 
> everything on a clean system on every version of OS X and every version of 
> Xcode before committing, nobody would ever commit anything because nobody has 
> the time and resources to do all that testing. We do have automated build 
> machines that do build every commit on a clean system with no ports installed 
> with several versions of OS X and the latest version of Xcode for those 
> systems. That automated system did catch this webkit2-gtk build problem on El 
> Capitan,


When I was testing wxWidgets, discovered a problem and submitted a
patch, I noticed what they are doing now (which is some light years
more advanced compared to what they did a few years back when most of
the tickets were stuck ignored at their Trac; similar to what happens
in many cases in MacPorts).

- user submits a patch
- the system checks whether the patch applies cleanly
- the system automatically builds wxWidgets on Windows in 6 different
ways (cygwin, mingw32 with msys, mingw, nmake with VS 14, nmake with
VS 9, msbuild), on Linux in five different ways (one is with clang
3.5), and on OS X 10.9
- I'm not sure whether wxWidgets does it as well, but it is also
possible to run tests as part of the build

The developers then only apply the patch if all of the checks mentioned pass.

The point is that this is all done *in advance* and avoids a lot of
problems. I would love to see something similar being done for patches
submitted to our Trac. Of course they would have to be submitted in a
different way and I'm aware that this is not really trivial to set up.
But it would be extremely helpful.

Mojca
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev