Re: [Msys2-users] could msys2 have a "stable" repository?

[Daniel Goldman]

Thank you for explaining more. To me, what you are doing sounds very 
useful and helpful. I have never tried to build / compile msys2 
packages, so I really don't understand the issues, except to know there 
are a lot of complexities. I just use msys2 to build external Windows 
software, and to carry out various system tasks.


Anyway, from my experience, a system should always (or almost always) 
compile correctly. A possible name to use is "smoke test", a set of 
scripts that automatically build everything and verify as much as 
possible correct operation. If the system does not compile correctly, it 
will likely "rot", getting more disorganized and difficult to maintain 
over time. Maybe what you are doing is also related to creating or 
improving the "smoke test" for msys2. In any case, no matter what it is 
called, thanks for what you are doing.


On 6/6/2017 2:31 AM, Mario Emmenlauer wrote:


Dear Daniel,

thanks for the thumbs up! See below for more:

On 05.06.2017 23:59, Daniel Goldman wrote:

I really appreciate the work you are doing.

Could you clarify difference between 'stable' and 'testing'? I thought 'stable' 
means 'archived release' and guarantees that everything works, and 'testing'
means 'development release' and perhaps includes new things not guaranteed to 
work. 'stable' and 'testing' do not seem synonyms to me. I could be totally 
mixed
up about this.

IMO, having a system that someone can download and know will work correctly, if 
that is possible, seems pretty useful.


There was some discussion about this topic in autumn 2016. When I
started using MSYS2 and the MINGW-packages git repo, I instantly fell
in love! Its by far the best effort on Windows software development
I've seen so far. But at the time, I experienced quite often that
packages failed to build. It could very well happen that the version
of a package X that was included in the repo could not be compiled
with the current snapshot of the repo. I find this a fair policy of
the "release fast and release often" policy. But it made my life
quite difficult whenever I needed to modify an MSYS2 package - there
was a good chance I would not be able to build my changes :-)

Since then, I've come to realize that the term "stable" has quite a
different meaning to different people. In Debian terminology, stable
is something that builds, tests, and is proven by a large number of
real-world users for a long time. What I try to achieve with my
effort is exclusively the following:
  - to have a well-defined snapshot of packages in specific versions
  - where all included packages guarantee to build against themselves,
  - where all prerequisite packages of included packages are also
included, and
  - where all enabled tests work, if any


So with that snapshot, you can be certain that you can recompile any
included package over and over, and the build should never fail. And
you can rebuild all prerequisites too. Additionally, I run a triple
bootstrap procedure: first the full toolchain is rebuild with the last
stable toolchain, then the full toolchain is rebuild with itself, and
then all packages are rebuild with the new toolchain. So all packages
will use the same headers, same gcc version, etc.


I would like to stress one important thing: My intention is that the
above testing procedure will identify build-issues in package inter-
dependencies that the current CI system does not pick up. I will report
those issues upstream and try to aid in fixing them (as much as time
permits). With this additional level of testing I hope that MSYS2
becomes more stable, and my binary repository is not needed at all!
Producing binaries is just a side effect of my CI effort, and not my
goal. My preferred outcome is to have procedures in place that allow
to flag breakages in the git repo early on, so they can be fixed
directly at the source.

All the best,

Mario






I can only speak for myself. I find msys2 quite useful. I mainly need something 
that works correctly, not necessarily the newest version of everything. I
installed msys2 several years ago. Aside from a few non-fatal glitches that I 
reported, everything has worked fine, and I have not attempted to do any 
updates.

Daniel

On 6/5/2017 8:12 AM, Mario Emmenlauer wrote:


Dear All,

I just wanted to let you know that I have not given up on the idea
of a stable (better named "testing") repository for MSYS2. My goal
is to have in regular, short intervals a fully bootstrapped rebuild
of all MSYS2 MinGW packages (based on a current whitelist). With
this effort I will try to isolate packages that are broken in such a
way that the current CI does not pick up. I.e. the current CI tests
that any package X will build correctly. But it does not test that
other packages depending on X still build against the updated X.
My CI will do this. The cost is a much longer build time. So instead
of triggering builds by commit, I will trigger builds with a timer.
The current build interval is ~30hrs,

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear Daniel,

thanks for the thumbs up! See below for more:

On 05.06.2017 23:59, Daniel Goldman wrote:
> I really appreciate the work you are doing.
> 
> Could you clarify difference between 'stable' and 'testing'? I thought 
> 'stable' means 'archived release' and guarantees that everything works, and 
> 'testing'
> means 'development release' and perhaps includes new things not guaranteed to 
> work. 'stable' and 'testing' do not seem synonyms to me. I could be totally 
> mixed
> up about this.
> 
> IMO, having a system that someone can download and know will work correctly, 
> if that is possible, seems pretty useful.

There was some discussion about this topic in autumn 2016. When I
started using MSYS2 and the MINGW-packages git repo, I instantly fell
in love! Its by far the best effort on Windows software development
I've seen so far. But at the time, I experienced quite often that
packages failed to build. It could very well happen that the version
of a package X that was included in the repo could not be compiled
with the current snapshot of the repo. I find this a fair policy of
the "release fast and release often" policy. But it made my life
quite difficult whenever I needed to modify an MSYS2 package - there
was a good chance I would not be able to build my changes :-)

Since then, I've come to realize that the term "stable" has quite a
different meaning to different people. In Debian terminology, stable
is something that builds, tests, and is proven by a large number of
real-world users for a long time. What I try to achieve with my
effort is exclusively the following:
 - to have a well-defined snapshot of packages in specific versions
 - where all included packages guarantee to build against themselves,
 - where all prerequisite packages of included packages are also
   included, and
 - where all enabled tests work, if any


So with that snapshot, you can be certain that you can recompile any
included package over and over, and the build should never fail. And
you can rebuild all prerequisites too. Additionally, I run a triple
bootstrap procedure: first the full toolchain is rebuild with the last
stable toolchain, then the full toolchain is rebuild with itself, and
then all packages are rebuild with the new toolchain. So all packages
will use the same headers, same gcc version, etc.


I would like to stress one important thing: My intention is that the
above testing procedure will identify build-issues in package inter-
dependencies that the current CI system does not pick up. I will report
those issues upstream and try to aid in fixing them (as much as time
permits). With this additional level of testing I hope that MSYS2
becomes more stable, and my binary repository is not needed at all!
Producing binaries is just a side effect of my CI effort, and not my
goal. My preferred outcome is to have procedures in place that allow
to flag breakages in the git repo early on, so they can be fixed
directly at the source.

All the best,

   Mario





> I can only speak for myself. I find msys2 quite useful. I mainly need 
> something that works correctly, not necessarily the newest version of 
> everything. I
> installed msys2 several years ago. Aside from a few non-fatal glitches that I 
> reported, everything has worked fine, and I have not attempted to do any 
> updates.
> 
> Daniel
> 
> On 6/5/2017 8:12 AM, Mario Emmenlauer wrote:
>>
>> Dear All,
>>
>> I just wanted to let you know that I have not given up on the idea
>> of a stable (better named "testing") repository for MSYS2. My goal
>> is to have in regular, short intervals a fully bootstrapped rebuild
>> of all MSYS2 MinGW packages (based on a current whitelist). With
>> this effort I will try to isolate packages that are broken in such a
>> way that the current CI does not pick up. I.e. the current CI tests
>> that any package X will build correctly. But it does not test that
>> other packages depending on X still build against the updated X.
>> My CI will do this. The cost is a much longer build time. So instead
>> of triggering builds by commit, I will trigger builds with a timer.
>> The current build interval is ~30hrs, but this may go up significantly
>> with a growing whitelist.
>>
>>
>> So in the past six months I set up the build system with dependency
>> tracking. Its not perfect but it should be quite ok. In 99.9% of the
>> cases I can identify the list of all recursive dependencies of an
>> MSYS2 MinGW package. I also have a small whitelist set up from which
>> I will start. Since a few months all this is working on a subset of
>> packages.
>>
>> But in my whitelist, I'd like to include Qt and VTK since both are
>> very relevant for me. And I'd like to make a release only when *all*
>> included packages can be successfully compiled against themselves. So
>> any failing package from the whitelist or the recursive prerequisites
>> is a blocker. As of now I can still not build mingw-w64-SDL2, mingw-
>> w64-icu and mingw-w64-firebird2-

Re: [Msys2-users] could msys2 have a "stable" repository?

[Daniel Goldman]

I really appreciate the work you are doing.

Could you clarify difference between 'stable' and 'testing'? I thought 
'stable' means 'archived release' and guarantees that everything works, 
and 'testing' means 'development release' and perhaps includes new 
things not guaranteed to work. 'stable' and 'testing' do not seem 
synonyms to me. I could be totally mixed up about this.


IMO, having a system that someone can download and know will work 
correctly, if that is possible, seems pretty useful.


I can only speak for myself. I find msys2 quite useful. I mainly need 
something that works correctly, not necessarily the newest version of 
everything. I installed msys2 several years ago. Aside from a few 
non-fatal glitches that I reported, everything has worked fine, and I 
have not attempted to do any updates.


Daniel

On 6/5/2017 8:12 AM, Mario Emmenlauer wrote:


Dear All,

I just wanted to let you know that I have not given up on the idea
of a stable (better named "testing") repository for MSYS2. My goal
is to have in regular, short intervals a fully bootstrapped rebuild
of all MSYS2 MinGW packages (based on a current whitelist). With
this effort I will try to isolate packages that are broken in such a
way that the current CI does not pick up. I.e. the current CI tests
that any package X will build correctly. But it does not test that
other packages depending on X still build against the updated X.
My CI will do this. The cost is a much longer build time. So instead
of triggering builds by commit, I will trigger builds with a timer.
The current build interval is ~30hrs, but this may go up significantly
with a growing whitelist.


So in the past six months I set up the build system with dependency
tracking. Its not perfect but it should be quite ok. In 99.9% of the
cases I can identify the list of all recursive dependencies of an
MSYS2 MinGW package. I also have a small whitelist set up from which
I will start. Since a few months all this is working on a subset of
packages.

But in my whitelist, I'd like to include Qt and VTK since both are
very relevant for me. And I'd like to make a release only when *all*
included packages can be successfully compiled against themselves. So
any failing package from the whitelist or the recursive prerequisites
is a blocker. As of now I can still not build mingw-w64-SDL2, mingw-
w64-icu and mingw-w64-firebird2-git. I've reported the SDL2-issue
upstream a long while ago and the problem is isolated, but the fix
is not in yet. icu is undergoing a discussion as I am writing this
email. And firebird2 is broken since more than six months but it has
not received too much attention yet.


Cutting a long story short: hopefully soon there should be fixes for
the three missing packages, and then I can make a first "testing"
release. From there I'd like to gather user feedback. I'd also like to
gather requests for further whitelisted packages.

All the best,

 Mario



On 30.08.2016 10:16, Mario Emmenlauer wrote:


I was wondering if people would like to have a "stable" repository?

I was thinking that when a plateau is reached (majority of packages
compiles fine), the current snapshot could be copied to stable. In my
eyes, the essential requirement would be that all base packages compile
fine. Everything else could be handled a bit ad-hoc.

All the best, Mario





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear All,

I just wanted to let you know that I have not given up on the idea
of a stable (better named "testing") repository for MSYS2. My goal
is to have in regular, short intervals a fully bootstrapped rebuild
of all MSYS2 MinGW packages (based on a current whitelist). With
this effort I will try to isolate packages that are broken in such a
way that the current CI does not pick up. I.e. the current CI tests
that any package X will build correctly. But it does not test that
other packages depending on X still build against the updated X.
My CI will do this. The cost is a much longer build time. So instead
of triggering builds by commit, I will trigger builds with a timer.
The current build interval is ~30hrs, but this may go up significantly
with a growing whitelist.


So in the past six months I set up the build system with dependency
tracking. Its not perfect but it should be quite ok. In 99.9% of the
cases I can identify the list of all recursive dependencies of an
MSYS2 MinGW package. I also have a small whitelist set up from which
I will start. Since a few months all this is working on a subset of
packages.

But in my whitelist, I'd like to include Qt and VTK since both are
very relevant for me. And I'd like to make a release only when *all*
included packages can be successfully compiled against themselves. So
any failing package from the whitelist or the recursive prerequisites
is a blocker. As of now I can still not build mingw-w64-SDL2, mingw-
w64-icu and mingw-w64-firebird2-git. I've reported the SDL2-issue
upstream a long while ago and the problem is isolated, but the fix
is not in yet. icu is undergoing a discussion as I am writing this
email. And firebird2 is broken since more than six months but it has
not received too much attention yet.


Cutting a long story short: hopefully soon there should be fixes for
the three missing packages, and then I can make a first "testing"
release. From there I'd like to gather user feedback. I'd also like to
gather requests for further whitelisted packages.

All the best,

Mario



On 30.08.2016 10:16, Mario Emmenlauer wrote:
> 
> I was wondering if people would like to have a "stable" repository?
> 
> I was thinking that when a plateau is reached (majority of packages
> compiles fine), the current snapshot could be copied to stable. In my
> eyes, the essential requirement would be that all base packages compile
> fine. Everything else could be handled a bit ad-hoc.
> 
> All the best, Mario
> 



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Chris Johns]

On 25/02/2017 21:03, Ray Donnelly wrote:
> It's entirely possible for someone to make a non rolling distribution
> from the msys2 repos. You'd set up your own repos and advise people to
> switch out our rolling ones for your ones. For it to be appealing you'd
> probably want to add some comprehensive tests that all the package
> versions you have selected work really well together.
>
> I somewhat like the idea (*) but it's not something I expect msys2 would
> do as we're doing it rolling as we consider it the best way, occasional
> breakages and all, and don't have the resources to commit to this
> unrolling variant.
>
> (*) Somewhat because if a significant number of people switched to it
> and away from upstream msys2 then bugs would get found less quickly and
> msys2 upstream would be less good.

I do not think this is the experience of other projects that provide 
releases as well as a current (your rolling release), for example Linux 
distros, the FreeBSD kernel and RTEMS which I am involved with. I 
suspect at the moment a lot of users avoid updating something that is 
working because it may break. If a newer version of package is only 
available on current or a later release users need to update.

For the RTEMS Project we would welcome a `stable` view of MSYS2 because 
it lets us specify something we know works and have tested against. At 
the moment we cannot do this via the MSYS2 project and I have resisted 
creating an RTEMS release of MSYS2 with our own repo because I do not 
want to become a project on its own. An RTEMS user deploying on Windows 
needs to create a local MSYS2 install or they may end up with a range of 
different MSYS2 installs depending on when they build each development 
workstation and if their project is for space flight this needs to be 
controlled and audited.

 From the responses so far there looks to be a need and I can be 
included on that list. With nothing to fill this role those who need 
this each have to separately create something similar which fragments 
the effort. It would be nice to have any efforts brought in under the 
MSYS2 project and governed so the quality that current exists can continue.

Which ever way this ends up I will respect the wishes of you and the 
others who run the MSYS2 project. It is a really great tool and I again 
thank you.

Chris

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear Ray,

I see your point here! Also about the users power in testing. But
I would like to add that, for what its worth, my original main goal
is to actually aid in msys2 testing in stability improvements.

I think the best value I can add is via another angle for the CI
system. Where the current CI has a focus on building only the modified
packages (to quickly get a response), I would like to iteratively
bootstrap the full system from ground up, in the hope that this
will uncover problems in inter-dependencies that might otherwise go
unnoticed. As an example, I have until now failed to build a full
toolchain when starting from a plain vanilla msys2 installation.
This might be due to issues on my side, but at least in some cases
I could already provide new insight. Furthermore, I think on every
toolchain update, it would be very helpful to test that the new
toolchain builds at least as many packages as the previous did (or
it better have a good excuse for not doing so :-)

In my humble opinion this is a very valuable test, and also some-
thing I can actually provide. For what its worth, this could already
uncover some problems that could be fixed and pushed upstream. I
hope this might address at least some concerns?
In any case I would not want to hinder msys development. And all of
the praise should go to the developers!!! Thanks for the awesome
msys2!!!

Cheers,

   Mario






On 25.02.2017 11:03, Ray Donnelly wrote:
> It's entirely possible for someone to make a non rolling distribution from 
> the msys2 repos. You'd set up your own repos and advise people to switch out 
> our
> rolling ones for your ones. For it to be appealing you'd probably want to add 
> some comprehensive tests that all the package versions you have selected work
> really well together.
> 
> I somewhat like the idea (*) but it's not something I expect msys2 would do 
> as we're doing it rolling as we consider it the best way, occasional 
> breakages and
> all, and don't have the resources to commit to this unrolling variant.
> 
> (*) Somewhat because if a significant number of people switched to it and 
> away from upstream msys2 then bugs would get found less quickly and msys2 
> upstream
> would be less good.
> 
> On Feb 24, 2017 4:53 PM, "Daniel Goldman"  > wrote:
> 
> Just my two bits. As a user, I would like to have a "tagged release",
> similar to how I run ubuntu LTS server for years before upgrading.
> 
> I run msys2 behind a firewall, so the security updates seem not
> important. If my firewall is penetrated, everything on my computer would
> be affected. I'm looking for a "stable, reproducible work environment".
> I would probably not use "rolling releases".
> 
> Of course, it would depend on the size of the distributions, but for
> most people, I believe saving disk space is no longer a big concern.
> 
> I have not run into any of the incompatibilities or problems that at
> least a few people seem to have reported. Perhaps that is because I have
> not updated anything for several years?
> 
> Daniel
> 
> On 2/24/2017 3:13 AM, Mario Emmenlauer wrote:
> >
> > Dear Matthew Postiff and Daniel Goldman,
> >
> > This is actually a valuable input that I did not consider before! This
> > would enable you to ensure a fixed environment as required i.e. for 
> build
> > and test systems, or for a shared work environment. While I can see good
> > arguments for rolling releases (security updates etc), I can also see
> > good benefit in tagged releases (stable, reproducible work environment).
> >
> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Msys2-users mailing list
> Msys2-users@lists.sourceforge.net 
> 
> https://lists.sourceforge.net/lists/listinfo/msys2-users 
> 
> 
> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> 
> 
> 
> ___
> Msys2-users mailing list
> Msys2-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/msys2-users
> 



Viele Gruesse,

Mario Emmenlauer


--
BioDataAnalysis GmbH, Mario Emmenlauer  Tel. Buero: +49-89-74677203
Balanstr. 43   mailto: memmenlauer * biodataanalysis.de
D-81669 München  http://www.biodataanalysis.de/

--
Check out the vibrant tech community on one of the world's most
engaging tech sites,

Re: [Msys2-users] could msys2 have a "stable" repository?

[Ray Donnelly]

It's entirely possible for someone to make a non rolling distribution from
the msys2 repos. You'd set up your own repos and advise people to switch
out our rolling ones for your ones. For it to be appealing you'd probably
want to add some comprehensive tests that all the package versions you have
selected work really well together.

I somewhat like the idea (*) but it's not something I expect msys2 would do
as we're doing it rolling as we consider it the best way, occasional
breakages and all, and don't have the resources to commit to this unrolling
variant.

(*) Somewhat because if a significant number of people switched to it and
away from upstream msys2 then bugs would get found less quickly and msys2
upstream would be less good.

On Feb 24, 2017 4:53 PM, "Daniel Goldman"  wrote:

> Just my two bits. As a user, I would like to have a "tagged release",
> similar to how I run ubuntu LTS server for years before upgrading.
>
> I run msys2 behind a firewall, so the security updates seem not
> important. If my firewall is penetrated, everything on my computer would
> be affected. I'm looking for a "stable, reproducible work environment".
> I would probably not use "rolling releases".
>
> Of course, it would depend on the size of the distributions, but for
> most people, I believe saving disk space is no longer a big concern.
>
> I have not run into any of the incompatibilities or problems that at
> least a few people seem to have reported. Perhaps that is because I have
> not updated anything for several years?
>
> Daniel
>
> On 2/24/2017 3:13 AM, Mario Emmenlauer wrote:
> >
> > Dear Matthew Postiff and Daniel Goldman,
> >
> > This is actually a valuable input that I did not consider before! This
> > would enable you to ensure a fixed environment as required i.e. for build
> > and test systems, or for a shared work environment. While I can see good
> > arguments for rolling releases (security updates etc), I can also see
> > good benefit in tagged releases (stable, reproducible work environment).
> >
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Msys2-users mailing list
> Msys2-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/msys2-users
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[philippe renon]

Just to say that providing some way to stabilize msys2 would be great.At 
LibrePilot, we switched to using msys2 and won't go back. Back to what is hard 
to say. Not much I guess...Thanks to msys2, the windows build system for 
LibrePilot is closer to what's available on linux and osx.
We use mostly Qt and a few third parties (osg, osgearth, soon gstreamer).

Msys2 is great but the rate of change (and breakage) is a problem.We tend to 
stay long times without updating the main branch while trying out updates 
regularly until things fall in place.
Again, msys2 is great and makes things so much easier, but the stability issue 
is a problem on the long run.Anything that will help in this area is more than 
welcome. 

Le Vendredi 24 février 2017 17h52, Daniel Goldman  a 
écrit :
 

 Just my two bits. As a user, I would like to have a "tagged release", 
similar to how I run ubuntu LTS server for years before upgrading.

I run msys2 behind a firewall, so the security updates seem not 
important. If my firewall is penetrated, everything on my computer would 
be affected. I'm looking for a "stable, reproducible work environment". 
I would probably not use "rolling releases".

Of course, it would depend on the size of the distributions, but for 
most people, I believe saving disk space is no longer a big concern.

I have not run into any of the incompatibilities or problems that at 
least a few people seem to have reported. Perhaps that is because I have 
not updated anything for several years?

Daniel

On 2/24/2017 3:13 AM, Mario Emmenlauer wrote:
>
> Dear Matthew Postiff and Daniel Goldman,
>
> This is actually a valuable input that I did not consider before! This
> would enable you to ensure a fixed environment as required i.e. for build
> and test systems, or for a shared work environment. While I can see good
> arguments for rolling releases (security updates etc), I can also see
> good benefit in tagged releases (stable, reproducible work environment).
>

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users


   --
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Daniel Goldman]

Just my two bits. As a user, I would like to have a "tagged release", 
similar to how I run ubuntu LTS server for years before upgrading.

I run msys2 behind a firewall, so the security updates seem not 
important. If my firewall is penetrated, everything on my computer would 
be affected. I'm looking for a "stable, reproducible work environment". 
I would probably not use "rolling releases".

Of course, it would depend on the size of the distributions, but for 
most people, I believe saving disk space is no longer a big concern.

I have not run into any of the incompatibilities or problems that at 
least a few people seem to have reported. Perhaps that is because I have 
not updated anything for several years?

Daniel

On 2/24/2017 3:13 AM, Mario Emmenlauer wrote:
>
> Dear Matthew Postiff and Daniel Goldman,
>
> This is actually a valuable input that I did not consider before! This
> would enable you to ensure a fixed environment as required i.e. for build
> and test systems, or for a shared work environment. While I can see good
> arguments for rolling releases (security updates etc), I can also see
> good benefit in tagged releases (stable, reproducible work environment).
>

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear Matthew Postiff and Daniel Goldman,

your emails share an idea here (or at least that is my impression),
so I will comment on both in one go:

On 21.02.2017 19:18, Matthew A. Postiff wrote:
> Constant changes in msys2 are also causing me lots of development 
> headaches. I also work with other developers and it is hard for us to 
> "sync" our msys2 working environments.
> 
> Could there be a way to tell pacman the revision of the package you want 
> by date or version tag so that it will check out packages as of that 
> date? Then anyone who runs a command like
> 
> pacman -S <02212017>
> 
> will get all the packages as of that date and no newer ones. It will 
> give them a consistent starting point.

This is actually a valuable input that I did not consider before! This
would enable you to ensure a fixed environment as required i.e. for build
and test systems, or for a shared work environment. While I can see good
arguments for rolling releases (security updates etc), I can also see
good benefit in tagged releases (stable, reproducible work environment).

From a purely technical point of view, I think it should be possible to
combine tagged releases as well as a rolling release. Actually, since the
concept seems straightforward, I am puzzled as to why this is not a default
for ArchLinux. Maybe I am overlooking something? Here the outline: use a
unique identifier "RELEASEVERSION" for each stable release, and store the
release packages in a repository exclusively for this release version. Then
link the updated files into a rolling release repository. For example the
repository could be set up in such a way that you can pull from either
"BASEURL" (to get the rolling release) or to "BASEURL/RELEASEVERSION" to
stick to that specific version. Technically I would use hardlinks or
symlinks such that "BASEURL" gets deduplicated copies of the latest
"BASEURL/RELEASEVERSION" packages. A suitable RELEASEVERSION may be either
a short time stamp or a monotonically increasing release version number.

I'm not a pacman expert, can a more knowledgeable person comment if this
may work? Furthermore to save disk space, it might be very helpful if
the server supports hardlinks or symlinks in order to deduplicate the
files. I'm not certain if the file servers available to this project have
this feature?

All the best,

   Mario



> Matt
> 
> On 2/20/2017 1:28 PM, Adrian Pop wrote:
>> Hi all,
>>
>> I really look forward to a *stable* msys2 repository.
>> Things are moving very fast with msys2 and while that
>> is great for having the latest and greatest it poses
>> problems with stability.
>>
>> Currently, we basically freeze in SVN an installed msys2
>> version that builds (and runs) all our software and then,
>> after half a year or so, we try to build with a newer msys2
>> and update the current version to a new one if everything
>> works out. While this sort of works you don't get the
>> latest developments, bug fixes and so on very quickly.
>>
>> Cheers,
>> Adrian Pop/
>>
>> On 2/20/2017 17:20, David Grayson wrote:
>>> Hello, Mario.
>>>
>>> Currently, I'm working on making a set of recipes using the Nix
>>> functional package manager that will let me compile native software
>>> for Windows.  Nix has its own language for specifying how to build
>>> software and *compositions* of software.  The recipes are typically
>>> stored in a Git repository.  The nixpkgs repository, for instance, has
>>> thousands of recipes for software in it and there are hundreds of
>>> users actively working on it:
>>>
>>> https://github.com/nixos/nixpkgs
>>>
>>> For people desiring stability, with Nix, you don't have to rely on a
>>> central package repository: when you build a piece of software, you
>>> get exactly what was specified in the local Nix recipes on your
>>> computer.  These recipes specify exactly how to build all the
>>> dependencies of your software, including the compiler toolchain.  So,
>>> you will almost always be able to rebuild all of the software you
>>> depend on, as long as you keep track of those recipes (e.g. in Git).
>>>
>>> Of course, it would be very slow to rebuild compiler toolchains every
>>> time you wanted to build a package.  Nix speeds things up for you in
>>> several ways.  First of all, builds are identified with long hash
>>> codes that take into account everything about the build and its
>>> dependencies.  The hash codes are used as a key for a local cache on
>>> your computer called the Nix store (/nix/store/) so you don't have to
>>> build software that you already have.  There is also a centralized
>>> binary cache that is built from the nixpkgs repository so you don't
>>> have to compile a lot of things yourself and can just download them
>>> instead, as long as the download was built in exactly the same way as
>>> the recipe on your computer.
>>>
>>> Because of the hash code, Nix can tell if you change any of the
>>> recipes for the software you depend on.  The next time you build your
>>> softwa

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear David,

your suggestion and pointers are very very interesting, and I
was admittedly completely ignorant of Nix until now! I will give
it a closer look soon, but its something that will require a bit
of learning on my side (or at least a review of my current work
style) :-) :-)

All the best,

   Mario



On 20.02.2017 17:20, David Grayson wrote:
> Hello, Mario.
> 
> Currently, I'm working on making a set of recipes using the Nix
> functional package manager that will let me compile native software
> for Windows.  Nix has its own language for specifying how to build
> software and *compositions* of software.  The recipes are typically
> stored in a Git repository.  The nixpkgs repository, for instance, has
> thousands of recipes for software in it and there are hundreds of
> users actively working on it:
> 
> https://github.com/nixos/nixpkgs
> 
> For people desiring stability, with Nix, you don't have to rely on a
> central package repository: when you build a piece of software, you
> get exactly what was specified in the local Nix recipes on your
> computer.  These recipes specify exactly how to build all the
> dependencies of your software, including the compiler toolchain.  So,
> you will almost always be able to rebuild all of the software you
> depend on, as long as you keep track of those recipes (e.g. in Git).
> 
> Of course, it would be very slow to rebuild compiler toolchains every
> time you wanted to build a package.  Nix speeds things up for you in
> several ways.  First of all, builds are identified with long hash
> codes that take into account everything about the build and its
> dependencies.  The hash codes are used as a key for a local cache on
> your computer called the Nix store (/nix/store/) so you don't have to
> build software that you already have.  There is also a centralized
> binary cache that is built from the nixpkgs repository so you don't
> have to compile a lot of things yourself and can just download them
> instead, as long as the download was built in exactly the same way as
> the recipe on your computer.
> 
> Because of the hash code, Nix can tell if you change any of the
> recipes for the software you depend on.  The next time you build your
> software, Nix will know to rebuild the changed package and everything
> that depends on it.  Compared to pacman, this causes lots of unneeded
> rebuilding, but it gives us a lot more confidence that the final
> software will work correctly, instead of failing because some DLL name
> changed or a library's ABI changed.
> 
> Since Nix does not run in Windows, I would be cross-compiling from
> Linux to Windows.  Nix could probably work inside the Windows
> Subsystem for Linux, which would allow us to easily run test suites
> for the software we compile, but I have not looked in to that.
> 
> The nixpkgs repository itself has support for cross-compiling to
> Windows but I get the impression that it's not very good, because the
> main toolchain does not even have winpthreads, and thus things like
> std::future do not work.  Ericson2314 on GitHub has made a bunch of
> pull requests to clean up the architecture for cross-compiling
> recently.  But the way that they build GCC and other things in nixpkgs
> is so complicated that I think I'd like to try writing my own
> expressions.  I have succeeded in making a mingw-w64 cross-compiler
> and compiling some console apps for Windows, like the pdcurses demos.
> My recipes are here:
> 
> https://github.com/DavidEGrayson/nixcrpkgs
> 
> Feel free to email me privately if you want to try out nixcrpkgs and
> want some help getting set up.
> 
> --David Grayson
> 
> On Wed, Sep 21, 2016 at 3:04 AM, Mario Emmenlauer  wrote:
>>
>> Hi,
>>
>> anything I can do to get started? Any pointers how your current
>> continuous integration system works? Somebody mentioned that the
>> CI on github is broken a while ago, is that still true?
>>
>> Cheers, Mario
>>
>>
>> On 30.08.2016 13:27, Mario Emmenlauer wrote:
>>> On 30.08.2016 13:06, Alexpux wrote:
> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
> написал(а):
>
> I was wondering if people would like to have a "stable" repository?
>
> I was thinking that when a plateau is reached (majority of packages
> compiles fine), the current snapshot could be copied to stable. In my
> eyes, the essential requirement would be that all base packages compile
> fine. Everything else could be handled a bit ad-hoc.


 Hi, Mario!
 If someone want to support «stable» repo - we can talk about it.
 There are some problem from my POV to support this.
 1. Who will decide that package is stable?
 2. You can’t just copy builded package from current repository and place 
 it «stable». You must rebuild it agains «stable» set of packages.
 3. Who will maintain all of this work? I will not do any work with this 
 sorry because my time is limited and now I have many other priorities than 
 MSYS2.
>>>
>>> I ca

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear Alan,

thanks a lot for the many nice pointers! Below more:

On 20.02.2017 03:32, Alan W. Irwin wrote:
> On 2017-02-19 23:08+0100 Mario Emmenlauer wrote:
> 
>>
>> Dear MSYS2 users / developers,
>>
>> I've start working towards a "stable" repository and would like suggestions!
>> Please comment!
> 
> For example, Debian (the only distro I have used since 2000) has two
> rolling releases (called "unstable", and "testing"), see
> . A given package gets promoted from
> "unstable" to "testing" only if that package and _all_ its package
> dependencies pass certain criteria (such as a successful build and for
> a fixed time interval after that build a lack of critical bug reports
> from users). That automated promotion system is so effective and
> breakage so rare on "testing" that most Debian desktop users tend to
> use the testing repository for Debian. (Debian stable is another
> story: it is a fixed-release repository that tends to be quite stable,
> but it is based on a very old testing snapshot which is why Debian
> desktop users in need of the latest desktop features tend to avoid
> it.)

I very much like this suggestion. In Debian-terminology, I think my
current effort would not yet qualify for "testing", since I currently
incorporate neither user feedback nor any actual testing beyond the
question "does it compile". However, since I envision to integrate
user feedback in a fashion as you suggest below, I would like to adopt
the name "testing", at least until somebody proposes something better.
For the sake of this email conversation, I might still refer to the
term "stable" since I used it in my original subject, but the actual
releases should be termed "testing".


> I haven't used ArchLinux, but it is an interesting distro from the
> MinGW-w64/MSYS2 perspective because pacman (before being ported for
> use by MinGW-w64/MSYS2) was originally developed for ArchLinux.  The
> repositories for that free software distro are summarized at
> .  From
> the description there, ArchLinux historically struggled with the
> packaging stability issue, but eventually resolved it by splitting the
> packages into various categories like core (a limited number of
> essential packages which have stringent stability requirements) and
> extra (with less stringent stability requirements).  All ArchLinux
> repositories are rolling releases so they have nothing equivalent to
> Debian stable, but my guess is core + extra have stability that is
> roughly equivalent to Debian "testing" (i.e., very good), and the
> ArchLinux "testing" versions of core and extra have stability roughly
> equivalent to Debian "unstable" and the current MinGW-w64/MSYS2 (i.e.,
> mostly adequate but sometimes broken).

This might be a very good idea. For my current effort it is still too
far-fetched because I want to provide something useful as soon as
possible. But I can clearly see "categories" as a goal for a second
milestone, and this can immensely reduce the pressure on overall
release quality. I think users may accept an issue in a rarely used
downstream package much more easily than an issue in the core tool-
chain.


> Developing a mostly automated system similar to Debian's for promoting
> current MinGW-w64/MSYS2 packages to a more stable rolling release
> version of MinGW-w64/MSYS2 might take quite an effort to implement,
> but in the long run that might be less work than following a system
> similar to the apparent subjective one that ArchLinux has for
> promoting packages in its testing repository to core or extra. So, for
> example, you might go with non-automated/subjective to start and then
> automate it later, but, of course, as the implementer such choices are
> up to you.
> 
> Anyhow, good luck with this potentially valuable project, and I hope
> the URL's I found above concerning the various choices made by other
> distros concerning package stability will be of some help to you.

Thanks to you, and please keep the comments/suggestions coming! :)

All the best,

Mario Emmenlauer


--
BioDataAnalysis GmbH, Mario Emmenlauer  Tel. Buero: +49-89-74677203
Balanstr. 43   mailto: memmenlauer * biodataanalysis.de
D-81669 München  http://www.biodataanalysis.de/

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Matthew A. Postiff]

Constant changes in msys2 are also causing me lots of development 
headaches. I also work with other developers and it is hard for us to 
"sync" our msys2 working environments.

Could there be a way to tell pacman the revision of the package you want 
by date or version tag so that it will check out packages as of that 
date? Then anyone who runs a command like

pacman -S <02212017>

will get all the packages as of that date and no newer ones. It will 
give them a consistent starting point.

Matt

On 2/20/2017 1:28 PM, Adrian Pop wrote:
> Hi all,
>
> I really look forward to a *stable* msys2 repository.
> Things are moving very fast with msys2 and while that
> is great for having the latest and greatest it poses
> problems with stability.
>
> Currently, we basically freeze in SVN an installed msys2
> version that builds (and runs) all our software and then,
> after half a year or so, we try to build with a newer msys2
> and update the current version to a new one if everything
> works out. While this sort of works you don't get the
> latest developments, bug fixes and so on very quickly.
>
> Cheers,
> Adrian Pop/
>
> On 2/20/2017 17:20, David Grayson wrote:
>> Hello, Mario.
>>
>> Currently, I'm working on making a set of recipes using the Nix
>> functional package manager that will let me compile native software
>> for Windows.  Nix has its own language for specifying how to build
>> software and *compositions* of software.  The recipes are typically
>> stored in a Git repository.  The nixpkgs repository, for instance, has
>> thousands of recipes for software in it and there are hundreds of
>> users actively working on it:
>>
>> https://github.com/nixos/nixpkgs
>>
>> For people desiring stability, with Nix, you don't have to rely on a
>> central package repository: when you build a piece of software, you
>> get exactly what was specified in the local Nix recipes on your
>> computer.  These recipes specify exactly how to build all the
>> dependencies of your software, including the compiler toolchain.  So,
>> you will almost always be able to rebuild all of the software you
>> depend on, as long as you keep track of those recipes (e.g. in Git).
>>
>> Of course, it would be very slow to rebuild compiler toolchains every
>> time you wanted to build a package.  Nix speeds things up for you in
>> several ways.  First of all, builds are identified with long hash
>> codes that take into account everything about the build and its
>> dependencies.  The hash codes are used as a key for a local cache on
>> your computer called the Nix store (/nix/store/) so you don't have to
>> build software that you already have.  There is also a centralized
>> binary cache that is built from the nixpkgs repository so you don't
>> have to compile a lot of things yourself and can just download them
>> instead, as long as the download was built in exactly the same way as
>> the recipe on your computer.
>>
>> Because of the hash code, Nix can tell if you change any of the
>> recipes for the software you depend on.  The next time you build your
>> software, Nix will know to rebuild the changed package and everything
>> that depends on it.  Compared to pacman, this causes lots of unneeded
>> rebuilding, but it gives us a lot more confidence that the final
>> software will work correctly, instead of failing because some DLL name
>> changed or a library's ABI changed.
>>
>> Since Nix does not run in Windows, I would be cross-compiling from
>> Linux to Windows.  Nix could probably work inside the Windows
>> Subsystem for Linux, which would allow us to easily run test suites
>> for the software we compile, but I have not looked in to that.
>>
>> The nixpkgs repository itself has support for cross-compiling to
>> Windows but I get the impression that it's not very good, because the
>> main toolchain does not even have winpthreads, and thus things like
>> std::future do not work.  Ericson2314 on GitHub has made a bunch of
>> pull requests to clean up the architecture for cross-compiling
>> recently.  But the way that they build GCC and other things in nixpkgs
>> is so complicated that I think I'd like to try writing my own
>> expressions.  I have succeeded in making a mingw-w64 cross-compiler
>> and compiling some console apps for Windows, like the pdcurses demos.
>> My recipes are here:
>>
>> https://github.com/DavidEGrayson/nixcrpkgs
>>
>> Feel free to email me privately if you want to try out nixcrpkgs and
>> want some help getting set up.
>>
>> --David Grayson
>>
>> On Wed, Sep 21, 2016 at 3:04 AM, Mario Emmenlauer  
>> wrote:
>>> Hi,
>>>
>>> anything I can do to get started? Any pointers how your current
>>> continuous integration system works? Somebody mentioned that the
>>> CI on github is broken a while ago, is that still true?
>>>
>>> Cheers, Mario
>>>
>>>
>>> On 30.08.2016 13:27, Mario Emmenlauer wrote:
 On 30.08.2016 13:06, Alexpux wrote:
>> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
>> написал(а):
>>
>> I was

Re: [Msys2-users] could msys2 have a "stable" repository?

[Adrian Pop]

Hi all,

I really look forward to a *stable* msys2 repository.
Things are moving very fast with msys2 and while that
is great for having the latest and greatest it poses
problems with stability.

Currently, we basically freeze in SVN an installed msys2
version that builds (and runs) all our software and then,
after half a year or so, we try to build with a newer msys2
and update the current version to a new one if everything
works out. While this sort of works you don't get the
latest developments, bug fixes and so on very quickly.

Cheers,
Adrian Pop/

On 2/20/2017 17:20, David Grayson wrote:
> Hello, Mario.
>
> Currently, I'm working on making a set of recipes using the Nix
> functional package manager that will let me compile native software
> for Windows.  Nix has its own language for specifying how to build
> software and *compositions* of software.  The recipes are typically
> stored in a Git repository.  The nixpkgs repository, for instance, has
> thousands of recipes for software in it and there are hundreds of
> users actively working on it:
>
> https://github.com/nixos/nixpkgs
>
> For people desiring stability, with Nix, you don't have to rely on a
> central package repository: when you build a piece of software, you
> get exactly what was specified in the local Nix recipes on your
> computer.  These recipes specify exactly how to build all the
> dependencies of your software, including the compiler toolchain.  So,
> you will almost always be able to rebuild all of the software you
> depend on, as long as you keep track of those recipes (e.g. in Git).
>
> Of course, it would be very slow to rebuild compiler toolchains every
> time you wanted to build a package.  Nix speeds things up for you in
> several ways.  First of all, builds are identified with long hash
> codes that take into account everything about the build and its
> dependencies.  The hash codes are used as a key for a local cache on
> your computer called the Nix store (/nix/store/) so you don't have to
> build software that you already have.  There is also a centralized
> binary cache that is built from the nixpkgs repository so you don't
> have to compile a lot of things yourself and can just download them
> instead, as long as the download was built in exactly the same way as
> the recipe on your computer.
>
> Because of the hash code, Nix can tell if you change any of the
> recipes for the software you depend on.  The next time you build your
> software, Nix will know to rebuild the changed package and everything
> that depends on it.  Compared to pacman, this causes lots of unneeded
> rebuilding, but it gives us a lot more confidence that the final
> software will work correctly, instead of failing because some DLL name
> changed or a library's ABI changed.
>
> Since Nix does not run in Windows, I would be cross-compiling from
> Linux to Windows.  Nix could probably work inside the Windows
> Subsystem for Linux, which would allow us to easily run test suites
> for the software we compile, but I have not looked in to that.
>
> The nixpkgs repository itself has support for cross-compiling to
> Windows but I get the impression that it's not very good, because the
> main toolchain does not even have winpthreads, and thus things like
> std::future do not work.  Ericson2314 on GitHub has made a bunch of
> pull requests to clean up the architecture for cross-compiling
> recently.  But the way that they build GCC and other things in nixpkgs
> is so complicated that I think I'd like to try writing my own
> expressions.  I have succeeded in making a mingw-w64 cross-compiler
> and compiling some console apps for Windows, like the pdcurses demos.
> My recipes are here:
>
> https://github.com/DavidEGrayson/nixcrpkgs
>
> Feel free to email me privately if you want to try out nixcrpkgs and
> want some help getting set up.
>
> --David Grayson
>
> On Wed, Sep 21, 2016 at 3:04 AM, Mario Emmenlauer  wrote:
>>
>> Hi,
>>
>> anything I can do to get started? Any pointers how your current
>> continuous integration system works? Somebody mentioned that the
>> CI on github is broken a while ago, is that still true?
>>
>> Cheers, Mario
>>
>>
>> On 30.08.2016 13:27, Mario Emmenlauer wrote:
>>> On 30.08.2016 13:06, Alexpux wrote:
> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
> написал(а):
>
> I was wondering if people would like to have a "stable" repository?
>
> I was thinking that when a plateau is reached (majority of packages
> compiles fine), the current snapshot could be copied to stable. In my
> eyes, the essential requirement would be that all base packages compile
> fine. Everything else could be handled a bit ad-hoc.


 Hi, Mario!
 If someone want to support «stable» repo - we can talk about it.
 There are some problem from my POV to support this.
 1. Who will decide that package is stable?
 2. You can’t just copy builded package from current repository and place 
 it «stable». 

Re: [Msys2-users] could msys2 have a "stable" repository?

[David Grayson]

Hello, Mario.

Currently, I'm working on making a set of recipes using the Nix
functional package manager that will let me compile native software
for Windows.  Nix has its own language for specifying how to build
software and *compositions* of software.  The recipes are typically
stored in a Git repository.  The nixpkgs repository, for instance, has
thousands of recipes for software in it and there are hundreds of
users actively working on it:

https://github.com/nixos/nixpkgs

For people desiring stability, with Nix, you don't have to rely on a
central package repository: when you build a piece of software, you
get exactly what was specified in the local Nix recipes on your
computer.  These recipes specify exactly how to build all the
dependencies of your software, including the compiler toolchain.  So,
you will almost always be able to rebuild all of the software you
depend on, as long as you keep track of those recipes (e.g. in Git).

Of course, it would be very slow to rebuild compiler toolchains every
time you wanted to build a package.  Nix speeds things up for you in
several ways.  First of all, builds are identified with long hash
codes that take into account everything about the build and its
dependencies.  The hash codes are used as a key for a local cache on
your computer called the Nix store (/nix/store/) so you don't have to
build software that you already have.  There is also a centralized
binary cache that is built from the nixpkgs repository so you don't
have to compile a lot of things yourself and can just download them
instead, as long as the download was built in exactly the same way as
the recipe on your computer.

Because of the hash code, Nix can tell if you change any of the
recipes for the software you depend on.  The next time you build your
software, Nix will know to rebuild the changed package and everything
that depends on it.  Compared to pacman, this causes lots of unneeded
rebuilding, but it gives us a lot more confidence that the final
software will work correctly, instead of failing because some DLL name
changed or a library's ABI changed.

Since Nix does not run in Windows, I would be cross-compiling from
Linux to Windows.  Nix could probably work inside the Windows
Subsystem for Linux, which would allow us to easily run test suites
for the software we compile, but I have not looked in to that.

The nixpkgs repository itself has support for cross-compiling to
Windows but I get the impression that it's not very good, because the
main toolchain does not even have winpthreads, and thus things like
std::future do not work.  Ericson2314 on GitHub has made a bunch of
pull requests to clean up the architecture for cross-compiling
recently.  But the way that they build GCC and other things in nixpkgs
is so complicated that I think I'd like to try writing my own
expressions.  I have succeeded in making a mingw-w64 cross-compiler
and compiling some console apps for Windows, like the pdcurses demos.
My recipes are here:

https://github.com/DavidEGrayson/nixcrpkgs

Feel free to email me privately if you want to try out nixcrpkgs and
want some help getting set up.

--David Grayson

On Wed, Sep 21, 2016 at 3:04 AM, Mario Emmenlauer  wrote:
>
> Hi,
>
> anything I can do to get started? Any pointers how your current
> continuous integration system works? Somebody mentioned that the
> CI on github is broken a while ago, is that still true?
>
> Cheers, Mario
>
>
> On 30.08.2016 13:27, Mario Emmenlauer wrote:
>> On 30.08.2016 13:06, Alexpux wrote:
 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
 написал(а):

 I was wondering if people would like to have a "stable" repository?

 I was thinking that when a plateau is reached (majority of packages
 compiles fine), the current snapshot could be copied to stable. In my
 eyes, the essential requirement would be that all base packages compile
 fine. Everything else could be handled a bit ad-hoc.
>>>
>>>
>>> Hi, Mario!
>>> If someone want to support «stable» repo - we can talk about it.
>>> There are some problem from my POV to support this.
>>> 1. Who will decide that package is stable?
>>> 2. You can’t just copy builded package from current repository and place it 
>>> «stable». You must rebuild it agains «stable» set of packages.
>>> 3. Who will maintain all of this work? I will not do any work with this 
>>> sorry because my time is limited and now I have many other priorities than 
>>> MSYS2.
>>
>> I can maybe do parts of this work. My company has a bit of free
>> resources. But you must help me understand what am I up against?
>> What is your current build system? Is it Windows-based? How many
>> resources does it need? What is the size of the package repository?
>> How much traffic do you currently have?
>>
>> I have one idle build server (4 Core Xeon with 16GB Ram) that can
>> host a build system. I guess "stable" does not change hourly, so
>> maybe one machine is sufficient? My company webserver has 0.5TB 

Re: [Msys2-users] could msys2 have a "stable" repository?

[Daniel Goldman]

FWIW (maybe not much), a few comments:

- msys2 is very useful for me and many others.

- I think a stable repository is *very* helpful, almost required. Many 
users, such as myself, do not need the latest and greatest. We just need 
something that works correctly, even if out of date. We are happy with 
an older stable version. Just like I run a LTS ubuntu release, and only 
upgrade once every few years.

- If the question of how to define a stable repository means which 
packages to include, I would think someone knowledgeable (not me) would 
start somewhere, and if someone else thinks another package should be 
included, then that could be considered. At least whatever is included 
should be supposed to work correctly for sure.

- Maybe I am wrong, but to me a stable repository might mean a 
point-in-time archived form of msys2 that someone could download and 
know that it is supposed to work correctly.

- I am OK with just downloading everything. I like pacman a lot, but 
would not mind just downloading a complete stable release.

- Obviously, I don't know how to create a msys2 stable repository. I am 
sure there are many technical details that need to be figured out and 
automated. I call it a "smoke test" where software can be automatically 
built at any time to verify it works. I'm sure it takes a lot of work to 
automate the needed stable repository.

Daniel

On 2/19/2017 2:08 PM, Mario Emmenlauer wrote:
>
> Dear MSYS2 users / developers,
>
> I've start working towards a "stable" repository and would like suggestions!
> Please comment!
>
>
> On 21.09.2016 12:04, Mario Emmenlauer wrote:
>> On 30.08.2016 13:27, Mario Emmenlauer wrote:
>>> On 30.08.2016 13:06, Alexpux wrote:
> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
> написал(а):
>
> I was wondering if people would like to have a "stable" repository?
>
> I was thinking that when a plateau is reached (majority of packages
> compiles fine), the current snapshot could be copied to stable. In my
> eyes, the essential requirement would be that all base packages compile
> fine. Everything else could be handled a bit ad-hoc.


 Hi, Mario!
 If someone want to support «stable» repo - we can talk about it.
 There are some problem from my POV to support this.
 1. Who will decide that package is stable?
 2. You can’t just copy builded package from current repository and place 
 it «stable». You must rebuild it agains «stable» set of packages.
 3. Who will maintain all of this work? I will not do any work with this 
 sorry because my time is limited and now I have many other priorities than 
 MSYS2.
>
> 1) Definition of "stable" can mean many things, please provide your thoughts!
> In my humble opinion, it would be great if a "stable" repository can fully
> build itself. This is my foremost goal. When I started, I picked some ~60
> packages from the repository but I could successfully build only ~30. This
> was very frustrating in the beginning. I hope to reduce this pain for new
> MSYS2 users.
> Currently when toolchain gets updates, its not transparent for me which
> packages will get broken and which get fixed. So in my eyes, the first
> "stable" release is a toolchain that fully builds itself plus all packages
> that cleanly compile and install. Then changes are accepted into "stable"
> when (a) they do not break any "stable" package, or (b) all broken 
> packages
> are removed from "stable". So "stable" for me just means "toolchain plus
> all packages that compile and install cleanly at that point in time".
>
> There is a big question in the room: what to do if an update breaks one
> package but fixes another? I do not know how to answer. Maybe it is ok
> to make a "stable release notifications" via email or twitter before
> removing a package from stable, and then users can avoid the update if
> they require the to-be-removed package?
>
>
> 2) I have now continuous integration for this task. I'm not an expert, but
> got some experience with linux-from-scratch. Please suggest, is the
> following a good protocol for checking in intervals of a few days the
> "stability" of the build system:
>   (a) build and install the toolchain-packages that have recent git 
> changes
>   (b) build and install all dependencies of the full toolchain
>   (c) build and install the full toolchain (not just changed packages)
>   (d) build and install all "stable" packages (including all toolchain
>   dependencies and full toolchain)
>   (e) try to build more packages from the archive which are not stable
>
> If (d) is successful, then the latest git should be safe for "stable".
> If (e) is successful then more packages can come into "stable". If (d)
> is not reached, then a decision must be taken if the broken packages
> will be removed from "stable" or will be fixed.
> I

Re: [Msys2-users] could msys2 have a "stable" repository?

[Alan W. Irwin]

On 2017-02-19 23:08+0100 Mario Emmenlauer wrote:

>
> Dear MSYS2 users / developers,
>
> I've start working towards a "stable" repository and would like suggestions!
> Please comment!

[...]

Hi Mario:

I think it would be an excellent idea to have a more stable repository for
those who are not comfortable with the rate of breakage that occurs
for the present MinGW-w64/MSYS2 repositories.

MinGW-w64/MSYS2 is not unique because packaging stability is a common
problem for all free software distribution efforts.  And the issue has
been dealt with in a number of different but effective ways for
various Linux distros. So you may want to look at what some of those
distros do before making any final decisions about implementing a more
stable version of MinGW-w64/MSYS2.

For example, Debian (the only distro I have used since 2000) has two
rolling releases (called "unstable", and "testing"), see
. A given package gets promoted from
"unstable" to "testing" only if that package and _all_ its package
dependencies pass certain criteria (such as a successful build and for
a fixed time interval after that build a lack of critical bug reports
from users). That automated promotion system is so effective and
breakage so rare on "testing" that most Debian desktop users tend to
use the testing repository for Debian. (Debian stable is another
story: it is a fixed-release repository that tends to be quite stable,
but it is based on a very old testing snapshot which is why Debian
desktop users in need of the latest desktop features tend to avoid
it.)

I haven't used ArchLinux, but it is an interesting distro from the
MinGW-w64/MSYS2 perspective because pacman (before being ported for
use by MinGW-w64/MSYS2) was originally developed for ArchLinux.  The
repositories for that free software distro are summarized at
.  From
the description there, ArchLinux historically struggled with the
packaging stability issue, but eventually resolved it by splitting the
packages into various categories like core (a limited number of
essential packages which have stringent stability requirements) and
extra (with less stringent stability requirements).  All ArchLinux
repositories are rolling releases so they have nothing equivalent to
Debian stable, but my guess is core + extra have stability that is
roughly equivalent to Debian "testing" (i.e., very good), and the
ArchLinux "testing" versions of core and extra have stability roughly
equivalent to Debian "unstable" and the current MinGW-w64/MSYS2 (i.e.,
mostly adequate but sometimes broken).

Developing a mostly automated system similar to Debian's for promoting
current MinGW-w64/MSYS2 packages to a more stable rolling release
version of MinGW-w64/MSYS2 might take quite an effort to implement,
but in the long run that might be less work than following a system
similar to the apparent subjective one that ArchLinux has for
promoting packages in its testing repository to core or extra. So, for
example, you might go with non-automated/subjective to start and then
automate it later, but, of course, as the implementer such choices are
up to you.

Anyhow, good luck with this potentially valuable project, and I hope
the URL's I found above concerning the various choices made by other
distros concerning package stability will be of some help to you.

Alan
__
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__

Linux-powered Science
__

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Dear MSYS2 users / developers,

I've start working towards a "stable" repository and would like suggestions!
Please comment!


On 21.09.2016 12:04, Mario Emmenlauer wrote:
> On 30.08.2016 13:27, Mario Emmenlauer wrote:
>> On 30.08.2016 13:06, Alexpux wrote:
 30 авг. 2016 г., в 11:16, Mario Emmenlauer  
 написал(а):

 I was wondering if people would like to have a "stable" repository?

 I was thinking that when a plateau is reached (majority of packages
 compiles fine), the current snapshot could be copied to stable. In my
 eyes, the essential requirement would be that all base packages compile
 fine. Everything else could be handled a bit ad-hoc.
>>>
>>>
>>> Hi, Mario!
>>> If someone want to support «stable» repo - we can talk about it.
>>> There are some problem from my POV to support this.
>>> 1. Who will decide that package is stable?
>>> 2. You can’t just copy builded package from current repository and place it 
>>> «stable». You must rebuild it agains «stable» set of packages.
>>> 3. Who will maintain all of this work? I will not do any work with this 
>>> sorry because my time is limited and now I have many other priorities than 
>>> MSYS2.

1) Definition of "stable" can mean many things, please provide your thoughts!
   In my humble opinion, it would be great if a "stable" repository can fully
   build itself. This is my foremost goal. When I started, I picked some ~60
   packages from the repository but I could successfully build only ~30. This
   was very frustrating in the beginning. I hope to reduce this pain for new
   MSYS2 users.
   Currently when toolchain gets updates, its not transparent for me which
   packages will get broken and which get fixed. So in my eyes, the first
   "stable" release is a toolchain that fully builds itself plus all packages
   that cleanly compile and install. Then changes are accepted into "stable"
   when (a) they do not break any "stable" package, or (b) all broken packages
   are removed from "stable". So "stable" for me just means "toolchain plus
   all packages that compile and install cleanly at that point in time".

   There is a big question in the room: what to do if an update breaks one
   package but fixes another? I do not know how to answer. Maybe it is ok
   to make a "stable release notifications" via email or twitter before
   removing a package from stable, and then users can avoid the update if
   they require the to-be-removed package?


2) I have now continuous integration for this task. I'm not an expert, but
   got some experience with linux-from-scratch. Please suggest, is the
   following a good protocol for checking in intervals of a few days the
   "stability" of the build system:
 (a) build and install the toolchain-packages that have recent git changes
 (b) build and install all dependencies of the full toolchain
 (c) build and install the full toolchain (not just changed packages)
 (d) build and install all "stable" packages (including all toolchain
 dependencies and full toolchain)
 (e) try to build more packages from the archive which are not stable

   If (d) is successful, then the latest git should be safe for "stable".
   If (e) is successful then more packages can come into "stable". If (d)
   is not reached, then a decision must be taken if the broken packages
   will be removed from "stable" or will be fixed.
   Is this a good protocol? The toolchain is rebuilt up to 3 times in this
   process to be 150% sure it is "cleanly" compiled only against newest
   dependencies.


3) I have automated task (2) with my CI system. So I can regularly (weekly,
   and by request?) test the latest snapshot for stability. I can also send
   emails about current status, and my build logs are public. But it would
   still be up to the developers to act upon this knowledge. I will also
   try to help with maintaining the "stable" state, but I also do not have
   much time.

Would this be helpful?

Cheers,

Mario



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Hi,

anything I can do to get started? Any pointers how your current
continuous integration system works? Somebody mentioned that the
CI on github is broken a while ago, is that still true?

Cheers, Mario


On 30.08.2016 13:27, Mario Emmenlauer wrote:
> On 30.08.2016 13:06, Alexpux wrote:
>>> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  написал(а):
>>>
>>> I was wondering if people would like to have a "stable" repository?
>>>
>>> I was thinking that when a plateau is reached (majority of packages
>>> compiles fine), the current snapshot could be copied to stable. In my
>>> eyes, the essential requirement would be that all base packages compile
>>> fine. Everything else could be handled a bit ad-hoc.
>>
>>
>> Hi, Mario!
>> If someone want to support «stable» repo - we can talk about it.
>> There are some problem from my POV to support this.
>> 1. Who will decide that package is stable?
>> 2. You can’t just copy builded package from current repository and place it 
>> «stable». You must rebuild it agains «stable» set of packages.
>> 3. Who will maintain all of this work? I will not do any work with this 
>> sorry because my time is limited and now I have many other priorities than 
>> MSYS2.
> 
> I can maybe do parts of this work. My company has a bit of free
> resources. But you must help me understand what am I up against?
> What is your current build system? Is it Windows-based? How many
> resources does it need? What is the size of the package repository?
> How much traffic do you currently have?
> 
> I have one idle build server (4 Core Xeon with 16GB Ram) that can
> host a build system. I guess "stable" does not change hourly, so
> maybe one machine is sufficient? My company webserver has 0.5TB of
> free disk, and 100MBit internet link in a good IT centre in Europe.
> I could host some packages, and maybe even your dokuwiki(?)



--
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Hi Andrew,

On 30.08.2016 14:39, Andrew Lowe wrote:
> On 30/08/16 19:27, Mario Emmenlauer wrote:
>>
>> Hi Alexey!
>>
>> On 30.08.2016 13:06, Alexpux wrote:
>> Who decides what is stable? From my eyes, the most important would
>> be that all packages in "stable" actually compile against each other :-)
>> Currently in trunk there are many packages that do not compile with
>> the gcc from trunk. That makes it harder for people to get started :-)
>>
>> But that's just my thinking, what do other say?
>>
>> Cheers,
>>
>> Mario
> 
>   I use Gentoo Linux which has both a stable and unstable branch. It 
> might be worth checking out their definition as to what stable and 
> unstable is.

I think it might be hard to compare MSYS2 to major Linux distributions.
The latter usually have more manpower, resources, and user base. So its
likely that they have significantly more testing and development before
a package is considered "stable".

I also assume that MSYS2 users are a bit more open to minor obstacles,
since this is not the most "standard" way to compile for Windows.

My personal goal of having "stable" would be that all "stable" packages
compile against themselves, so users should at all times be able to
make source compiles. Currently this isn't possible. In that sense, I
think currently MSYS2 compares more to Debian "unstable", and I would
prefer to have something that compares to Debian "testing". Maybe that
is anyways the better naming scheme to adopt, "unstable" and "testing"?
And I'd love if MSYS2 gets a "stable" when its ready :-)

But again, that's just me.

Cheers,

Mario



--
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Andrew Lowe]

On 30/08/16 19:27, Mario Emmenlauer wrote:
>
> Hi Alexey!
>
> On 30.08.2016 13:06, Alexpux wrote:
>>> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  написал(а):
>>>
>>> I was wondering if people would like to have a "stable" repository?
>>>
>>> I was thinking that when a plateau is reached (majority of packages
>>> compiles fine), the current snapshot could be copied to stable. In my
>>> eyes, the essential requirement would be that all base packages compile
>>> fine. Everything else could be handled a bit ad-hoc.
>>
>>
>> Hi, Mario!
>> If someone want to support «stable» repo - we can talk about it.
>> There are some problem from my POV to support this.
>> 1. Who will decide that package is stable?
>> 2. You can’t just copy builded package from current repository and place it 
>> «stable». You must rebuild it agains «stable» set of packages.
>> 3. Who will maintain all of this work? I will not do any work with this 
>> sorry because my time is limited and now I have many other priorities than 
>> MSYS2.
>
> I can maybe do parts of this work. My company has a bit of free
> resources. But you must help me understand what am I up against?
> What is your current build system? Is it Windows-based? How many
> resources does it need? What is the size of the package repository?
> How much traffic do you currently have?
>
> I have one idle build server (4 Core Xeon with 16GB Ram) that can
> host a build system. I guess "stable" does not change hourly, so
> maybe one machine is sufficient? My company webserver has 0.5TB of
> free disk, and 100MBit internet link in a good IT centre in Europe.
> I could host some packages, and maybe even your dokuwiki(?)
>
> Who decides what is stable? From my eyes, the most important would
> be that all packages in "stable" actually compile against each other :-)
> Currently in trunk there are many packages that do not compile with
> the gcc from trunk. That makes it harder for people to get started :-)
>
> But that's just my thinking, what do other say?
>
> Cheers,
>
> Mario

I use Gentoo Linux which has both a stable and unstable branch. It 
might be worth checking out their definition as to what stable and 
unstable is.

Andrew


--
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Mario Emmenlauer]

Hi Alexey!

On 30.08.2016 13:06, Alexpux wrote:
>> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  написал(а):
>>
>> I was wondering if people would like to have a "stable" repository?
>>
>> I was thinking that when a plateau is reached (majority of packages
>> compiles fine), the current snapshot could be copied to stable. In my
>> eyes, the essential requirement would be that all base packages compile
>> fine. Everything else could be handled a bit ad-hoc.
> 
> 
> Hi, Mario!
> If someone want to support «stable» repo - we can talk about it.
> There are some problem from my POV to support this.
> 1. Who will decide that package is stable?
> 2. You can’t just copy builded package from current repository and place it 
> «stable». You must rebuild it agains «stable» set of packages.
> 3. Who will maintain all of this work? I will not do any work with this sorry 
> because my time is limited and now I have many other priorities than MSYS2.

I can maybe do parts of this work. My company has a bit of free
resources. But you must help me understand what am I up against?
What is your current build system? Is it Windows-based? How many
resources does it need? What is the size of the package repository?
How much traffic do you currently have?

I have one idle build server (4 Core Xeon with 16GB Ram) that can
host a build system. I guess "stable" does not change hourly, so
maybe one machine is sufficient? My company webserver has 0.5TB of
free disk, and 100MBit internet link in a good IT centre in Europe.
I could host some packages, and maybe even your dokuwiki(?)

Who decides what is stable? From my eyes, the most important would
be that all packages in "stable" actually compile against each other :-)
Currently in trunk there are many packages that do not compile with
the gcc from trunk. That makes it harder for people to get started :-)

But that's just my thinking, what do other say?

Cheers,

Mario



--
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] could msys2 have a "stable" repository?

[Alexpux]

> 30 авг. 2016 г., в 11:16, Mario Emmenlauer  написал(а):
> 
> 
> I was wondering if people would like to have a "stable" repository?
> 
> I was thinking that when a plateau is reached (majority of packages
> compiles fine), the current snapshot could be copied to stable. In my
> eyes, the essential requirement would be that all base packages compile
> fine. Everything else could be handled a bit ad-hoc.


Hi, Mario!
If someone want to support «stable» repo - we can talk about it.
There are some problem from my POV to support this.
1. Who will decide that package is stable?
2. You can’t just copy builded package from current repository and place it 
«stable». You must rebuild it agains «stable» set of packages.
3. Who will maintain all of this work? I will not do any work with this sorry 
because my time is limited and now I have many other priorities than MSYS2.

Regards,
Alexey.

> 
> All the best, Mario
> 
> --
> ___
> Msys2-users mailing list
> Msys2-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/msys2-users


--
___
Msys2-users mailing list
Msys2-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/msys2-users