Re: [Discuss-gnuradio] Changes to the development model

[CEL]

Yes, I wished I had a definite number at this point; we'll probably
actually will orient on the release cycles of the more popular Linux
distros, but, to be honest, I don't think it'd be 100% appropriate to
promise that "we will _fully_ support 3.7 till 2023, because it's what
Ubuntu 18.04 LTS ships" at this point. That's just too long a timeline
to make certain statements about. Also, I'd argue that people really
into long-term support will probably rather use RHEL or CentOS, but
that's just a wild guess, and certainly doesn't make the support
timeframe shorter. An interesting question really also is how
interested users of 3.7 in e.g. two years will be in more bugfixing –
at that point, you're probably using a specific piece of software that
relies on specific, potentially buggy, behaviour of your framework, so
you might be hesitant to even install bugfix updates.

So, the honest answer really is: we'll be listening to our users before
we shut anything down. Also, "not actively maintained" doesn't mean
"breaks instantly"; if you're on an old platform using old software
with an old GNU Radio release, nothing would ever break (I do know a
couple of GNU Radio 3.4.2 users. You can't even build 3.4.2 on a modern
Linux without a lot of handholding, but they are happy with their
current installation and don't *need* updates).
The interesting question really is how to move forward without
sacrificing all backwards compatibility in the future, and how to make
it easy for users of previous versions to take advantage of new
developments without burdening down the development process so that GNU
Radio becomes obsolete by relying on obsolete dependencies. That's why
for 3.8, not only the move to Qt5 is on our agenda, but also the move
to Python3, and away from some of the dependencies that we used to
carry around in-tree. Pretty exciting stuff, but an unavoidable change
in API. 
Still, I'm confident that most of GNU Radio itself translates to this
new dependencies relatively smoothly in the next weeks, and that the
OOT and flow graph ecosystem will in fact adapt with relative ease –
we've literally got a heap of work in the merge pipeline that makes a
lot of this saner than it used to be. So, I'm afraid 3.8 will bring
quite a bit of breakage, but we've got the best glue to put everyone's
applications back together, and resolve a couple of the stranger things
that people had to do to make things work under 3.7.

With that, I'll leave the discussion for tonight; in the end, what we
do doesn't only depend on the stable release cycle vision that I have,
but simply on how demand for new versions and features turns out, and
very importantly, on how many developers can dedicate steady time into
the project, and on which parts they work. A long term goal is
definitely to upstream more commonly reinvented blocks – making the
project more complete ("batteries included"), but also allowing us to
actually notice when we break applications. So, and I know this really
doesn't apply to you overly much, as you've been working with the rest
of GR very intimately on your code, the best way to make sure that
it'll be easy to run your own code with GNU Radio of 2023 will be
making sure that your blocks are universally useful, and getting them
into the main GNU Radio source tree medium- to longterm. That way, the
responsibility of making it work with the next release of GNU Radio
becomes part of the maintenance cycle – and hence, of automated
testing, influencing patches and feature branches to be merged, whilst
saving you developer time (and money, maybe?) by only bringing your
code up to current GNU Radio standards once instead of perpetually.
That, I hope, works rather well because stakeholders simply contribute
their code, ensuring that their interests are being respected on a
technical level; in exchange for their contribution and time caring for
the more specific aspects of their code later on, the GNU Radio project
offers them long-term integration testing and thus, stability.
This is going to be an interesting balance between upstreaming as much
as feasible and keeping the amount of cruft and in-tree duplication at
bay, whilst simultaneously keeping dependencies manageable. On a
packaging level, that might mean that we will opt for more modular
builds with separately installable gnuradio-runtime, gr-uhd, gr-qtgui,
gr-dtv, gr-world-dominance-over-the-web etc with separate dependencies
each. Some distros actually already have a pretty good way of doing
that, and I'd like to technologically foster that in our build system.
Maybe that actually means that we might be able to declare even stuff
that has far-off-core-dependencies to become part of the official GNU
Radio source tree one day. We'll see how this plays out!

I'm pretty set on not making any rash decisions in that direction.
There'll be plenty of discussion revolving around how to make sure we
don't break everyone's application whilst still gathering enough
developers, and 

Re: [Discuss-gnuradio] Changes to the development model

[Kartik Patel]

Hello Marcus,

All I wanted to know was the typical time frame of the maint-X.Y branch
(not 3.7 in specific). I believe the answer is (from your reply), it will
depend on the particular version. Maybe maint-3.7 will have a long life and
other maint-X.Y may not.

With the Debian and QT4 example, it's more clear on why we need this. I am
not worried about anything in particular though. It was a general question
for more information on this workflow.

Thanks for the detailed response. :)

Regards,
Kartik Patel


On Mon, Feb 26, 2018 at 4:18 PM, Müller, Marcus (CEL) 
wrote:

> Hello Kartik,
>
> there will not be a "maint-3.7.12", there will only be a "maint-3.7";
> that's enough fragmentation for me :)
>
> Yes, it will be actively maintained for a finite time, just like
> anything in this world :) That timeframe probably will be a bit longer
> than for future maint-X.Y branches, as 3.7 has been around for so very
> long that there's by now a lot of software that actually relies on all
> the special kinks and features that GNU Radio 3.7 has, and we don't
> want to stop supporting any of that!
> Still, it's important to be forward-facing: Dependencies are always
> moving, and for example, without extensive manual labour of Maitland,
> GNU Radio 3.7 would already have been thrown out of Debian because it's
> still using Qt4! We must work on all contributions, especially user-
> facing stuff like GUI frontends, become compatible with 3.8; don't
> worry, we won't realease 3.8 without making sure it's usable, nor will
> we abandon versions that are widely used.
>
> What I cannot tell you at this point is how many years the support for
> maint-3.7 will go on. Time will tell, and if there's high demand, we
> might at some point even find a maintainer just for that branch who's
> job would be to backport patches and keep 3.7 alive for those who can't
> switch. Again, this is by no means us trying to abandon existing users,
> this is us being very worried that you won't even be able to build
> upstream GNU Radio on mainstream linux distros for much longer, and
> actively working on making it work for the future, thus, ensuring that
> applications continue to run, with as little modification as possible.
>
> Does that answer your question? Are you worried about something in
> particular? This is an excellent time to come forward about that,
> either here or in private, if it's something sensitive; the GR
> leadership is more than interested in not breaking everyone's use case!
>
> Best regards,
> Marcus
>
> On Mon, 2018-02-26 at 11:16 -0600, Kartik Patel wrote:
> > Hello Marcus,
> >
> > I believe this workflow is an excellent improvement and stepping
> > stone to acknowledge that GNU Radio is now a "medium-to-large" size
> > projects.
> >
> > Only thing I did not understand was the connection with Linux
> > Distros. I think you want to say that suppose 3.7.12 version is
> > released, then the corresponding maint-3.7.12 will be active only for
> > a given timeframe? Is that correct? How long would that timeframe be?
> >
> > Looking forward to the blog post!
> >
> > Regards,
> > Kartik Patel
> >
> > On Mon, Feb 26, 2018 at 10:49 AM, Müller, Marcus (CEL)  > du> wrote:
> > > Hello everyone,
> > >
> > > as the last days have been pretty heavy on discussion between the
> > > new
> > > GNU Radio lead team, it’s now become clearer how we’ll actually
> > > deal
> > > with the day-to-day maintenance work as well with the release
> > > management.
> > >
> > > So, as you might have noticed, we’ve been crunching through the
> > > Pull
> > > Request Queue on github. We weren’t able to merge all the PRs, but
> > > some
> > > of these are blocked by the legal side of things, others simply
> > > need
> > > small tweaks. What we were, however, able to do: Defining what
> > > should
> > > be in release 3.7.12 (not all of this is visible on the PR tracker,
> > > but
> > > we have a pretty good idea about it). So, once these remaining PRs
> > > are
> > > closed, and all issues tagged with this milestone are solved,
> > > 3.7.12
> > > will be released.
> > >
> > > That release will also mark a radical change to the git workflow
> > > that
> > > we’ll stick to:
> > >
> > > We’re killing the idea that you usually submit PRs against maint.
> > > In
> > > fact, there won’t be a maint branch anymore:
> > >
> > > New releases will be tagged from the master branch. That means once
> > > we
> > > released a version, for example 4.10, there will be a maint-4.10
> > > branch, onto which we can merge fixes.
> > > As versioning scheme, we’ll be adhering to Semantic Versioning,
> > > with
> > > the first digit being the "paradigm" digit ("3" for the time
> > > being),
> > > the second the "API" digit, meaning that we won't change how
> > > programmers use GNU Radio without increasing the second digit, the
> > > third being the "ABI" digit, meaning that as long as the first
> > > three
> > > digit stay the same, you can 

Re: [Discuss-gnuradio] Changes to the development model

[CEL]

Hello Kartik,

there will not be a "maint-3.7.12", there will only be a "maint-3.7";
that's enough fragmentation for me :)

Yes, it will be actively maintained for a finite time, just like
anything in this world :) That timeframe probably will be a bit longer
than for future maint-X.Y branches, as 3.7 has been around for so very
long that there's by now a lot of software that actually relies on all
the special kinks and features that GNU Radio 3.7 has, and we don't
want to stop supporting any of that!
Still, it's important to be forward-facing: Dependencies are always
moving, and for example, without extensive manual labour of Maitland,
GNU Radio 3.7 would already have been thrown out of Debian because it's
still using Qt4! We must work on all contributions, especially user-
facing stuff like GUI frontends, become compatible with 3.8; don't
worry, we won't realease 3.8 without making sure it's usable, nor will
we abandon versions that are widely used.

What I cannot tell you at this point is how many years the support for
maint-3.7 will go on. Time will tell, and if there's high demand, we
might at some point even find a maintainer just for that branch who's
job would be to backport patches and keep 3.7 alive for those who can't
switch. Again, this is by no means us trying to abandon existing users,
this is us being very worried that you won't even be able to build
upstream GNU Radio on mainstream linux distros for much longer, and
actively working on making it work for the future, thus, ensuring that
applications continue to run, with as little modification as possible.

Does that answer your question? Are you worried about something in
particular? This is an excellent time to come forward about that,
either here or in private, if it's something sensitive; the GR
leadership is more than interested in not breaking everyone's use case!

Best regards,
Marcus

On Mon, 2018-02-26 at 11:16 -0600, Kartik Patel wrote:
> Hello Marcus,
> 
> I believe this workflow is an excellent improvement and stepping
> stone to acknowledge that GNU Radio is now a "medium-to-large" size
> projects.
> 
> Only thing I did not understand was the connection with Linux
> Distros. I think you want to say that suppose 3.7.12 version is
> released, then the corresponding maint-3.7.12 will be active only for
> a given timeframe? Is that correct? How long would that timeframe be?
> 
> Looking forward to the blog post!
> 
> Regards,
> Kartik Patel
> 
> On Mon, Feb 26, 2018 at 10:49 AM, Müller, Marcus (CEL)  du> wrote:
> > Hello everyone,
> > 
> > as the last days have been pretty heavy on discussion between the
> > new
> > GNU Radio lead team, it’s now become clearer how we’ll actually
> > deal
> > with the day-to-day maintenance work as well with the release
> > management.
> > 
> > So, as you might have noticed, we’ve been crunching through the
> > Pull
> > Request Queue on github. We weren’t able to merge all the PRs, but
> > some
> > of these are blocked by the legal side of things, others simply
> > need
> > small tweaks. What we were, however, able to do: Defining what
> > should
> > be in release 3.7.12 (not all of this is visible on the PR tracker,
> > but
> > we have a pretty good idea about it). So, once these remaining PRs
> > are
> > closed, and all issues tagged with this milestone are solved,
> > 3.7.12
> > will be released.
> > 
> > That release will also mark a radical change to the git workflow
> > that
> > we’ll stick to:
> > 
> > We’re killing the idea that you usually submit PRs against maint.
> > In
> > fact, there won’t be a maint branch anymore:
> > 
> > New releases will be tagged from the master branch. That means once
> > we
> > released a version, for example 4.10, there will be a maint-4.10
> > branch, onto which we can merge fixes.
> > As versioning scheme, we’ll be adhering to Semantic Versioning,
> > with
> > the first digit being the "paradigm" digit ("3" for the time
> > being),
> > the second the "API" digit, meaning that we won't change how
> > programmers use GNU Radio without increasing the second digit, the
> > third being the "ABI" digit, meaning that as long as the first
> > three
> > digit stay the same, you can just replace one libgnuradio*.so with
> > another one without rebuilding your binary (that's a feature that I
> > think software distributors will be most happy about), and the
> > fourth
> > digit being the patchlevel.
> > 
> > The lifetime of these branches will reflect the life time of the
> > (primarily) Linux distributions that ship that package; it’s our
> > outspoken goal to not break GNU Radio for anyone who uses it on
> > widely
> > used, actively maintained platforms. This will necessitate
> > maintenance
> > of Long-Term Support releases.
> > 
> > Development happens on and against master. If there is a bug fix,
> > we do
> > that on master (if it applies to master, at least), and backport
> > fixes
> > to the maint-X.Y branches that we still support. This 

Re: [Discuss-gnuradio] Changes to the development model

[Kartik Patel]

Hello Marcus,

I believe this workflow is an excellent improvement and stepping stone to
acknowledge that GNU Radio is now a "medium-to-large" size projects.

Only thing I did not understand was the connection with Linux Distros. I
think you want to say that suppose 3.7.12 version is released, then the
corresponding maint-3.7.12 will be active only for a given timeframe? Is
that correct? How long would that timeframe be?

Looking forward to the blog post!

Regards,
Kartik Patel

On Mon, Feb 26, 2018 at 10:49 AM, Müller, Marcus (CEL) 
wrote:

> Hello everyone,
>
> as the last days have been pretty heavy on discussion between the new
> GNU Radio lead team, it’s now become clearer how we’ll actually deal
> with the day-to-day maintenance work as well with the release
> management.
>
> So, as you might have noticed, we’ve been crunching through the Pull
> Request Queue on github. We weren’t able to merge all the PRs, but some
> of these are blocked by the legal side of things, others simply need
> small tweaks. What we were, however, able to do: Defining what should
> be in release 3.7.12 (not all of this is visible on the PR tracker, but
> we have a pretty good idea about it). So, once these remaining PRs are
> closed, and all issues tagged with this milestone are solved, 3.7.12
> will be released.
>
> That release will also mark a radical change to the git workflow that
> we’ll stick to:
>
> We’re killing the idea that you usually submit PRs against maint. In
> fact, there won’t be a maint branch anymore:
>
> New releases will be tagged from the master branch. That means once we
> released a version, for example 4.10, there will be a maint-4.10
> branch, onto which we can merge fixes.
> As versioning scheme, we’ll be adhering to Semantic Versioning, with
> the first digit being the "paradigm" digit ("3" for the time being),
> the second the "API" digit, meaning that we won't change how
> programmers use GNU Radio without increasing the second digit, the
> third being the "ABI" digit, meaning that as long as the first three
> digit stay the same, you can just replace one libgnuradio*.so with
> another one without rebuilding your binary (that's a feature that I
> think software distributors will be most happy about), and the fourth
> digit being the patchlevel.
>
> The lifetime of these branches will reflect the life time of the
> (primarily) Linux distributions that ship that package; it’s our
> outspoken goal to not break GNU Radio for anyone who uses it on widely
> used, actively maintained platforms. This will necessitate maintenance
> of Long-Term Support releases.
>
> Development happens on and against master. If there is a bug fix, we do
> that on master (if it applies to master, at least), and backport fixes
> to the maint-X.Y branches that we still support. This requires a bit of
> consequence from PR authors: If you do happen to submit a PR that
> contains a bug fix, please do note that in the PR itself, and make the
> bugfix a commit of its own, so that it's easily cherry-pickable on the
> appropriate maint-X.Y branch. We don't know yet whether that'll be easy
> for every possible bugfix out there, but that's something we'd have to
> figure out from case to case, anyways.
>
> These branches are there so that distros etc. can get GNU Radio
> bugfixes, and we don't force users to upgrade GNU Radio to "Bleeding
> Edge" just to get a bugfix.
>
> How does the "next" branch fit into this? Long story short: in its
> current shape, it doesn’t. As soon we released the next 3.7.12 release
> (which means tagging master “v3.7.12.0”), “master” becomes “maint-3.7”;
> “next” becomes “master”. The following release that we’ll do is 3.8, if
> all goes well (but honestly, at this point, what shouldn’t?).
> The v3.8.0.0 tagged commit will also the anchor of the “maint-3.8”
> branch. Note that it’s not “maint-3.8.0.0”. We'll limit the number of
> heads we have to merge things into as far as possible (very much in the
> sense of not letting this become a hydra).
>
> Obviously, this implies that we have to increase the X.Y release
> frequency. But: I think that’s totally doable. Only thing we need to
> make sure is that our changelogs are really good enough for people to
> track when features were added.
>
> This all isn’t really easy to grok for people who’ve not worked with
> medium-to-large git projects before, so I’ll have to make a nice
> drawing and a blog post, but I recon it’s better to keep the mailing
> list updated now than having a nice blog post in a month.
>
> So, looking forward to countless PRs,
>
> Marcus
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>


smime.p7s
Description: S/MIME cryptographic signature
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

[Discuss-gnuradio] Changes to the development model

[CEL]

Hello everyone,

as the last days have been pretty heavy on discussion between the new
GNU Radio lead team, it’s now become clearer how we’ll actually deal
with the day-to-day maintenance work as well with the release
management.

So, as you might have noticed, we’ve been crunching through the Pull
Request Queue on github. We weren’t able to merge all the PRs, but some
of these are blocked by the legal side of things, others simply need
small tweaks. What we were, however, able to do: Defining what should
be in release 3.7.12 (not all of this is visible on the PR tracker, but
we have a pretty good idea about it). So, once these remaining PRs are
closed, and all issues tagged with this milestone are solved, 3.7.12
will be released.

That release will also mark a radical change to the git workflow that
we’ll stick to:

We’re killing the idea that you usually submit PRs against maint. In
fact, there won’t be a maint branch anymore:

New releases will be tagged from the master branch. That means once we
released a version, for example 4.10, there will be a maint-4.10
branch, onto which we can merge fixes.
As versioning scheme, we’ll be adhering to Semantic Versioning, with
the first digit being the "paradigm" digit ("3" for the time being),
the second the "API" digit, meaning that we won't change how
programmers use GNU Radio without increasing the second digit, the
third being the "ABI" digit, meaning that as long as the first three
digit stay the same, you can just replace one libgnuradio*.so with
another one without rebuilding your binary (that's a feature that I
think software distributors will be most happy about), and the fourth
digit being the patchlevel.

The lifetime of these branches will reflect the life time of the
(primarily) Linux distributions that ship that package; it’s our
outspoken goal to not break GNU Radio for anyone who uses it on widely
used, actively maintained platforms. This will necessitate maintenance
of Long-Term Support releases.

Development happens on and against master. If there is a bug fix, we do
that on master (if it applies to master, at least), and backport fixes
to the maint-X.Y branches that we still support. This requires a bit of
consequence from PR authors: If you do happen to submit a PR that
contains a bug fix, please do note that in the PR itself, and make the
bugfix a commit of its own, so that it's easily cherry-pickable on the
appropriate maint-X.Y branch. We don't know yet whether that'll be easy
for every possible bugfix out there, but that's something we'd have to
figure out from case to case, anyways.

These branches are there so that distros etc. can get GNU Radio
bugfixes, and we don't force users to upgrade GNU Radio to "Bleeding
Edge" just to get a bugfix.

How does the "next" branch fit into this? Long story short: in its
current shape, it doesn’t. As soon we released the next 3.7.12 release
(which means tagging master “v3.7.12.0”), “master” becomes “maint-3.7”; 
“next” becomes “master”. The following release that we’ll do is 3.8, if
all goes well (but honestly, at this point, what shouldn’t?).
The v3.8.0.0 tagged commit will also the anchor of the “maint-3.8”
branch. Note that it’s not “maint-3.8.0.0”. We'll limit the number of
heads we have to merge things into as far as possible (very much in the
sense of not letting this become a hydra).

Obviously, this implies that we have to increase the X.Y release
frequency. But: I think that’s totally doable. Only thing we need to
make sure is that our changelogs are really good enough for people to
track when features were added.

This all isn’t really easy to grok for people who’ve not worked with
medium-to-large git projects before, so I’ll have to make a nice
drawing and a blog post, but I recon it’s better to keep the mailing
list updated now than having a nice blog post in a month.

So, looking forward to countless PRs,

Marcus

smime.p7s
Description: S/MIME cryptographic signature
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio