Re: [gluster-packaging] Release 4.0: Making it happen! (GlusterD2)

[Shyam Ranganathan]

On 01/11/2018 02:01 AM, Kaushal M wrote:
>>   - (thought/concern) Jenkins smoke job (or other jobs) that builds RPMs
>> will not build GD2 (as the source is not available) and will continue as
>> is (which means there is enough spec file magic here that we can specify
>> during release packaging to additionally build GD2)
> The glusterfs spec file can be updated to include building GD2 from
> its release tarball. I don't remember exactly but, rpmbuild might have
> ways to automatically download sources/dependencies. We can check if
> this is true.
> 

So, I do not know if this is a real problem (yet!). But I do hope I have
conveyed my concern/problem right, hence restating below.

The in (glusterfs) tree spec file if modified to build GD2 in addition
by default, will fail to do so (or need changes) in the smoke jobs that
we have, and this may need addressing.

Kaushal, based on your thoughts, I assume the in tree spec file is
changed to point to a specific release tarball (always) and remaining
rpmbuild voodoo will make this happen.

The question/concern though is that we need this updated for every
release (minor or major) IMO. But, worth a start.

My thought/suggestion is to write the spec in such a fashion (if at all
possible) that makes it optional to build with GD2, hence can be used
for packaging, but avoided in the smoke.

Shyam
___
packaging mailing list
packaging@gluster.org
http://lists.gluster.org/mailman/listinfo/packaging

Re: [gluster-packaging] Release 4.0: Making it happen! (GlusterD2)

[Shyam Ranganathan]

On 01/11/2018 02:04 AM, Kaushal M wrote:
> On Thu, Jan 11, 2018 at 1:56 AM, Kaleb S. KEITHLEY  
> wrote:
>> comments inline
>>
>> On 01/10/2018 02:08 PM, Shyam Ranganathan wrote:
>>
>> Hi, (GD2 team, packaging team, please read)
>>
>> Here are some things we need to settle so that we can ship/release GD2
>> along with Gluster 4.0 release (considering this is a separate
>> repository as of now).
>>
>> 1) Generating release package (read as RPM for now) to go with Gluster
>> 4.0 release
>>
>> Proposal:
>>   - GD2 makes github releases, as in [1]
>>
>>   - GD2 Releases (tagging etc.) are made in tandem to Gluster releases
>> - So, when an beta1/RC0 is tagged for gluster release, this will
>> receive a coordinated release (if required) from the GD2 team
>> - GD2 team will receive *at-least* a 24h notice on a tentative
>> Gluster tagging date/time, to aid the GD2 team to prepare the required
>> release tarball in github
>>
>> This is a no-op. In github creating a tag or a release automatically creates
>> the tar source file.

Ah! yes so it does. Was not aware of this, but I guess what Kaushal adds
below maybe valid as well. Will let you folks arrive at the right
tarball (and possibly right location to pick it up from). The tarball
mechanism as such (as discussed with Kaleb) stands I assume.

> 
> While true, this tarball isn't enough. The GD2 build scripts lookup
> versioning from git tags or from a VERSION file (same as glusterfs).
> Both of these are not present in the tarball github generates.
> The GD2 release script generates tarballs that have everything
> required to build a properly versioned GD2.
> 
>>
>>   - Post a gluster tag being created, and the subsequent release job is
>> run for gluster 4.0, the packaging team will be notified about which GD2
>> tag to pick up for packaging, with this gluster release
>> - IOW, a response to the Jenkins generated packaging job, with the
>> GD2 version/tag/release to pick up
>>
>>   - GD2 will be packaged as a sub-package of the glusterfs package, and
>> hence will have appropriate changes to the glusterfs spec file (or other
>> variants of packaging as needed), to generate one more package (RPM) to
>> post in the respective download location
>>
>>   - The GD2 sub-package version would be the same as the release version
>> that GD2 makes (it will not be the gluster package version, at least for
>> now)
>>
>> IMO it's clearer if the -glusterd2 sub-package has the same version as the
>> rest of the glusterfs-* packages.
>>
> 
> +1. We will follow glusterfs versioning not just for the packages, but
> for the source itself.

Agreed!

> 
>> The -glusterd2 sub-package's Summary and/or its %description can be used to
>> identify the version of GD2.
>>
>> Emphasis on IMO. It is possible for the -glusterd sub-package to have a
>> version that's different than the parent package(s).
>>
>>   - For now, none of the gluster RPMs would be dependent on the GD2 RPM
>> in the downloads, so any user wanting to use GD2 would have to install
>> the package specifically and then proceed as needed
>>
>>   - (thought/concern) Jenkins smoke job (or other jobs) that builds RPMs
>> will not build GD2 (as the source is not available) and will continue as
>> is (which means there is enough spec file magic here that we can specify
>> during release packaging to additionally build GD2)
>>
>> 2) Generate a quick start or user guide, to aid using GD2 with 4.0
>>
>> @Kaushal if this is generated earlier (say with beta builds of 4.0
>> itself) we could get help from the community to test drive the same and
>> provide feedback to improve the guide for users by the release (as
>> discussed in the maintainers meeting)
>>
>> One thing not covered above is what happens when GD2 fixes a high priority
>> bug between releases of glusterfs.
>>
>> Once option is we wait until the next release of glusterfs to include the
>> update to GD2.
>>
>> Or we can respin (rerelease) the glusterfs packages with the updated GD2.
>> I.e. glusterfs-4.0.0-1 (containing GD2-1.0.0) -> glusterfs-4.0.0-2
>> (containing GD2-1.0.1).
>>
>> Or we can decide not to make a hard rule and do whatever makes the most
>> sense at the time. If the fix is urgent, we respin. If the fix is not urgent
>> it waits for the next Gluster release. (From my perspective though I'd
>> rather not do respins, I've already got plenty of work doing the regular
>> releases.)

I would think we follow what we need to do for the gluster package (and
its sub-packages) as it stands now. If there is an important enough fix
(critical/security etc.) that requires a one-off build (ie. not a
maintenance release or a regular release) we respin the whole thing
(which is more work).

I think if it is a GD2 specific fix then just re-spinning that
sub-package makes more sense and possibly less work.

I am going to leave the decision of re-spinning the whole thing or just
the GD2 package to the packaging folks, but state that re-spin 

Re: [gluster-packaging] Release 4.0: Making it happen! (GlusterD2)

[Kaushal M]

On Thu, Jan 11, 2018 at 1:56 AM, Kaleb S. KEITHLEY  wrote:
> comments inline
>
> On 01/10/2018 02:08 PM, Shyam Ranganathan wrote:
>
> Hi, (GD2 team, packaging team, please read)
>
> Here are some things we need to settle so that we can ship/release GD2
> along with Gluster 4.0 release (considering this is a separate
> repository as of now).
>
> 1) Generating release package (read as RPM for now) to go with Gluster
> 4.0 release
>
> Proposal:
>   - GD2 makes github releases, as in [1]
>
>   - GD2 Releases (tagging etc.) are made in tandem to Gluster releases
> - So, when an beta1/RC0 is tagged for gluster release, this will
> receive a coordinated release (if required) from the GD2 team
> - GD2 team will receive *at-least* a 24h notice on a tentative
> Gluster tagging date/time, to aid the GD2 team to prepare the required
> release tarball in github
>
> This is a no-op. In github creating a tag or a release automatically creates
> the tar source file.

While true, this tarball isn't enough. The GD2 build scripts lookup
versioning from git tags or from a VERSION file (same as glusterfs).
Both of these are not present in the tarball github generates.
The GD2 release script generates tarballs that have everything
required to build a properly versioned GD2.

>
>   - Post a gluster tag being created, and the subsequent release job is
> run for gluster 4.0, the packaging team will be notified about which GD2
> tag to pick up for packaging, with this gluster release
> - IOW, a response to the Jenkins generated packaging job, with the
> GD2 version/tag/release to pick up
>
>   - GD2 will be packaged as a sub-package of the glusterfs package, and
> hence will have appropriate changes to the glusterfs spec file (or other
> variants of packaging as needed), to generate one more package (RPM) to
> post in the respective download location
>
>   - The GD2 sub-package version would be the same as the release version
> that GD2 makes (it will not be the gluster package version, at least for
> now)
>
> IMO it's clearer if the -glusterd2 sub-package has the same version as the
> rest of the glusterfs-* packages.
>

+1. We will follow glusterfs versioning not just for the packages, but
for the source itself.

> The -glusterd2 sub-package's Summary and/or its %description can be used to
> identify the version of GD2.
>
> Emphasis on IMO. It is possible for the -glusterd sub-package to have a
> version that's different than the parent package(s).
>
>   - For now, none of the gluster RPMs would be dependent on the GD2 RPM
> in the downloads, so any user wanting to use GD2 would have to install
> the package specifically and then proceed as needed
>
>   - (thought/concern) Jenkins smoke job (or other jobs) that builds RPMs
> will not build GD2 (as the source is not available) and will continue as
> is (which means there is enough spec file magic here that we can specify
> during release packaging to additionally build GD2)
>
> 2) Generate a quick start or user guide, to aid using GD2 with 4.0
>
> @Kaushal if this is generated earlier (say with beta builds of 4.0
> itself) we could get help from the community to test drive the same and
> provide feedback to improve the guide for users by the release (as
> discussed in the maintainers meeting)
>
> One thing not covered above is what happens when GD2 fixes a high priority
> bug between releases of glusterfs.
>
> Once option is we wait until the next release of glusterfs to include the
> update to GD2.
>
> Or we can respin (rerelease) the glusterfs packages with the updated GD2.
> I.e. glusterfs-4.0.0-1 (containing GD2-1.0.0) -> glusterfs-4.0.0-2
> (containing GD2-1.0.1).
>
> Or we can decide not to make a hard rule and do whatever makes the most
> sense at the time. If the fix is urgent, we respin. If the fix is not urgent
> it waits for the next Gluster release. (From my perspective though I'd
> rather not do respins, I've already got plenty of work doing the regular
> releases.)
>
> The alternative to all of the above is to package GD2 in its own package.
> This entails opening a New Package Request and going through the packaging
> reviews. All in all it's a lot of work. If GD2 source is eventually going to
> be moved into the main glusterfs source though this probably doesn't make
> sense.
>
> --
>
> Kaleb
>
>
>
>
> ___
> packaging mailing list
> packaging@gluster.org
> http://lists.gluster.org/mailman/listinfo/packaging
>
___
packaging mailing list
packaging@gluster.org
http://lists.gluster.org/mailman/listinfo/packaging

Re: [gluster-packaging] Release 4.0: Making it happen! (GlusterD2)

[Kaushal M]

On Thu, Jan 11, 2018 at 12:38 AM, Shyam Ranganathan  wrote:
> Hi, (GD2 team, packaging team, please read)
>
> Here are some things we need to settle so that we can ship/release GD2
> along with Gluster 4.0 release (considering this is a separate
> repository as of now).
>
> 1) Generating release package (read as RPM for now) to go with Gluster
> 4.0 release
>
> Proposal:
>   - GD2 makes github releases, as in [1]
>
>   - GD2 Releases (tagging etc.) are made in tandem to Gluster releases
> - So, when an beta1/RC0 is tagged for gluster release, this will
> receive a coordinated release (if required) from the GD2 team
> - GD2 team will receive *at-least* a 24h notice on a tentative
> Gluster tagging date/time, to aid the GD2 team to prepare the required
> release tarball in github

Sounds good.

>
>   - Post a gluster tag being created, and the subsequent release job is
> run for gluster 4.0, the packaging team will be notified about which GD2
> tag to pick up for packaging, with this gluster release
> - IOW, a response to the Jenkins generated packaging job, with the
> GD2 version/tag/release to pick up
>
>   - GD2 will be packaged as a sub-package of the glusterfs package, and
> hence will have appropriate changes to the glusterfs spec file (or other
> variants of packaging as needed), to generate one more package (RPM) to
> post in the respective download location
>
>   - The GD2 package version would be the same as the release version
> that GD2 makes (it will not be the gluster package version, at least for
> now)

I prefer if GD2 follows gluster versioning. Keeps things simpler.
Anyone packaging will have to just pick the same version of GD2.
We already version our perview releases as v4.0dev.

>
>   - For now, none of the gluster RPMs would be dependent on the GD2 RPM
> in the downloads, so any user wanting to use GD2 would have to install
> the package specifically and then proceed as needed

Yes. The glusterfs-server package will not depend on GD2 right now.
This will be changed later when GD2 becomes the default.

>
>   - (thought/concern) Jenkins smoke job (or other jobs) that builds RPMs
> will not build GD2 (as the source is not available) and will continue as
> is (which means there is enough spec file magic here that we can specify
> during release packaging to additionally build GD2)

The glusterfs spec file can be updated to include building GD2 from
its release tarball. I don't remember exactly but, rpmbuild might have
ways to automatically download sources/dependencies. We can check if
this is true.

>
> 2) Generate a quick start or user guide, to aid using GD2 with 4.0
>
> @Kaushal if this is generated earlier (say with beta builds of 4.0
> itself) we could get help from the community to test drive the same and
> provide feedback to improve the guide for users by the release (as
> discussed in the maintainers meeting).

We will do this.

>
> Thanks,
> Shyam
>
> [1] github GD2 releases: https://github.com/gluster/glusterd2/releases
> ___
> packaging mailing list
> packaging@gluster.org
> http://lists.gluster.org/mailman/listinfo/packaging
___
packaging mailing list
packaging@gluster.org
http://lists.gluster.org/mailman/listinfo/packaging