Re: [openstack-dev] [tripleo][ui] Dependency management

[Julie Pichon]

On 2 March 2018 at 22:52, Alan Pevec  wrote:
> On Mon, Jan 22, 2018 at 9:30 AM, Julie Pichon  wrote:
>> On 19 January 2018 at 18:43, Honza Pokorny  wrote:
>>> We've recently discovered an issue with the way we handle dependencies for
>>> tripleo-ui.  This is an explanation of the problem, and a proposed solution.
>>> I'm looking for feedback.
>>>
>>> Before the upgrade to zuul v3 in TripleO CI, we had two types of jobs for
>>> tripleo-ui:
>>>
>>> * Native npm jobs
>>> * Undercloud integration jobs
>>>
>>> After the upgrade, the integration jobs went away.  Our goal is to add them
>>> back.
>>>
>>> There is a difference in how these two types of jobs handle dependencies.
>>> Native npm jobs use the "npm install" command to acquire packages, and
>>> undercloud jobs use RPMs.  The tripleo-ui project uses a separate RPM for
>>> dependencies called openstack-tripleo-ui-deps.
>>>
>>> Because of the requirement to use a separate RPM for dependencies, there is 
>>> some
>>> extra work needed when a new dependency is introduced, or an existing one is
>>> upgraded.  Once the patch that introduces the dependency is merged, we have 
>>> to
>>> increment the version of the -deps package, and rebuild it.  It then shows 
>>> up in
>>> the yum repos used by the undercloud.
>>>
>>> To make matters worse, we recently upgraded our infrastructure to nodejs 
>>> 8.9.4
>>> and npm 5.6.0 (latest stable).  This makes it so we can't write "purist" 
>>> patches
>>> that simply introduce a new dependency to package.json, and nothing more.  
>>> The
>>> code that uses the new dependency must be included.  I tend to think that 
>>> each
>>> commit should work on its own so this can be seen as a plus.
>>>
>>> This presents a problem: you can't get a patch that introduces a new 
>>> dependency
>>> merged because it's not included in the RPM needed by the undercloud ci job.
>>>
>>> So, here is a proposal on how that might work:
>>>
>>> 1. Submit a patch for review that introduces the dependency, along with code
>>>changes to support it and validate its inclusion
>>> 2. Native npm jobs will pass
>>> 3. Undercloud gate job will fail because the dependency isn't in -deps RPM
>>> 4. We ask RDO to review for licensing
>>> 5. Once reviewed, new -deps package is built
>>> 6. Recheck
>>> 7. All jobs pass
>>
>> Perhaps there should be a step after 3 or 4 to have the patch normally
>> reviewed, and wait for it to have two +2s before building the new
>> package? Otherwise we may end up with wasted work to get a new package
>> ready for dependencies that were eventually dismissed.
>
> Thanks Julie for reminding me of  this thread!
>
> I agree we can only build ui-deps package when the review is about to merge.
> I've proposed https://github.com/rdo-common/openstack-tripleo-ui-deps/pull/19
> which allows us to build the package with the review and patchset
> numbers, before it's merged.
> Please review and we can try it on the next deps update!

Thanks Alan! Let's do that :)

Glad to see the pull request merged. If we're happy with the new
suggested process here, I proposed a patch to update the docs with it
at [1]. Hopefully we can move ahead with this and also merge the patch
to reenable the undercloud job [2] to get back minimal sanity checking
on the UI rpms.

Thanks!

Julie

[1] https://review.openstack.org/#/c/560846/
[2] https://review.openstack.org/#/c/526430/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][ui] Dependency management

[Alan Pevec]

On Mon, Jan 22, 2018 at 9:30 AM, Julie Pichon  wrote:
> On 19 January 2018 at 18:43, Honza Pokorny  wrote:
>> We've recently discovered an issue with the way we handle dependencies for
>> tripleo-ui.  This is an explanation of the problem, and a proposed solution.
>> I'm looking for feedback.
>>
>> Before the upgrade to zuul v3 in TripleO CI, we had two types of jobs for
>> tripleo-ui:
>>
>> * Native npm jobs
>> * Undercloud integration jobs
>>
>> After the upgrade, the integration jobs went away.  Our goal is to add them
>> back.
>>
>> There is a difference in how these two types of jobs handle dependencies.
>> Native npm jobs use the "npm install" command to acquire packages, and
>> undercloud jobs use RPMs.  The tripleo-ui project uses a separate RPM for
>> dependencies called openstack-tripleo-ui-deps.
>>
>> Because of the requirement to use a separate RPM for dependencies, there is 
>> some
>> extra work needed when a new dependency is introduced, or an existing one is
>> upgraded.  Once the patch that introduces the dependency is merged, we have 
>> to
>> increment the version of the -deps package, and rebuild it.  It then shows 
>> up in
>> the yum repos used by the undercloud.
>>
>> To make matters worse, we recently upgraded our infrastructure to nodejs 
>> 8.9.4
>> and npm 5.6.0 (latest stable).  This makes it so we can't write "purist" 
>> patches
>> that simply introduce a new dependency to package.json, and nothing more.  
>> The
>> code that uses the new dependency must be included.  I tend to think that 
>> each
>> commit should work on its own so this can be seen as a plus.
>>
>> This presents a problem: you can't get a patch that introduces a new 
>> dependency
>> merged because it's not included in the RPM needed by the undercloud ci job.
>>
>> So, here is a proposal on how that might work:
>>
>> 1. Submit a patch for review that introduces the dependency, along with code
>>changes to support it and validate its inclusion
>> 2. Native npm jobs will pass
>> 3. Undercloud gate job will fail because the dependency isn't in -deps RPM
>> 4. We ask RDO to review for licensing
>> 5. Once reviewed, new -deps package is built
>> 6. Recheck
>> 7. All jobs pass
>
> Perhaps there should be a step after 3 or 4 to have the patch normally
> reviewed, and wait for it to have two +2s before building the new
> package? Otherwise we may end up with wasted work to get a new package
> ready for dependencies that were eventually dismissed.

Thanks Julie for reminding me of  this thread!

I agree we can only build ui-deps package when the review is about to merge.
I've proposed https://github.com/rdo-common/openstack-tripleo-ui-deps/pull/19
which allows us to build the package with the review and patchset
numbers, before it's merged.
Please review and we can try it on the next deps update!

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][ui] Dependency management

[Julie Pichon]

On 19 January 2018 at 18:43, Honza Pokorny  wrote:
> We've recently discovered an issue with the way we handle dependencies for
> tripleo-ui.  This is an explanation of the problem, and a proposed solution.
> I'm looking for feedback.
>
> Before the upgrade to zuul v3 in TripleO CI, we had two types of jobs for
> tripleo-ui:
>
> * Native npm jobs
> * Undercloud integration jobs
>
> After the upgrade, the integration jobs went away.  Our goal is to add them
> back.
>
> There is a difference in how these two types of jobs handle dependencies.
> Native npm jobs use the "npm install" command to acquire packages, and
> undercloud jobs use RPMs.  The tripleo-ui project uses a separate RPM for
> dependencies called openstack-tripleo-ui-deps.
>
> Because of the requirement to use a separate RPM for dependencies, there is 
> some
> extra work needed when a new dependency is introduced, or an existing one is
> upgraded.  Once the patch that introduces the dependency is merged, we have to
> increment the version of the -deps package, and rebuild it.  It then shows up 
> in
> the yum repos used by the undercloud.
>
> To make matters worse, we recently upgraded our infrastructure to nodejs 8.9.4
> and npm 5.6.0 (latest stable).  This makes it so we can't write "purist" 
> patches
> that simply introduce a new dependency to package.json, and nothing more.  The
> code that uses the new dependency must be included.  I tend to think that each
> commit should work on its own so this can be seen as a plus.
>
> This presents a problem: you can't get a patch that introduces a new 
> dependency
> merged because it's not included in the RPM needed by the undercloud ci job.
>
> So, here is a proposal on how that might work:
>
> 1. Submit a patch for review that introduces the dependency, along with code
>changes to support it and validate its inclusion
> 2. Native npm jobs will pass
> 3. Undercloud gate job will fail because the dependency isn't in -deps RPM
> 4. We ask RDO to review for licensing
> 5. Once reviewed, new -deps package is built
> 6. Recheck
> 7. All jobs pass

Perhaps there should be a step after 3 or 4 to have the patch normally
reviewed, and wait for it to have two +2s before building the new
package? Otherwise we may end up with wasted work to get a new package
ready for dependencies that were eventually dismissed.

Julie

> There is the obvious issue with building an RPM based on an unmerged patch.
>
> What do you think?  Is that possible?  Any other solutions?
>
> Honza Pokorny

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][ui] Dependency management

[Jason E. Rist]

On 01/19/2018 11:43 AM, Honza Pokorny wrote:
> We've recently discovered an issue with the way we handle dependencies for
> tripleo-ui.  This is an explanation of the problem, and a proposed solution.
> I'm looking for feedback.
>
> Before the upgrade to zuul v3 in TripleO CI, we had two types of jobs for
> tripleo-ui:
>
> * Native npm jobs
> * Undercloud integration jobs
>
> After the upgrade, the integration jobs went away.  Our goal is to add them
> back.
>
> There is a difference in how these two types of jobs handle dependencies.
> Native npm jobs use the "npm install" command to acquire packages, and
> undercloud jobs use RPMs.  The tripleo-ui project uses a separate RPM for
> dependencies called openstack-tripleo-ui-deps.
>
> Because of the requirement to use a separate RPM for dependencies, there is 
> some
> extra work needed when a new dependency is introduced, or an existing one is
> upgraded.  Once the patch that introduces the dependency is merged, we have to
> increment the version of the -deps package, and rebuild it.  It then shows up 
> in
> the yum repos used by the undercloud.
>
> To make matters worse, we recently upgraded our infrastructure to nodejs 8.9.4
> and npm 5.6.0 (latest stable).  This makes it so we can't write "purist" 
> patches
> that simply introduce a new dependency to package.json, and nothing more.  The
> code that uses the new dependency must be included.  I tend to think that each
> commit should work on its own so this can be seen as a plus.
>
> This presents a problem: you can't get a patch that introduces a new 
> dependency
> merged because it's not included in the RPM needed by the undercloud ci job.
>
> So, here is a proposal on how that might work:
>
> 1. Submit a patch for review that introduces the dependency, along with code
>    changes to support it and validate its inclusion
> 2. Native npm jobs will pass
> 3. Undercloud gate job will fail because the dependency isn't in -deps RPM
> 4. We ask RDO to review for licensing
> 5. Once reviewed, new -deps package is built
> 6. Recheck
> 7. All jobs pass
>
> There is the obvious issue with building an RPM based on an unmerged patch.
>
> What do you think?  Is that possible?  Any other solutions?
>
> Honza Pokorny
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
I can't think of another way that would work better at the moment. I'd love if 
we had even less complexity here, but for the time being just any working 
solution is a positive.  Thanks for proposing, and +1 to your solution.

-J

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo][ui] Dependency management

[Honza Pokorny]

We've recently discovered an issue with the way we handle dependencies for
tripleo-ui.  This is an explanation of the problem, and a proposed solution.
I'm looking for feedback.

Before the upgrade to zuul v3 in TripleO CI, we had two types of jobs for
tripleo-ui:

* Native npm jobs
* Undercloud integration jobs

After the upgrade, the integration jobs went away.  Our goal is to add them
back.

There is a difference in how these two types of jobs handle dependencies.
Native npm jobs use the "npm install" command to acquire packages, and
undercloud jobs use RPMs.  The tripleo-ui project uses a separate RPM for
dependencies called openstack-tripleo-ui-deps.

Because of the requirement to use a separate RPM for dependencies, there is some
extra work needed when a new dependency is introduced, or an existing one is
upgraded.  Once the patch that introduces the dependency is merged, we have to
increment the version of the -deps package, and rebuild it.  It then shows up in
the yum repos used by the undercloud.

To make matters worse, we recently upgraded our infrastructure to nodejs 8.9.4
and npm 5.6.0 (latest stable).  This makes it so we can't write "purist" patches
that simply introduce a new dependency to package.json, and nothing more.  The
code that uses the new dependency must be included.  I tend to think that each
commit should work on its own so this can be seen as a plus.

This presents a problem: you can't get a patch that introduces a new dependency
merged because it's not included in the RPM needed by the undercloud ci job.

So, here is a proposal on how that might work:

1. Submit a patch for review that introduces the dependency, along with code
   changes to support it and validate its inclusion
2. Native npm jobs will pass
3. Undercloud gate job will fail because the dependency isn't in -deps RPM 
4. We ask RDO to review for licensing
5. Once reviewed, new -deps package is built
6. Recheck
7. All jobs pass

There is the obvious issue with building an RPM based on an unmerged patch.

What do you think?  Is that possible?  Any other solutions?

Honza Pokorny

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev