Re: Packaging buildbot-www

[Ryan Schmidt]

On May 29, 2019, at 06:21, Rajdeep Bharati wrote:

> On Wed, May 29, 2019 at 1:12 AM Rainer Müller wrote:
> 
>> Out of curiosity, where does this screenshot come from?
> 
> It's present in the UI: https://nine.buildbot.net/#/about 


>> To me it looks like all of these (minified) JavaScript libraries and
>> also FontAwesome are already included in the release tarball. You do not
>> need to create ports for these, the software will use the included copies.
> 
> I didn't understand how it would use the included copies.

buildbot-www is a web server. It presumably knows that it should serve files 
from the static directory. index.html presumably includes styles.css and 
scripts.js, and styles.css presumably references the fonts and images.


> $ tar xf buildbot-www-2.3.1.tar.gz
> $ $ tree buildbot-www-2.3.1/buildbot_www/static
> buildbot-www-2.3.1/buildbot_www/static/
> ├── d3.min.js
> ├── fonts
> │   ├── FontAwesome.otf
> │   ├── fontawesome-webfont.eot
> │   ├── fontawesome-webfont.svg
> │   ├── fontawesome-webfont.ttf
> │   └── fontawesome-webfont.woff
> ├── img
> │   ├── favicon.ico
> │   ├── icon.png
> │   ├── icon.svg
> │   ├── icon16.svg
> │   └── nobody.png
> ├── index.html
> ├── scripts.js
> ├── styles.css
> └── tests.js

Re: Packaging buildbot-www

[Rajdeep Bharati]

On Wed, May 29, 2019 at 1:12 AM Rainer Müller  wrote:

> On 28.05.19 09:11, Rajdeep Bharati wrote:
> > I am packaging buildbot 2.x as a port. However, a key part of buildbot 2
> > is the web interface, buildbot-www, which uses many javascript
> > dependencies (those aren't available as MacPorts packages yet). Please
> > find the attached image which shows the dependencies.
> >
> >  Screenshot 2019-05-28 at 11.36.55 AM.png
> > <
> https://drive.google.com/file/d/1CnjcmMvd4fsXtJqhjrTRGVC0Rfn2s9gW/view?usp=drive_web
> >
> >
> > What would be the right way to package this? Would it be a python port?
> > Would it be better to package all of the above JS libraries?
>
> Out of curiosity, where does this screenshot come from?
>
> It's present in the UI: https://nine.buildbot.net/#/about

> To me it looks like all of these (minified) JavaScript libraries and
> also FontAwesome are already included in the release tarball. You do not
> need to create ports for these, the software will use the included copies.
>
> I didn't understand how it would use the included copies.

> $ tar xf buildbot-www-2.3.1.tar.gz
> $ $ tree buildbot-www-2.3.1/buildbot_www/static
> buildbot-www-2.3.1/buildbot_www/static/
> ├── d3.min.js
> ├── fonts
> │   ├── FontAwesome.otf
> │   ├── fontawesome-webfont.eot
> │   ├── fontawesome-webfont.svg
> │   ├── fontawesome-webfont.ttf
> │   └── fontawesome-webfont.woff
> ├── img
> │   ├── favicon.ico
> │   ├── icon.png
> │   ├── icon.svg
> │   ├── icon16.svg
> │   └── nobody.png
> ├── index.html
> ├── scripts.js
> ├── styles.css
> └── tests.js
>
> Rainer
>

Rajdeep

Re: Packaging buildbot-www

[Rainer Müller]

On 28.05.19 09:11, Rajdeep Bharati wrote:
> I am packaging buildbot 2.x as a port. However, a key part of buildbot 2
> is the web interface, buildbot-www, which uses many javascript
> dependencies (those aren't available as MacPorts packages yet). Please
> find the attached image which shows the dependencies.
> 
>  Screenshot 2019-05-28 at 11.36.55 AM.png
> 
> 
> What would be the right way to package this? Would it be a python port?
> Would it be better to package all of the above JS libraries?

Out of curiosity, where does this screenshot come from?

To me it looks like all of these (minified) JavaScript libraries and
also FontAwesome are already included in the release tarball. You do not
need to create ports for these, the software will use the included copies.

$ tar xf buildbot-www-2.3.1.tar.gz
$ $ tree buildbot-www-2.3.1/buildbot_www/static
buildbot-www-2.3.1/buildbot_www/static/
├── d3.min.js
├── fonts
│   ├── FontAwesome.otf
│   ├── fontawesome-webfont.eot
│   ├── fontawesome-webfont.svg
│   ├── fontawesome-webfont.ttf
│   └── fontawesome-webfont.woff
├── img
│   ├── favicon.ico
│   ├── icon.png
│   ├── icon.svg
│   ├── icon16.svg
│   └── nobody.png
├── index.html
├── scripts.js
├── styles.css
└── tests.js

Rainer

Re: Packaging buildbot-www

[Rajdeep Bharati]

Sure. I have created a draft PR:
https://github.com/macports/macports-ports/pull/4489
I am having some trouble with the tests and post-destroot, etc. So I
haven't added it yet.

Rajdeep

On Tue, May 28, 2019 at 6:34 PM Mojca Miklavec  wrote:

> On Tue, 28 May 2019 at 14:38, Ryan Schmidt wrote:
> > On May 28, 2019, at 02:11, Rajdeep Bharati wrote:
> >
> > > I am packaging buildbot 2.x as a port.
> >
> > That's great, thanks. I would suggest updating (and where applicable
> renaming) the existing buildbot ports. I would not make new "buildbot2"
> ports, since buildbot 2.x is not different from buildbot 1.x; rather,
> buildbot 0.8 and earlier is different from buildbot 0.9 and later.
> >
> > Since we still need buildbot 0.8 for our current buildbot setup, which
> is installed using MacPorts, I would like to see new buildbot-0.8 ports
> that install the old 0.8 versions. You don't necessarily have to be the one
> to do that; I might get to it at some point.
>
> Once we have the new ports working it should be trivial for anyone of
> us to take care of this renaming part.
> And yes, I would somehow rename the existing ports to buildbot-0.8 or
> so, and make buildbot whatever the current version is.
>
> Also:
> - The buildbot-worker now has a different name than the old
> buildbot-slave, so there we won't even have the name conflict (it
> would still be nice to provide consistent naming though, so 0.8 suffix
> sounds reasonable).
> - Most of what's in buildbot would probably be packages as a python
> module (I guess?).
>
> We can discuss further once Rajdeep submits the ports. (To Rajdeep:
> it's also not a problem to submit what you currently have as a Draft
> PR.)
>
> Mojca
>

Re: Packaging buildbot-www

[Mojca Miklavec]

On Tue, 28 May 2019 at 14:38, Ryan Schmidt wrote:
> On May 28, 2019, at 02:11, Rajdeep Bharati wrote:
>
> > I am packaging buildbot 2.x as a port.
>
> That's great, thanks. I would suggest updating (and where applicable 
> renaming) the existing buildbot ports. I would not make new "buildbot2" 
> ports, since buildbot 2.x is not different from buildbot 1.x; rather, 
> buildbot 0.8 and earlier is different from buildbot 0.9 and later.
>
> Since we still need buildbot 0.8 for our current buildbot setup, which is 
> installed using MacPorts, I would like to see new buildbot-0.8 ports that 
> install the old 0.8 versions. You don't necessarily have to be the one to do 
> that; I might get to it at some point.

Once we have the new ports working it should be trivial for anyone of
us to take care of this renaming part.
And yes, I would somehow rename the existing ports to buildbot-0.8 or
so, and make buildbot whatever the current version is.

Also:
- The buildbot-worker now has a different name than the old
buildbot-slave, so there we won't even have the name conflict (it
would still be nice to provide consistent naming though, so 0.8 suffix
sounds reasonable).
- Most of what's in buildbot would probably be packages as a python
module (I guess?).

We can discuss further once Rajdeep submits the ports. (To Rajdeep:
it's also not a problem to submit what you currently have as a Draft
PR.)

Mojca

Re: Packaging buildbot-www

[Ryan Schmidt]

On May 28, 2019, at 02:11, Rajdeep Bharati wrote:

> I am packaging buildbot 2.x as a port.

That's great, thanks. I would suggest updating (and where applicable renaming) 
the existing buildbot ports. I would not make new "buildbot2" ports, since 
buildbot 2.x is not different from buildbot 1.x; rather, buildbot 0.8 and 
earlier is different from buildbot 0.9 and later.

Since we still need buildbot 0.8 for our current buildbot setup, which is 
installed using MacPorts, I would like to see new buildbot-0.8 ports that 
install the old 0.8 versions. You don't necessarily have to be the one to do 
that; I might get to it at some point.

Re: Packaging buildbot-www

[Rajdeep Bharati]

On Tue, May 28, 2019 at 2:39 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> [Removing the -users mailing list as regular users should not be
> concerned with ugly implementation details of MacPorts :), and CC-ing
> Karan, Renee & Cyril.]
>
> On Tue, 28 May 2019 at 08:56, Rajdeep Bharati wrote:
> >
> > Dear developers,
> >
> > I am packaging buildbot 2.x as a port. However, a key part of buildbot 2
> is the web interface, buildbot-www, which uses many javascript dependencies
> (those aren't available as MacPorts packages yet). Please find the attached
> image which shows the dependencies.
> >
> > https://drive.google.com/file/d/1CnjcmMvd4fsXtJqhjrTRGVC0Rfn2s9gW/view
> >
> >  Screenshot 2019-05-28 at 11.36.55 AM.png
> >
> > What would be the right way to package this? Would it be a python port?
> > Would it be better to package all of the above JS libraries?
> >
> > It would be great if someone could throw some light on this.
> >
> > Thanks and Regards
> > Rajdeep
>
> If I didn't make an "OCR error" myself, here's the list of packages in
> plain text:
> jquery, angular, angular-animate, angular-bootstrap,
> angular-ui-router, angular-recursion, loadash, guanlecoja-ui, moment,
> d3, font-awesome, bootstrap, buildbot-data,
> angular-bootstrap-multiselect
>
> I'm not entirely sure about the best strategy either. It would
> definitely make sense to check what other distributions are doing and
> also get recommendations from buildbot developers. I'm pretty sure
> that FreeBSD, Debian, RedHat etc. all package buildbot and it would
> make sense to check what package dependencies they declare / what
> files the packages install.


As you can see in the freebsd implementation:
https://svnweb.freebsd.org/ports/head/devel/py-buildbot-www/Makefile?revision=501623=markup
They have specified the pip dependencies only (although I don't know how
freshports works internally)

> I would say that it sounds acceptable to install those dependencies on
> the fly and mention installation instructions in port notes as the
> first step (I'm just not sure where to install those files). I would
> like to avoid you having to waste too much time trying to get all of
> the javascript dependencies in place instead of spending some
> productive time fixing bugs and implementing new functionality in
> buildbot. Porting buildbot itself is one thing, dealing with a lot of
> other unrelated dependencies something else, and one thing that we
> already struggled with in the past, not being quite sure what
> precisely to do.
>
> On the other hand, this potentially sounds like a good real-world test
> case for Karan who already wrote both npm backend for upt and the
> macports frontend. I'm not sure how mature that code is and what Karan
> is currently working on, but it might be worth trying it out.
> (The following is meant more as a joke, I know that we don't have
> nearly sufficient information available, but ideal upt
> frontend/backend would already take care of all the dependency hell
> and automatically declare & create also the javascript dependencies
> for buildbot :) :) :)
>
> Mojca
>

The portfiles that I have written (using the awesome upt-macports tool) are
analogous to
freshports https://www.freshports.org/devel/py-buildbot
https://www.freshports.org/devel/py-buildbot-pkg
https://www.freshports.org/devel/py-buildbot-worker
But the py-buildbot-www Portfile doesn't work in the desired manner since
the JS dependencies
haven't been installed.

Re: Packaging buildbot-www

[Mojca Miklavec]

Dear Rajdeep,

[Removing the -users mailing list as regular users should not be
concerned with ugly implementation details of MacPorts :), and CC-ing
Karan, Renee & Cyril.]

On Tue, 28 May 2019 at 08:56, Rajdeep Bharati wrote:
>
> Dear developers,
>
> I am packaging buildbot 2.x as a port. However, a key part of buildbot 2 is 
> the web interface, buildbot-www, which uses many javascript dependencies 
> (those aren't available as MacPorts packages yet). Please find the attached 
> image which shows the dependencies.
>
> https://drive.google.com/file/d/1CnjcmMvd4fsXtJqhjrTRGVC0Rfn2s9gW/view
>
>  Screenshot 2019-05-28 at 11.36.55 AM.png
>
> What would be the right way to package this? Would it be a python port?
> Would it be better to package all of the above JS libraries?
>
> It would be great if someone could throw some light on this.
>
> Thanks and Regards
> Rajdeep

If I didn't make an "OCR error" myself, here's the list of packages in
plain text:
jquery, angular, angular-animate, angular-bootstrap,
angular-ui-router, angular-recursion, loadash, guanlecoja-ui, moment,
d3, font-awesome, bootstrap, buildbot-data,
angular-bootstrap-multiselect

I'm not entirely sure about the best strategy either. It would
definitely make sense to check what other distributions are doing and
also get recommendations from buildbot developers. I'm pretty sure
that FreeBSD, Debian, RedHat etc. all package buildbot and it would
make sense to check what package dependencies they declare / what
files the packages install.

I would say that it sounds acceptable to install those dependencies on
the fly and mention installation instructions in port notes as the
first step (I'm just not sure where to install those files). I would
like to avoid you having to waste too much time trying to get all of
the javascript dependencies in place instead of spending some
productive time fixing bugs and implementing new functionality in
buildbot. Porting buildbot itself is one thing, dealing with a lot of
other unrelated dependencies something else, and one thing that we
already struggled with in the past, not being quite sure what
precisely to do.

On the other hand, this potentially sounds like a good real-world test
case for Karan who already wrote both npm backend for upt and the
macports frontend. I'm not sure how mature that code is and what Karan
is currently working on, but it might be worth trying it out.
(The following is meant more as a joke, I know that we don't have
nearly sufficient information available, but ideal upt
frontend/backend would already take care of all the dependency hell
and automatically declare & create also the javascript dependencies
for buildbot :) :) :)

Mojca

Packaging buildbot-www

[Rajdeep Bharati]

Dear developers,

I am packaging buildbot 2.x as a port. However, a key part of buildbot 2 is
the web interface, buildbot-www, which uses many javascript dependencies
(those aren't available as MacPorts packages yet). Please find the attached
image which shows the dependencies.

 Screenshot 2019-05-28 at 11.36.55 AM.png


What would be the right way to package this? Would it be a python port?
Would it be better to package all of the above JS libraries?

It would be great if someone could throw some light on this.

Thanks and Regards
Rajdeep