Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Rajdeep Bharati]

Ok cool!

Thanks
Rajdeep

On Wed, May 15, 2019 at 1:29 PM Pierre Tardy  wrote:

> Hi Rajdeep,
>
> Buildbot calculates the version number based on tags. You you have to
> fetch all the tags so that the version is correctly calculated.
>
> git fetch --tags --all
>
> then you can re-do the make virtualenv
>
> Regards
> Pierre
>
> Le mar. 14 mai 2019 à 11:39, Rajdeep Bharati
>  a écrit :
> >
> > Thanks for the feedback!
> >
> > I had a doubt in buildbot:
> > When I install buildbot from source (github), then why is it showing
> version 2.1.1dev... installed? Shouldn't it show the latest version
> (2.3.0)? It is up to date with the master branch.
> >
> > Thank you.
> > Rajdeep
> >
> > On Fri, May 10, 2019 at 3:17 AM Mojca Miklavec 
> wrote:
> >>
> >> On Wed, 8 May 2019 at 21:00, Rajdeep Bharati wrote:
> >> >
> >> > Hi,
> >> >
> >> > I created some issues and found some which would be relevant for the
> project (as discussed in the IRC meeting):
> >> >
> >> > https://github.com/buildbot/buildbot/issues/4760
> >> > https://github.com/buildbot/buildbot/issues/4761
> >> > https://github.com/buildbot/buildbot/issues/4709
> >> > https://github.com/buildbot/buildbot/issues/3471
> >> > https://github.com/buildbot/buildbot/issues/3470
> >> > https://github.com/buildbot/buildbot/issues/4750
> >>
> >> Thank you.
> >> (Nice to see that I'm not the first one requesting these :)
> >>
> >> I assume that we'll also want to add other items for the rest of the
> >> summer to that list?
> >>
> >> > @Mojca Miklavec Yesterday you mentioned:
> >> >>
> >> >> 16:33:15  regarding the features of the default waterfall
> view, another problem I saw was that unless one of the builds was done
> recently, the builder doesn't even show in waterfall; for example, there is
> no texlive or pplib displayed on
> https://build.contextgarden.net/#/waterfall
> >>
> >> And later Pierre said this was a feature :)
> >>
> >> > However, I am unable to find this problem in
> https://nine.buildbot.net/#/waterfall (it's showing the builders not
> having recent builds). I'm not sure (it might have been configured in that
> manner in nine.buildbot, and not present by default).
> >> >
> >> > I will try to replicate this issue in my local setup.
> >>
> >> If you look at
> >> https://build.contextgarden.net/#/console
> >> you can see that there is a huge number of "fake commits" from one
> >> project (from periodic scheduler) after the last build of another
> >> project was done. Maybe there's a setting which tells you how many
> >> latest builds / commits to fetch when drawing the waterfall view. I
> >> didn't try to investigate what limits the number of displayed
> >> builders.
> >>
> >> Now: for me by far more important missing feature is in fact support
> >> for filtering on both console and waterfall. Once filtering and
> >> removal of old builders gets implemented, this particular issue will
> >> be less important.
> >>
> >> Mojca
>

Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Pierre Tardy]

Hi Rajdeep,

Buildbot calculates the version number based on tags. You you have to
fetch all the tags so that the version is correctly calculated.

git fetch --tags --all

then you can re-do the make virtualenv

Regards
Pierre

Le mar. 14 mai 2019 à 11:39, Rajdeep Bharati
 a écrit :
>
> Thanks for the feedback!
>
> I had a doubt in buildbot:
> When I install buildbot from source (github), then why is it showing version 
> 2.1.1dev... installed? Shouldn't it show the latest version (2.3.0)? It is up 
> to date with the master branch.
>
> Thank you.
> Rajdeep
>
> On Fri, May 10, 2019 at 3:17 AM Mojca Miklavec  wrote:
>>
>> On Wed, 8 May 2019 at 21:00, Rajdeep Bharati wrote:
>> >
>> > Hi,
>> >
>> > I created some issues and found some which would be relevant for the 
>> > project (as discussed in the IRC meeting):
>> >
>> > https://github.com/buildbot/buildbot/issues/4760
>> > https://github.com/buildbot/buildbot/issues/4761
>> > https://github.com/buildbot/buildbot/issues/4709
>> > https://github.com/buildbot/buildbot/issues/3471
>> > https://github.com/buildbot/buildbot/issues/3470
>> > https://github.com/buildbot/buildbot/issues/4750
>>
>> Thank you.
>> (Nice to see that I'm not the first one requesting these :)
>>
>> I assume that we'll also want to add other items for the rest of the
>> summer to that list?
>>
>> > @Mojca Miklavec Yesterday you mentioned:
>> >>
>> >> 16:33:15  regarding the features of the default waterfall view, 
>> >> another problem I saw was that unless one of the builds was done 
>> >> recently, the builder doesn't even show in waterfall; for example, there 
>> >> is no texlive or pplib displayed on 
>> >> https://build.contextgarden.net/#/waterfall
>>
>> And later Pierre said this was a feature :)
>>
>> > However, I am unable to find this problem in 
>> > https://nine.buildbot.net/#/waterfall (it's showing the builders not 
>> > having recent builds). I'm not sure (it might have been configured in that 
>> > manner in nine.buildbot, and not present by default).
>> >
>> > I will try to replicate this issue in my local setup.
>>
>> If you look at
>> https://build.contextgarden.net/#/console
>> you can see that there is a huge number of "fake commits" from one
>> project (from periodic scheduler) after the last build of another
>> project was done. Maybe there's a setting which tells you how many
>> latest builds / commits to fetch when drawing the waterfall view. I
>> didn't try to investigate what limits the number of displayed
>> builders.
>>
>> Now: for me by far more important missing feature is in fact support
>> for filtering on both console and waterfall. Once filtering and
>> removal of old builders gets implemented, this particular issue will
>> be less important.
>>
>> Mojca

Re: Slack-like chat (also for GSOC)

[Umesh Singla]

I remember seeing Gitter.im being used by more than a usual number of open
source communities in the past. While I am personally fine with Github
Issues/PRs and emails, a chat-like application does come in handy for quick
replies.

I logged in just now to my old account from over 2 years back and I see no
limits on the messages. You can sign in with Github/Twitter. There is this
idea of community and rooms like IRC, and while anyone can read the
community messages by visiting the link, one needs to sign in to talk. It's
linked to a Github organization and owners of the organization act like
admins. It also allows for one-on-one conversations and rooms where only
invited users can join.

An example would be: https://gitter.im/coala/coala

@Mojca would you be willing to give this https://gitter.im/macports-gsoc a
try? I tried creating one suitable for us. Let me know if you are able to
see the secret room. :)

Thanks,
Umesh

On Wed, May 15, 2019 at 2:02 AM Mojca Miklavec  wrote:

> On Tue, 14 May 2019 at 18:11, Rainer Müller wrote:
> >
> > I think Slack workspaces are also always invite only, but please correct
> > me if I am wrong (maybe only for paid plans?). Assuming we might want to
> > replace IRC one day as the official development and support chat, that
> > would not work well.
> >
> > I think it is important that this would be open for anyone interested to
> > join, with private group chats as an optional feature.
>
> +1
>
> > > We don't need to all agree at once. I don't see anything wrong with
> > > giving it some testing first and decide what's best (no need to ask
> > > everyone to turn IRC off :).
> >
> > I recently tried the Matrix IRC Bridge to FreeNode IRC. By using riot.im
> > it actually works quite well and was easy to set up. It also solves the
> > "always-on" problem of IRC as it acts like a bouncer.
> >
> > >>> Would it be realistic to install such a service on breaburn if
> needed?
> > >>> (Or is it too complex / too much work?)
> > >>
> > >> I'd prefer a SaaS offering here. Self-hosting just increases the
> > >> maintenance burden and I don't think we need the configurability.
> >
> > For the self-hosted options, Rocket Chat would be an option. However,
> > when we used it at work, after a while I started to miss some kind of
> > threading for longer conversations. Although we also usually do not have
> > long conversations or that much activity on IRC, so maybe this is not
> > that important here.
>
> I don't have any experience with Matrix, but I maybe I should try it once.
>
> I'm not familiar with Rocket Chat either, but if you missed a feature,
> I trust your opinion.
>
> I do believe that longer conversations are important. Think of GSOC,
> where the same project runs for 5 months or longer. It does make sense
> to keep it well-organised.
>
> Zulip offers topics (which they heavily advertise as one of their
> "superpowers") which I find to be quite a nice "substitute" for
> threads like those in emails. If we pick that one, I would certainly
> go for GitHub OAuth and IRC mirror.
>
> I would discard the idea of using Slack. Based on general feedback
> that probably leaves the following top candidates?
> - Matrix (might work without self-hosting)
> - Zulip
> - Mattermost
>
> Rainer, you did not answer about whether you would be willing to try
> to install / maintain one of those on the server if we wanted to
> self-host the chat?
>
> Regarding Matrix: is anyone willing to set up one ("in the cloud") for
> testing?
>
> Mojca
>

Re: GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

Dear Arjun,

I'm CC-ing the list as that's potentially interesting to others as well.

On Tue, 14 May 2019 at 19:45, Arjun Salyan wrote:

> Summary of today's discussion:
>
> 1. *Repository: *The existing repository for the web app would be moved
> from the staging area to the official MacPorts organisations's
> repositories. And then we will setup issues and milestones (after making
> changes to the timeline, if needed). The previous macports-webapp
> repository (from last year) can be key separate to the new one by making
> some name changes.
>
> 2. *Docker Containers: *The infra team has suggested that for initial
> deployment of the app we make a docker container so that our app would not
> pose any security issues to rest of the server.
>
> 3. *mpstats: *We can add a new port (mpstats 2.0 or some other name) to
> submit statistics to our app, so that when we start working on stats part-
> we will have some data to test the setup. The newer version would drop
> submission of "inactive ports" and "gcc".
>
> 4. *Buildbot*: I will be learning buildbot more thoroughly.
>

Apart from
https://github.com/macports/macports-infrastructure/tree/master/buildbot
see also
https://github.com/rajdeepbharati/macports-buildbot/

Students with similar projects are certainly encouraged to follow (and
provide feedback) to the other similar projects.


> 5. *Database Design: *It would be better to store all weekly submissions
> instead of just the latest one in the base table (the one which is used to
> populate all other tables).
>
> 6. *Variants: *A low priority task. We could pickup default variant for
> any port from our database of port info.
>
> I hope I haven't missed anything important.
>
> I have set up the app to receive submissions from mpstats at:
> https://frozen-falls-98471.herokuapp.com/statistics/submit/ . I just
> replaced the url in mpstats and it is working correctly.
>
> Thank You
>

I forgot to mention another super important point. You should use this time
to get familiar with writing:

   - *Unit tests*: You already created an empty file, but it's about high
   time to populate it with some real tests, like preparing a very simple set
   of portfiles, generate json, enter them to database and check whether all
   the tables are exactly as we want them to be (in particular with respect to
   authors, categories, ...), and then some checks for subsequent port
   updates, ... Those tests could also run on Travis (or some other service).

Ad:

   - *Database design*: prepare the design (for statistics submissions in
   particular). My suggestion was for the port installation statistics to
   store something like (submission_id, user_uuid, timestamp, os_version, ...)
   in one table, and something like (submission_id, port, version,
   is_requested, ) to another table, make a view
   joining the two, and then be able to answer queries like "how many distinct
   users had port 'python27' installed between 2019-05-01 and 2019-05-31" with
   some advanced sql statements (I hope they are still supported by Django),
   for arbitrary dates etc. Results could of course later be cached for faster
   rendering. This might need some playing around.

Mojca

Re: Slack-like chat (also for GSOC)

[Mojca Miklavec]

On Tue, 14 May 2019 at 18:11, Rainer Müller wrote:
>
> I think Slack workspaces are also always invite only, but please correct
> me if I am wrong (maybe only for paid plans?). Assuming we might want to
> replace IRC one day as the official development and support chat, that
> would not work well.
>
> I think it is important that this would be open for anyone interested to
> join, with private group chats as an optional feature.

+1

> > We don't need to all agree at once. I don't see anything wrong with
> > giving it some testing first and decide what's best (no need to ask
> > everyone to turn IRC off :).
>
> I recently tried the Matrix IRC Bridge to FreeNode IRC. By using riot.im
> it actually works quite well and was easy to set up. It also solves the
> "always-on" problem of IRC as it acts like a bouncer.
>
> >>> Would it be realistic to install such a service on breaburn if needed?
> >>> (Or is it too complex / too much work?)
> >>
> >> I'd prefer a SaaS offering here. Self-hosting just increases the
> >> maintenance burden and I don't think we need the configurability.
>
> For the self-hosted options, Rocket Chat would be an option. However,
> when we used it at work, after a while I started to miss some kind of
> threading for longer conversations. Although we also usually do not have
> long conversations or that much activity on IRC, so maybe this is not
> that important here.

I don't have any experience with Matrix, but I maybe I should try it once.

I'm not familiar with Rocket Chat either, but if you missed a feature,
I trust your opinion.

I do believe that longer conversations are important. Think of GSOC,
where the same project runs for 5 months or longer. It does make sense
to keep it well-organised.

Zulip offers topics (which they heavily advertise as one of their
"superpowers") which I find to be quite a nice "substitute" for
threads like those in emails. If we pick that one, I would certainly
go for GitHub OAuth and IRC mirror.

I would discard the idea of using Slack. Based on general feedback
that probably leaves the following top candidates?
- Matrix (might work without self-hosting)
- Zulip
- Mattermost

Rainer, you did not answer about whether you would be willing to try
to install / maintain one of those on the server if we wanted to
self-host the chat?

Regarding Matrix: is anyone willing to set up one ("in the cloud") for testing?

Mojca

Re: Slack-like chat (also for GSOC)

[Mojca Miklavec]

On Tue, 14 May 2019 at 19:15, Nils Breunese wrote:
>
> Slack provides shareable invite links, but they expire. This blog post lists 
> some other options:
>
> https://intoli.com/blog/make-a-public-slack-community/
>
> Looks like it requires some work, but it’s possible to setup.
> Note that I don’t have a big opinion on whether Slack is a good idea or not.

Apart from being closed-source (well, so is GitHub!), Slack does the
job well only when you are either a paying customer, or when you don't
care about the chat history. Given the good alternatives, I'm not too
thrilled to use it, except if we need to set up something quickly.

Mojca

Re: Slack-like chat (also for GSOC)

[Nils Breunese]

Rainer Müller wrote:

> I think Slack workspaces are also always invite only, but please correct
> me if I am wrong (maybe only for paid plans?).

Slack provides shareable invite links, but they expire. This blog post lists 
some other options:

https://intoli.com/blog/make-a-public-slack-community/

Looks like it requires some work, but it’s possible to setup.

Note that I don’t have a big opinion on whether Slack is a good idea or not.

Nils.

Re: Slack-like chat (also for GSOC)

[Rainer Müller]

On 12.05.19 21:49, Mojca Miklavec wrote:
> On Sun, 12 May 2019 at 21:15, Clemens Lang wrote:
>> On Thu, May 09, 2019 at 11:38:16PM +0200, Mojca Miklavec wrote:
>>> I know that our official channels only include mailing lists, IRC and
>>> PR/ticket discussion so far, but I'm curious about your stand with
>>> respect to setting up a service like Zulip Chat / Rocket Chat / ...
>>> for aiding the GSOC communication when higher frequency of potentially
>>> simple questions would be needed. I know IRC could fulfill such a
>>> task, but it doesn't work too well for me. (I'm offline when at work,
>>> cannot really use phone efficiently for communication etc..)
>>
>> I don't have particularly strong feelings about any of these solutions
>> with a slight tendency towards an open source solution.
> 
> I would prefer OpenSource as well.
> 
> My experience from work was that we started with Slack (which btw.
> works very nice) and initially everyone said that we would use it just
> as real-time chat and that nobody would ever need the archives. That
> is: until someone needed them. At that point the solution was
> considered way to expensive to be used long-term and we switched to
> something else.

I think Slack workspaces are also always invite only, but please correct
me if I am wrong (maybe only for paid plans?). Assuming we might want to
replace IRC one day as the official development and support chat, that
would not work well.

I think it is important that this would be open for anyone interested to
join, with private group chats as an optional feature.

>> I do see the
>> benefits compared to IRC also from experience at work. If that's
>> something we as a community agree on, I wouldn't mind switching my IRC
>> presence.
> 
> We don't need to all agree at once. I don't see anything wrong with
> giving it some testing first and decide what's best (no need to ask
> everyone to turn IRC off :).

I recently tried the Matrix IRC Bridge to FreeNode IRC. By using riot.im
it actually works quite well and was easy to set up. It also solves the
"always-on" problem of IRC as it acts like a bouncer.

>>> Would it be realistic to install such a service on breaburn if needed?
>>> (Or is it too complex / too much work?)
>>
>> I'd prefer a SaaS offering here. Self-hosting just increases the
>> maintenance burden and I don't think we need the configurability.

For the self-hosted options, Rocket Chat would be an option. However,
when we used it at work, after a while I started to miss some kind of
threading for longer conversations. Although we also usually do not have
long conversations or that much activity on IRC, so maybe this is not
that important here.

Rainer

Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Rajdeep Bharati]

Thanks for the feedback!

I had a doubt in buildbot:
When I install buildbot from source (github), then why is it showing
version 2.1.1dev... installed? Shouldn't it show the latest version
(2.3.0)? It is up to date with the master branch.

Thank you.
Rajdeep

On Fri, May 10, 2019 at 3:17 AM Mojca Miklavec  wrote:

> On Wed, 8 May 2019 at 21:00, Rajdeep Bharati wrote:
> >
> > Hi,
> >
> > I created some issues and found some which would be relevant for the
> project (as discussed in the IRC meeting):
> >
> > https://github.com/buildbot/buildbot/issues/4760
> > https://github.com/buildbot/buildbot/issues/4761
> > https://github.com/buildbot/buildbot/issues/4709
> > https://github.com/buildbot/buildbot/issues/3471
> > https://github.com/buildbot/buildbot/issues/3470
> > https://github.com/buildbot/buildbot/issues/4750
>
> Thank you.
> (Nice to see that I'm not the first one requesting these :)
>
> I assume that we'll also want to add other items for the rest of the
> summer to that list?
>
> > @Mojca Miklavec Yesterday you mentioned:
> >>
> >> 16:33:15  regarding the features of the default waterfall view,
> another problem I saw was that unless one of the builds was done recently,
> the builder doesn't even show in waterfall; for example, there is no
> texlive or pplib displayed on https://build.contextgarden.net/#/waterfall
>
> And later Pierre said this was a feature :)
>
> > However, I am unable to find this problem in
> https://nine.buildbot.net/#/waterfall (it's showing the builders not
> having recent builds). I'm not sure (it might have been configured in that
> manner in nine.buildbot, and not present by default).
> >
> > I will try to replicate this issue in my local setup.
>
> If you look at
> https://build.contextgarden.net/#/console
> you can see that there is a huge number of "fake commits" from one
> project (from periodic scheduler) after the last build of another
> project was done. Maybe there's a setting which tells you how many
> latest builds / commits to fetch when drawing the waterfall view. I
> didn't try to investigate what limits the number of displayed
> builders.
>
> Now: for me by far more important missing feature is in fact support
> for filtering on both console and waterfall. Once filtering and
> removal of old builders gets implemented, this particular issue will
> be less important.
>
> Mojca
>

Re: Slack-like chat (also for GSOC)

[Mark Anderson]

We could also try discord. It’s free like slack, but doesn’t have
limitations in the free version. RocketChat or the like might be worth it
if we want to run it on heroku or aws or something. With Slack there may be
Open Source project exceptions that let us use the premium stuff for free.
(I’ve been finding a lot of stuff like that. For instance JIRA cloud has
free options for open source.) I don’t mind IRC but I’d prefer to be able
to sign in with my phone since I’m on Eastern time and a lot of you are on
Pacific and various European times so I never know when people are around.

—Mark

On Sun, May 12, 2019 at 6:15 PM Andrew Janke  wrote:

>
> On 5/12/19 3:49 PM, Mojca Miklavec wrote:
> > I looked at Zulip chat which was used for GSOC meeting. That one would
> > also only retain the last few thousand messages unless we payed an
> > amount that we could not afford as a project.
> > Mattermost probably doesn't offer any free SaaS option?
> > Discord would probably work as a free service, but I'm not too keen
> about it.
> >
> > I wouldn't mind cloud solutions, but it probably boils down to either
> > paying, or being limited by X, or hosting it yourself.
>
> You might consider Matrix as a first candidate, then? I've been playing
> around with the various Slack clones for my own group recently, and my
> understanding is that the free hosted SaaS version of Matrix on
> matrix.org both retains all messages permanently, and has a good
> migration path if and when you decide to host your own Matrix instance:
> when users on your own instance ("homeserver") join Rooms originated on
> the matrix.org instance, that will cause propagation of the entire Room
> history to your server. You just need to make sure you configure the
> room so that all members can see the entire room history, not just since
> they joined; I don't recall if that's the default. If you configure it
> so that all members can see the entire history of the room, you then
> have a public archive.
>
> And Matrix seems to be the open source-iest of this crop of clones.
> They're all about the open federated protocol, which is being developed
> in public and ostensibly to be handed over to an independent foundation
> eventually, and then provide an open-source client on top of that.
>
> > Mattermost probably doesn't offer any free SaaS option?
>
> Mattermost only offers a free 30-day trial for their SaaS option. Their
> model is really more about self-hosting, and selling a hosting service.
>
> Cheers,
> Andrew
>
-- 
Sent from Gmail Mobile on iPhone

Re: Slack-like chat (also for GSOC)

[Andrew Janke]

On 5/12/19 3:49 PM, Mojca Miklavec wrote:
> I looked at Zulip chat which was used for GSOC meeting. That one would
> also only retain the last few thousand messages unless we payed an
> amount that we could not afford as a project.
> Mattermost probably doesn't offer any free SaaS option?
> Discord would probably work as a free service, but I'm not too keen about it.
> 
> I wouldn't mind cloud solutions, but it probably boils down to either
> paying, or being limited by X, or hosting it yourself.

You might consider Matrix as a first candidate, then? I've been playing
around with the various Slack clones for my own group recently, and my
understanding is that the free hosted SaaS version of Matrix on
matrix.org both retains all messages permanently, and has a good
migration path if and when you decide to host your own Matrix instance:
when users on your own instance ("homeserver") join Rooms originated on
the matrix.org instance, that will cause propagation of the entire Room
history to your server. You just need to make sure you configure the
room so that all members can see the entire room history, not just since
they joined; I don't recall if that's the default. If you configure it
so that all members can see the entire history of the room, you then
have a public archive.

And Matrix seems to be the open source-iest of this crop of clones.
They're all about the open federated protocol, which is being developed
in public and ostensibly to be handed over to an independent foundation
eventually, and then provide an open-source client on top of that.

> Mattermost probably doesn't offer any free SaaS option?

Mattermost only offers a free 30-day trial for their SaaS option. Their
model is really more about self-hosting, and selling a hosting service.

Cheers,
Andrew

Re: Slack-like chat (also for GSOC)

[Mojca Miklavec]

On Sun, 12 May 2019 at 21:15, Clemens Lang wrote:
> On Thu, May 09, 2019 at 11:38:16PM +0200, Mojca Miklavec wrote:
> > I know that our official channels only include mailing lists, IRC and
> > PR/ticket discussion so far, but I'm curious about your stand with
> > respect to setting up a service like Zulip Chat / Rocket Chat / ...
> > for aiding the GSOC communication when higher frequency of potentially
> > simple questions would be needed. I know IRC could fulfill such a
> > task, but it doesn't work too well for me. (I'm offline when at work,
> > cannot really use phone efficiently for communication etc..)
>
> I don't have particularly strong feelings about any of these solutions
> with a slight tendency towards an open source solution.

I would prefer OpenSource as well.

My experience from work was that we started with Slack (which btw.
works very nice) and initially everyone said that we would use it just
as real-time chat and that nobody would ever need the archives. That
is: until someone needed them. At that point the solution was
considered way to expensive to be used long-term and we switched to
something else.

> I do see the
> benefits compared to IRC also from experience at work. If that's
> something we as a community agree on, I wouldn't mind switching my IRC
> presence.

We don't need to all agree at once. I don't see anything wrong with
giving it some testing first and decide what's best (no need to ask
everyone to turn IRC off :).

> > Would it be realistic to install such a service on breaburn if needed?
> > (Or is it too complex / too much work?)
>
> I'd prefer a SaaS offering here. Self-hosting just increases the
> maintenance burden and I don't think we need the configurability.

One can in principle integrate many things (commit messages, buildbot
reports, pull request notifications, irc gateway, ...)

I looked at Zulip chat which was used for GSOC meeting. That one would
also only retain the last few thousand messages unless we payed an
amount that we could not afford as a project.
Mattermost probably doesn't offer any free SaaS option?
Discord would probably work as a free service, but I'm not too keen about it.

I wouldn't mind cloud solutions, but it probably boils down to either
paying, or being limited by X, or hosting it yourself.

Mojca

Re: Slack-like chat (also for GSOC)

[Clemens Lang]

Hi,

On Thu, May 09, 2019 at 11:38:16PM +0200, Mojca Miklavec wrote:
> I know that our official channels only include mailing lists, IRC and
> PR/ticket discussion so far, but I'm curious about your stand with
> respect to setting up a service like Zulip Chat / Rocket Chat / ...
> for aiding the GSOC communication when higher frequency of potentially
> simple questions would be needed. I know IRC could fulfill such a
> task, but it doesn't work too well for me. (I'm offline when at work,
> cannot really use phone efficiently for communication etc..)

I don't have particularly strong feelings about any of these solutions
with a slight tendency towards an open source solution. I do see the
benefits compared to IRC also from experience at work. If that's
something we as a community agree on, I wouldn't mind switching my IRC
presence.


> Would it be realistic to install such a service on breaburn if needed?
> (Or is it too complex / too much work?)

I'd prefer a SaaS offering here. Self-hosting just increases the
maintenance burden and I don't think we need the configurability.


-- 
Clemens

Re: Slack-like chat (also for GSOC)

[Chris Jones]

Hi,

I think setting up something like this would be a great idea. Like mojca, irc 
is no use to me for a number of reasons. I need to be able to be offline for 
periods, and switch between multiple different devices without missing 
messages, and with irc this is tough.. We use a self hosted mattermost service 
at work, which is basically just like slack... Its macOS and iOS clients are 
pretty good.

Chris



> On 9 May 2019, at 10:48 pm, Ao Liu  wrote:
> 
> I strongly recommend Slack. It has been used in many national labs nowadays. 
> Smartphone apps are also done nicely. 
> 
>> On Thu, May 9, 2019 at 4:43 PM Ruben Di Battista  
>> wrote:
>> I suggest to get a look to Matrix.org. They perfectly integrate with IRC, 
>> but they have bridges for everything. No need to self host (but you can and 
>> it's federated with the rest of the fédération)
>> 
>> It's a pretty cool technology IMHO. 
>> 
>> 
>>> On Thu, 9 May 2019, 23:38 Mojca Miklavec,  wrote:
>>> Hi,
>>> 
>>> I know that our official channels only include mailing lists, IRC and
>>> PR/ticket discussion so far, but I'm curious about your stand with
>>> respect to setting up a service like Zulip Chat / Rocket Chat / ...
>>> for aiding the GSOC communication when higher frequency of potentially
>>> simple questions would be needed. I know IRC could fulfill such a
>>> task, but it doesn't work too well for me. (I'm offline when at work,
>>> cannot really use phone efficiently for communication etc..)
>>> 
>>> Would it be realistic to install such a service on breaburn if needed?
>>> (Or is it too complex / too much work?)
>>> 
>>> I strongly suspect that one of those services also integrates pretty
>>> well with IRC. I've been using Slack, Zulip, Discord, Gitter at
>>> various occasions and I like the way they work.
>>> 
>>> Thank you,
>>> Mojca

Re: Slack-like chat (also for GSOC)

[Ao Liu]

I strongly recommend Slack. It has been used in many national labs
nowadays. Smartphone apps are also done nicely.

On Thu, May 9, 2019 at 4:43 PM Ruben Di Battista 
wrote:

> I suggest to get a look to Matrix.org. They perfectly integrate with IRC,
> but they have bridges for everything. No need to self host (but you can and
> it's federated with the rest of the fédération)
>
> It's a pretty cool technology IMHO.
>
>
> On Thu, 9 May 2019, 23:38 Mojca Miklavec,  wrote:
>
>> Hi,
>>
>> I know that our official channels only include mailing lists, IRC and
>> PR/ticket discussion so far, but I'm curious about your stand with
>> respect to setting up a service like Zulip Chat / Rocket Chat / ...
>> for aiding the GSOC communication when higher frequency of potentially
>> simple questions would be needed. I know IRC could fulfill such a
>> task, but it doesn't work too well for me. (I'm offline when at work,
>> cannot really use phone efficiently for communication etc..)
>>
>> Would it be realistic to install such a service on breaburn if needed?
>> (Or is it too complex / too much work?)
>>
>> I strongly suspect that one of those services also integrates pretty
>> well with IRC. I've been using Slack, Zulip, Discord, Gitter at
>> various occasions and I like the way they work.
>>
>> Thank you,
>> Mojca
>>
>

Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Mojca Miklavec]

On Wed, 8 May 2019 at 21:00, Rajdeep Bharati wrote:
>
> Hi,
>
> I created some issues and found some which would be relevant for the project 
> (as discussed in the IRC meeting):
>
> https://github.com/buildbot/buildbot/issues/4760
> https://github.com/buildbot/buildbot/issues/4761
> https://github.com/buildbot/buildbot/issues/4709
> https://github.com/buildbot/buildbot/issues/3471
> https://github.com/buildbot/buildbot/issues/3470
> https://github.com/buildbot/buildbot/issues/4750

Thank you.
(Nice to see that I'm not the first one requesting these :)

I assume that we'll also want to add other items for the rest of the
summer to that list?

> @Mojca Miklavec Yesterday you mentioned:
>>
>> 16:33:15  regarding the features of the default waterfall view, 
>> another problem I saw was that unless one of the builds was done recently, 
>> the builder doesn't even show in waterfall; for example, there is no texlive 
>> or pplib displayed on https://build.contextgarden.net/#/waterfall

And later Pierre said this was a feature :)

> However, I am unable to find this problem in 
> https://nine.buildbot.net/#/waterfall (it's showing the builders not having 
> recent builds). I'm not sure (it might have been configured in that manner in 
> nine.buildbot, and not present by default).
>
> I will try to replicate this issue in my local setup.

If you look at
https://build.contextgarden.net/#/console
you can see that there is a huge number of "fake commits" from one
project (from periodic scheduler) after the last build of another
project was done. Maybe there's a setting which tells you how many
latest builds / commits to fetch when drawing the waterfall view. I
didn't try to investigate what limits the number of displayed
builders.

Now: for me by far more important missing feature is in fact support
for filtering on both console and waterfall. Once filtering and
removal of old builders gets implemented, this particular issue will
be less important.

Mojca

Re: Slack-like chat (also for GSOC)

[Ruben Di Battista]

I suggest to get a look to Matrix.org. They perfectly integrate with IRC,
but they have bridges for everything. No need to self host (but you can and
it's federated with the rest of the fédération)

It's a pretty cool technology IMHO.


On Thu, 9 May 2019, 23:38 Mojca Miklavec,  wrote:

> Hi,
>
> I know that our official channels only include mailing lists, IRC and
> PR/ticket discussion so far, but I'm curious about your stand with
> respect to setting up a service like Zulip Chat / Rocket Chat / ...
> for aiding the GSOC communication when higher frequency of potentially
> simple questions would be needed. I know IRC could fulfill such a
> task, but it doesn't work too well for me. (I'm offline when at work,
> cannot really use phone efficiently for communication etc..)
>
> Would it be realistic to install such a service on breaburn if needed?
> (Or is it too complex / too much work?)
>
> I strongly suspect that one of those services also integrates pretty
> well with IRC. I've been using Slack, Zulip, Discord, Gitter at
> various occasions and I like the way they work.
>
> Thank you,
> Mojca
>

Slack-like chat (also for GSOC)

[Mojca Miklavec]

Hi,

I know that our official channels only include mailing lists, IRC and
PR/ticket discussion so far, but I'm curious about your stand with
respect to setting up a service like Zulip Chat / Rocket Chat / ...
for aiding the GSOC communication when higher frequency of potentially
simple questions would be needed. I know IRC could fulfill such a
task, but it doesn't work too well for me. (I'm offline when at work,
cannot really use phone efficiently for communication etc..)

Would it be realistic to install such a service on breaburn if needed?
(Or is it too complex / too much work?)

I strongly suspect that one of those services also integrates pretty
well with IRC. I've been using Slack, Zulip, Discord, Gitter at
various occasions and I like the way they work.

Thank you,
Mojca

Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Pierre Tardy]

> Could you add them to the project ?

Done.

Re: Please welcome the four GSOC projects this year

[Marcus Calhoun-Lopez]

Greeting, welcome, and congratulations.

Speaking only for myself, e-mail is the best way to communicate.
If the topic is of general interest or concern, a public forum might be 
appropriate (please CC me however).

This GSOC should be a great experience for all.

-Marcus

> On May 7, 2019, at 10:39 PM, Satryaji Aulia  wrote:
> 
> Hello everyone,
> 
> Thank you for selecting me for GSOC 2019, I'm very excited
> to contribute. Congratulations to the others too.
> 
> To my mentors Marcus and Clemens:
> For private communications, is e-mail the most effective way
> or do you prefer I message through something else?
> 
> I will track my deadlines and milestones through github issues
> on my macports-base fork, if that's ok with you. I'll set it up in a
> few days and ask for feedback.
> 
> Besides that, I'll use this period until May 27 to prepare for the
> coding period in general. Please let me know if you have any
> suggestions.
> 
> Thank you again, I look forward to your guidance in the future.
> 
> 
> 
> On Tue, May 7, 2019 at 5:27 AM Mojca Miklavec  wrote:
>> 
>> Dear MacPorters,
>> 
>> I'm happy to announce that MacPorts will be participating at Google's
>> GSOC with four awesome projects this year.
>> 
>> (Those who followed the heavy traffic on the development mailing list
>> are probably already aware of the excessive amount of traffic we
>> received in March and April.)
>> 
>> The selected projects are listed here:
>>https://summerofcode.withgoogle.com/organizations/4814249782673408/
>> 
>> 1.) Ahmad Satryaji Aulia: Phase Out Xcode Dependency
>> Mentored by: Marcus Calhoun-Lopez, Clemens Lang
>> A project touching the MacPorts base should remove the need to install
>> full Xcode.
>> 
>> 2.) Arjun Salyan: Web App (with API) to Collect PortIndex, Build
>> History and Installation Statistics
>> Mentored by: Mojca Miklavec, Umesh Singla
>> A standalone Django app telling you everything about individual ports.
>> 
>> 3.) Karan Sheth: Automating Packaging for Macports
>> Mentored by: Cyril Roelandt (upt author), Renee Otten
>> A python package to supersede cpan2port, pypi2port etc.
>> 
>> 4.) Rajdeep Bharati: Macports Custom Views Plugin for Buildbot
>> Mentored by: Pierre Tardy, Povilas Kanapickas (both from buildbot),
>> Mojca Miklavec
>> Upgrading our build infrastructure to the latest version of buildbot
>> with new and better views and other improvements.
>> 
>> I would like to congratulate all the students for getting selected and
>> thank all the mentors, in particular the three external ones who
>> jumped in to help us deal with the much higher amount of good
>> proposals than in previous years. We could sadly not accept all the
>> projects that we would have wanted.
>> 
>> The next three weeks are devoted to the community bonding period when
>> active participation of student is expected (with mentors' help) to
>> learn more about the tools and community, improve the planning and
>> potentially revise the goals and deadlines if needed, and get up to
>> speed for the great summer coding experience.
>> 
>> Mojca

Re: GSOC 2019 project: Improvements of Buildbot views (@ MacPorts)

[Rajdeep Bharati]

Hi,

I created some issues and found some which would be relevant for the
project (as discussed in the IRC meeting):

https://github.com/buildbot/buildbot/issues/4760
https://github.com/buildbot/buildbot/issues/4761
https://github.com/buildbot/buildbot/issues/4709
https://github.com/buildbot/buildbot/issues/3471
https://github.com/buildbot/buildbot/issues/3470
https://github.com/buildbot/buildbot/issues/4750

Could you add them to the project
 ?
I'm studying the codebase and will submit patches.

@Mojca Miklavec  Yesterday you mentioned:

> 16:33:15  regarding the features of the default waterfall view, 
> another problem I saw was that unless one of the builds was done recently, 
> the builder doesn't even show in waterfall; for example, there is no texlive 
> or pplib displayed on https://build.contextgarden.net/#/waterfall
>
> However, I am unable to find this problem in
https://nine.buildbot.net/#/waterfall (it's showing the builders not having
recent builds). I'm not sure (it might have been configured in that manner
in nine.buildbot, and not present by default).

I will try to replicate this issue in my local setup.

Thank you.
Rajdeep

On Tue, May 7, 2019 at 9:24 PM Rajdeep Bharati 
wrote:

> Yes, I have registered myself and joined the chat.
>
> Thanks.
>
> On Tue, May 7, 2019 at 9:21 PM Mojca Miklavec  wrote:
>
>> On Tue, 7 May 2019 at 15:07, Rajdeep Bharati 
>> wrote:
>> >
>> > Dear mentors,
>> >
>> > Yes, I would be available at that time. Do let me know where to join
>> you for discussion.
>> > Looking forward to making this a successful project.
>>
>> Are you familiar with IRC?
>>
>> The meeting is at freenode, channel #buildbot, in 10 minutes.
>>
>> You can join via web interface, but you might need to register first.
>> If you follow instructions on webchat you should be able to register
>> username & password with two commands (I forgot which ones).
>> https://webchat.freenode.net
>>
>> Mojca
>>
>

Re: Please welcome the four GSOC projects this year

[Satryaji Aulia]

Hello everyone,

Thank you for selecting me for GSOC 2019, I'm very excited
to contribute. Congratulations to the others too.

To my mentors Marcus and Clemens:
For private communications, is e-mail the most effective way
or do you prefer I message through something else?

I will track my deadlines and milestones through github issues
on my macports-base fork, if that's ok with you. I'll set it up in a
few days and ask for feedback.

Besides that, I'll use this period until May 27 to prepare for the
coding period in general. Please let me know if you have any
suggestions.

Thank you again, I look forward to your guidance in the future.



On Tue, May 7, 2019 at 5:27 AM Mojca Miklavec  wrote:
>
> Dear MacPorters,
>
> I'm happy to announce that MacPorts will be participating at Google's
> GSOC with four awesome projects this year.
>
> (Those who followed the heavy traffic on the development mailing list
> are probably already aware of the excessive amount of traffic we
> received in March and April.)
>
> The selected projects are listed here:
> https://summerofcode.withgoogle.com/organizations/4814249782673408/
>
> 1.) Ahmad Satryaji Aulia: Phase Out Xcode Dependency
> Mentored by: Marcus Calhoun-Lopez, Clemens Lang
> A project touching the MacPorts base should remove the need to install
> full Xcode.
>
> 2.) Arjun Salyan: Web App (with API) to Collect PortIndex, Build
> History and Installation Statistics
> Mentored by: Mojca Miklavec, Umesh Singla
> A standalone Django app telling you everything about individual ports.
>
> 3.) Karan Sheth: Automating Packaging for Macports
> Mentored by: Cyril Roelandt (upt author), Renee Otten
> A python package to supersede cpan2port, pypi2port etc.
>
> 4.) Rajdeep Bharati: Macports Custom Views Plugin for Buildbot
> Mentored by: Pierre Tardy, Povilas Kanapickas (both from buildbot),
> Mojca Miklavec
> Upgrading our build infrastructure to the latest version of buildbot
> with new and better views and other improvements.
>
> I would like to congratulate all the students for getting selected and
> thank all the mentors, in particular the three external ones who
> jumped in to help us deal with the much higher amount of good
> proposals than in previous years. We could sadly not accept all the
> projects that we would have wanted.
>
> The next three weeks are devoted to the community bonding period when
> active participation of student is expected (with mentors' help) to
> learn more about the tools and community, improve the planning and
> potentially revise the goals and deadlines if needed, and get up to
> speed for the great summer coding experience.
>
> Mojca

Re: Please welcome the four GSOC projects this year

[Rajdeep Bharati]

Dear mentors,

Thank you for selecting me and I'm really excited about the project.
I would also like to thank the MacPorts and buildbot community for
resolving my queries and helping me solve problems.
Looking forward to a wonderful summer! I will give my best.

Rajdeep

On Tue, May 7, 2019 at 3:57 AM Mojca Miklavec  wrote:

> Dear MacPorters,
>
> I'm happy to announce that MacPorts will be participating at Google's
> GSOC with four awesome projects this year.
>
> (Those who followed the heavy traffic on the development mailing list
> are probably already aware of the excessive amount of traffic we
> received in March and April.)
>
> The selected projects are listed here:
> https://summerofcode.withgoogle.com/organizations/4814249782673408/
>
> 1.) Ahmad Satryaji Aulia: Phase Out Xcode Dependency
> Mentored by: Marcus Calhoun-Lopez, Clemens Lang
> A project touching the MacPorts base should remove the need to install
> full Xcode.
>
> 2.) Arjun Salyan: Web App (with API) to Collect PortIndex, Build
> History and Installation Statistics
> Mentored by: Mojca Miklavec, Umesh Singla
> A standalone Django app telling you everything about individual ports.
>
> 3.) Karan Sheth: Automating Packaging for Macports
> Mentored by: Cyril Roelandt (upt author), Renee Otten
> A python package to supersede cpan2port, pypi2port etc.
>
> 4.) Rajdeep Bharati: Macports Custom Views Plugin for Buildbot
> Mentored by: Pierre Tardy, Povilas Kanapickas (both from buildbot),
> Mojca Miklavec
> Upgrading our build infrastructure to the latest version of buildbot
> with new and better views and other improvements.
>
> I would like to congratulate all the students for getting selected and
> thank all the mentors, in particular the three external ones who
> jumped in to help us deal with the much higher amount of good
> proposals than in previous years. We could sadly not accept all the
> projects that we would have wanted.
>
> The next three weeks are devoted to the community bonding period when
> active participation of student is expected (with mentors' help) to
> learn more about the tools and community, improve the planning and
> potentially revise the goals and deadlines if needed, and get up to
> speed for the great summer coding experience.
>
> Mojca
>

Re: Please welcome the four GSOC projects this year

[Arjun Salyan via macports-dev]

Thank You Mojca!

Congrats Satryaji, Karan and Rajdeep.

The community has shown amazing support for all of us during the
application period itself. I can't thank enough for the help I received in
tackling all the problems and doubts.

Looking forward to a really productive summer ahead while being able to
contribute something valuable.

Thank you everyone

Re: GSoC Proposal

[Karan Sheth via macports-dev]

Hey,

I really want to thank you guys for selecting me.
I am really overjoyed and excited about working with you guys!:):)

I will make sure to always put in my best efforts and to do the job
I'm assigned to the best of my abilities. I look forward to an awesome
summer, coding and getting to know the community better and to
contributing as much as possible now and also after the completion of
GSoC.

Thanks,
Karan Sheth

-- 

 <https://www.somaiya.edu>        <http://www.somaiya-ayurvihar.org>  
<http://nareshwadi.org>  <http://somaiya.com>  <http://www.helpachild.in>  
<http://nareshwadi.org>

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

>
>
>
>
> From what I understand from the stackoverflow post you're right that
> cmxpchg16b will not give a consistent view of the 16 bytes of memory
> across multiple NUMA nodes. However, maybe two 4 byte values right next
> to each other would be sufficient for your use case and could then be
> casted to a 8 byte values for CAS?
>

Thats a great idea. The offset shall never go out of range of 4 byte but
still I will put a check there for ensuring. I will update you when I am
done with testing on main base code.

Regards,
Mihir

Re: GSoC Proposal

[Mojca Miklavec]

Dear Karan,

On Fri, 3 May 2019 at 23:27, Karan Sheth wrote:
>
> If PEAR ports are useful to the community then I would like to take
> this as stretch goals / after GSoC goal.

For anyone putting the GSOC mails to "ignore mode" due to heavy
traffic ... maybe leave the comment inside
https://github.com/macports/macports-ports/pull/4192
or in the first ticket.

I have absolutely no clue what they are and whether they are
important, so someone else needs to decide.

To me personally php is kind of a dying language. Not exactly the
case, but I can imagine that any "serious developers" would take the
individual modules from whatever internal build/install system it
offers anyway as opposed to relying on MacPorts, similar to using
"pip". I didn't do the analysis on any of those ports, I would look at
the number of commits and the number of tickets filed. If there is or
was a serious demand, I would expect some activity. Admittedly Haskell
is way more obscure and much less used. But probably the only reason
why there's demand for it (from a more substantial number of people)
is Pandoc.

That said, the decision about whether or not to remove ports now is
relatively orthogonal to whether or not it makes sense to create an
import later.

Mojca

Re: GSoC Proposal

[Karan Sheth via macports-dev]

Hey,

So I came across this recent mail in mailing list[1] stating the PR
for deletion of PHP PEAR ports.
I went through a few of the ports and it seems that once the update
functionality of UPT is completed the updation of these ports could be
easily automated just by adding a PEAR frontend to upt which won't
take much time.
If PEAR ports are useful to the community then I would like to take
this as stretch goals / after GSoC goal.

Thanks,
Karan Sheth

[1] https://lists.macports.org/pipermail/macports-dev/2019-May/040664.html

-- 

 <https://www.somaiya.edu>        <http://www.somaiya-ayurvihar.org>  
<http://nareshwadi.org>  <http://somaiya.com>  <http://www.helpachild.in>  
<http://nareshwadi.org>

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi Clemens,

For making my implementation of shared memory data structure more space
efficient, I was trying to implement a stack which stores offsets to unused
locations in the shared memory file. But as stack is being shared it also
needs to be edited in a lock free way. While editing stack I need to
atomically CAS both top of stack and the element on it. For this I found
double compare and swap. Also, in my data structure at one point I need
DCAS as well to ensure correct editing.
To implement DCAS I came across some instruction “cmpxchg16”. But I think
its still not as per my need. [1]

Do you know any alternative with which I can DCAS atomically or anything
which atomically checks 2 old values before replacing value at an address?

[1]
https://stackoverflow.com/questions/7646018/sse-instructions-which-cpus-can-do-atomic-16b-memory-operations

Regards,
Mihir

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi Clemens,


> What's your current progress? Do you have some code already?
>


I made a complete offset based ctrie implementation in which any process
can insert and search on basis of a shared memory. Kindly provide me with
your views on it :)

All main code is in [1].
The header file contains all definitions.[2]

Shell script [3] runs the makefile and executes [4].

[4] is a basic test file which takes strings as arguments and all those
strings are attempted to be inserted into ctrie by different threads.

initSharedMemory() does all initialising of shared memory and mmaping and
returns a SharedMemoryManager type struct.
insert() and search() need that struct as an argument from the calling
process along with string to be inserted and permission.

I have tested it with few threads working concurrently and inserting few
strings.

This works as intended till now.
The implementation still needs to be made space efficient and needs to
increase memory size if we run out of it.



[1]
https://github.com/MihirLuthra/offsetBasedCtrie/blob/master/libpathSearch.c
[2] https://github.com/MihirLuthra/offsetBasedCtrie/blob/master/PathSearch.h
[3] https://github.com/MihirLuthra/offsetBasedCtrie/blob/master/make.sh
[4]
https://github.com/MihirLuthra/offsetBasedCtrie/blob/master/pathSearchTest2.c

Regards,
Mihir

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi,

Sorry for the late response.

What's your current progress? Do you have some code already?
>

Currently I am done with a basic Ctrie implementation as suggested in the
arXiv paper that is capable of inserting and searching paths. The
implementation is still with pointers which I will convert soon to offsets
[1]
I have only tested the working manually till now and with a single thread.
I will soon make a better test file to test the ctrie when being used by
multiple threads.

[1] https://github.com/MihirLuthra/basicPathSearch

Regards,
Mihir

Re: Any objections to using GitHub issue tracker for GSOC projects?

[Mark Anderson]

Nope. Makes sense to me.

—Mark

On Mon, Apr 22, 2019 at 11:08 PM Mark Anderson  wrote:

> Nope. Makes sense to me.
>
> —Mark
> ___
> Mark E. Anderson 
>
>
> On Mon, Apr 22, 2019 at 1:43 PM Mojca Miklavec  wrote:
>
>> Hi,
>>
>> When we switched to GitHub Ryan made a test conversion of Trac tickets
>> to GitHub issues, but there were several problems with that approach,
>> the biggest one being inability to easily filter by ports, and
>> generally the GitHub's issues were less powerful than what Trac is
>> offering, so it made sense to stick with Trac.
>>
>> The situation is slightly different with smaller standalone projects
>> where issues might in fact be a better fit and easier to use.
>>
>> For smaller standalone projects like those suggested for GSOC
>> (excluding base-related ones) I would like to experiment with the
>> built-in issue tracker, including milestones etc. Like this one, for
>> example:
>> https://github.com/macports/macports-webapp/issues
>> These would mainly be projects on which we would likely have a single
>> developer at the moment, but more contributors in the future. Are
>> there any objections?
>>
>> To make it clear: I don't want to suggest moving any of the
>> macports-base or macports-ports tickets to GitHub, at least not at
>> this point. I'm just curious whether anyone would be strongly opposed
>> to keeping some issues at a different location from trac for the sake
>> of easier GSOC administration.
>>
>> Thank you very much,
>> Mojca
>>
>

Any objections to using GitHub issue tracker for GSOC projects?

[Mojca Miklavec]

Hi,

When we switched to GitHub Ryan made a test conversion of Trac tickets
to GitHub issues, but there were several problems with that approach,
the biggest one being inability to easily filter by ports, and
generally the GitHub's issues were less powerful than what Trac is
offering, so it made sense to stick with Trac.

The situation is slightly different with smaller standalone projects
where issues might in fact be a better fit and easier to use.

For smaller standalone projects like those suggested for GSOC
(excluding base-related ones) I would like to experiment with the
built-in issue tracker, including milestones etc. Like this one, for
example:
https://github.com/macports/macports-webapp/issues
These would mainly be projects on which we would likely have a single
developer at the moment, but more contributors in the future. Are
there any objections?

To make it clear: I don't want to suggest moving any of the
macports-base or macports-ports tickets to GitHub, at least not at
this point. I'm just curious whether anyone would be strongly opposed
to keeping some issues at a different location from trac for the sake
of easier GSOC administration.

Thank you very much,
Mojca

Re: Speed up trace mode (GSoC Project)

[Clemens Lang]

Hi,

On Sun, Apr 21, 2019 at 11:01:09PM +0530, Mihir Luthra wrote:
> This is almost the same way as in the arXiv paper you shared with me.
> As we are dealing with paths, I saw implementing the above hash
> function would make the search faster as this happens quite a few
> times that files with same name have same path length from the root.
> So to get information about a file “abc.txt”, we just need to get the
> length of the path to this file from the root. Suppose the length of
> string of path from root is 16. Then the hash function takes the input
> “abc.txt” and makes it “16-abc.txt”. Now in the tree, the first level
> categorises it as per the path length, which just leaves the need to
> check the file name and in the worst case where files with same name
> have same length of path from the root, they are chained at the end of
> the tree.

Sounds like it's a good approach then.

> Although I haven’t tested the time complexities yet but even if this
> structure is not to be implemented, it won’t make much difference
> because all we would need is to put the hash function and simply
> search for the complete length instead of complete path which seems to
> be a quite easy replacement.
> I would stick with the basic structure which is less complex and
> tested and will only try to switch and test the this when all other
> things are done, that maybe a good idea.

What's your current progress? Do you have some code already?

-- 
Clemens

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi,

Thanks for the tips. ^_^
This is almost the same way as in the arXiv paper you shared with me.
As we are dealing with paths, I saw implementing the above hash function
would make the search faster as this happens quite a few times that files
with same name have same path length from the root. So to get information
about a file “abc.txt”, we just need to get the length of the path to this
file from the root. Suppose the length of string of path from root is 16.
Then the hash function takes the input “abc.txt” and makes it
“16-abc.txt”. Now in the tree, the first level categorises it as per the
path length, which just leaves the need to check the file name and in the
worst case where files with same name have same length of path from the
root, they are chained at the end of the tree.

Although I haven’t tested the time complexities yet but even if this
structure is not to be implemented, it won’t make much difference because
all we would need is to put the hash function and simply search for the
complete length instead of complete path which seems to be a quite easy
replacement.
I would stick with the basic structure which is less complex and tested and
will only try to switch and test the this when all other things are done,
that maybe a good idea.

Regards,
Mihir

Re: Speed up trace mode (GSoC Project)

[Clemens Lang]

Hi Mihir,

On Tue, Apr 16, 2019 at 11:59:53PM +0530, Mihir Luthra wrote:
> Kindly provide your suggestions for this.
> In the path search Ctrie data structure, I categorised the paths with
> the hash function working like:
> 
> If I input a path /test/files/abc.h for check
> Here we have open to abc.h, the hash function simply makes it
> “12:abc.h" where 12 is the length of path "/test/files/" to abc.h
> The first level of the categorises the paths by the length of their
> real path.
> After this the search is made as per the file name i.e. abc.h and at
> the final node key-value pair(s) are matched.
> 
> 1st level being categorised by real path length to the file reduces
> the search time is most cases.
> After that character by character, file name is checked.
> Two files with same name having same length of path from the root
> seems rare, but if any two have the same, chaining is done(which seems
> to be appropriate here because of less number of such cases).
> 
> https://drive.google.com/file/d/16HCVUpljPSz1Wn4UEx_gYQ3qDg3JyfH_/view?usp=sharing

Did you design this yourself, or did you come across a paper that
suggests doing it this way? In general, I think most times, it's wise to
rely on published research in such occasions, because peer-reviewed
papers will usually have done comparison measurements. 

-- 
Clemens

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Sure, I will try them out. Thanks

On Thu, Apr 18, 2019 at 11:29 AM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> On Tue, 16 Apr 2019 at 12:38, Rajdeep Bharati wrote:
> >
> > About the npm dependencies: Can't it be taken care of by using
> buildbot-www from pypi?
>
> While I'm not yet sure about the npm packages situation, I looked a
> bit around and bumped into
> https://www.freshports.org/devel/py-buildbot-www/
>
> You could cheat a bit (and copy stuff :) regarding dependencies etc.,
> but in any case I believe that the packaging of buildbot on FreeBSD is
> very closely resembling what we want to do in MacPorts. I believe we
> could also simply package py37-buildbot-pkg, py37-buildbot,
> py37-buildbot-www, and maybe add some more packages (whatever is
> needed). Those packages are basically pure python modules (and
> buildbot-www depends on buildbot, which is one reason more to package
> it as a python module). On top of py37-buildbot we could create a port
> simply called "buildbot" that depends on py37-buildbot and takes care
> of startupitem stuff etc.
>
> For setting up the initial version of these python packages you may
> test the new upt tool (and report any potential issues you find):
> https://github.com/macports/upt-macports/pull/1
> and feel free to ask if something is not clear with respect to packaging.
>
> Mojca
>

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

Dear Rajdeep,

On Tue, 16 Apr 2019 at 12:38, Rajdeep Bharati wrote:
>
> About the npm dependencies: Can't it be taken care of by using buildbot-www 
> from pypi?

While I'm not yet sure about the npm packages situation, I looked a
bit around and bumped into
https://www.freshports.org/devel/py-buildbot-www/

You could cheat a bit (and copy stuff :) regarding dependencies etc.,
but in any case I believe that the packaging of buildbot on FreeBSD is
very closely resembling what we want to do in MacPorts. I believe we
could also simply package py37-buildbot-pkg, py37-buildbot,
py37-buildbot-www, and maybe add some more packages (whatever is
needed). Those packages are basically pure python modules (and
buildbot-www depends on buildbot, which is one reason more to package
it as a python module). On top of py37-buildbot we could create a port
simply called "buildbot" that depends on py37-buildbot and takes care
of startupitem stuff etc.

For setting up the initial version of these python packages you may
test the new upt tool (and report any potential issues you find):
https://github.com/macports/upt-macports/pull/1
and feel free to ask if something is not clear with respect to packaging.

Mojca

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi Clemens,

Kindly provide your suggestions for this.
In the path search Ctrie data structure, I categorised the paths with the
hash function working like:

If I input a path /test/files/abc.h for check
Here we have open to abc.h, the hash function simply makes it “12:abc.h"
where 12 is the length of path "/test/files/" to abc.h
The first level of the categorises the paths by the length of their real
path.
After this the search is made as per the file name i.e. abc.h and at the
final node key-value pair(s) are matched.

1st level being categorised by real path length to the file reduces the
search time is most cases.
After that character by character, file name is checked.
Two files with same name having same length of path from the root seems
rare, but if any two have the same, chaining is done(which seems to be
appropriate here because of less number of such cases).


https://drive.google.com/file/d/16HCVUpljPSz1Wn4UEx_gYQ3qDg3JyfH_/view?usp=sharing

Regards,
Mihir

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

On Tue, 16 Apr 2019 at 12:38, Rajdeep Bharati wrote:
>
> I think one more port needs to be created: autobahn. 
> (https://docs.buildbot.net/current/manual/installation/requirements.html)

If it's not in the ports tree yet, I believe I have this port on my
computer already. (I can check when I come home and submit it to spare
you some extra work.)
The reason I didn't submit it was because each version introduced a
zillion incompatibilities with the previous version and the port where
I would have needed it no longer worked with the latest version
anyway.

(Otherwise you can always use pypi2port and put it under
python/py-autobahn. Maybe that would already work out of the box? You
can try to install and use pypi2port via macports to do the legwork
for you when creating the py-autobahn package; or potentially try the
new upt, but that one is not yet tested and not yet merged to be used
as a simple "port install". Or simply wait a bit.)

> About the npm dependencies: Can't it be taken care of by using buildbot-www 
> from pypi?

This could definitely be packaged as a separate port. I'm not sure if
this belongs under python/py-buildbot-www or simply next to the
existing buildbot port. It doesn't seem to be a regular python module,
so maybe the latter? I would need to check it to see where the
individual files should go. Can these be installed system-wide and
used for multiple masters or do they belong next to each individual
project?

That package seems to contain just static files, I don't see any npm
dependencies there?
(I don't care if we initially need to run npm manually as long as it's
documented what to do.)

(I probably need to try it out on my computer when I come home to be
able to give you a slightly more competent answer. And maybe check
what other distros are doing / how they package it.)

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

I think one more port needs to be created: autobahn
. (
https://docs.buildbot.net/current/manual/installation/requirements.html)

About the npm dependencies: Can't it be taken care of by using buildbot-www
 from pypi?

Rajdeep

On Tue, Apr 16, 2019 at 3:26 PM Rajdeep Bharati 
wrote:

> Dear Mojca,
> Root privileges worked using vim. The new ports are shown in the port
> search. Thanks a lot.
>
> Rajdeep
>
> On Tue, Apr 16, 2019 at 3:09 PM Mojca Miklavec  wrote:
>
>> Dear Rajdeep,
>>
>> I'm not sure to what extent my suggestion helps. If the file is in
>> fact locked, this won't do it, but I find it strange that this would
>> ever happen (if it happened to me, I would probably reboot the
>> machine). If using admin privileges won't help, maybe ask if someone
>> can help you on our IRC channel?
>>
>> Mojca
>>
>> On Tue, 16 Apr 2019 at 09:34, Mojca Miklavec  wrote:
>> >
>> > On Tue, 16 Apr 2019 at 09:12, Rajdeep Bharati wrote:
>> > >
>> > > Hello,
>> > > For testing a new Portfile in my local repository, the Macports guide
>> says that I need to edit the sources.conf file. However, this file is
>> locked and I'm not able to unlock it.
>> >
>> > Did you try with root privileges?
>> >
>> > I don't know what editor you used, but
>> >sudo vim /opt/local/etc/macports/sources.conf
>> > (or some other editor of your choice) should work.
>> >
>> > If I open the file with TextMate and modify & change it, I get a
>> > prompt to enter the password and then I can save the file.
>> >
>> > (The best way is probably to clone the full macports-ports repository
>> > and point sources.conf to your clone. Or at least that's what I do.
>> > You need to make sure to generate the PortIndex in any case.)
>>
>

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Dear Mojca,
Root privileges worked using vim. The new ports are shown in the port
search. Thanks a lot.

Rajdeep

On Tue, Apr 16, 2019 at 3:09 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> I'm not sure to what extent my suggestion helps. If the file is in
> fact locked, this won't do it, but I find it strange that this would
> ever happen (if it happened to me, I would probably reboot the
> machine). If using admin privileges won't help, maybe ask if someone
> can help you on our IRC channel?
>
> Mojca
>
> On Tue, 16 Apr 2019 at 09:34, Mojca Miklavec  wrote:
> >
> > On Tue, 16 Apr 2019 at 09:12, Rajdeep Bharati wrote:
> > >
> > > Hello,
> > > For testing a new Portfile in my local repository, the Macports guide
> says that I need to edit the sources.conf file. However, this file is
> locked and I'm not able to unlock it.
> >
> > Did you try with root privileges?
> >
> > I don't know what editor you used, but
> >sudo vim /opt/local/etc/macports/sources.conf
> > (or some other editor of your choice) should work.
> >
> > If I open the file with TextMate and modify & change it, I get a
> > prompt to enter the password and then I can save the file.
> >
> > (The best way is probably to clone the full macports-ports repository
> > and point sources.conf to your clone. Or at least that's what I do.
> > You need to make sure to generate the PortIndex in any case.)
>

Re: GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

On Fri, 12 Apr 2019 at 19:50, Arjun Salyan wrote:
> On Fri, Apr 12, 2019 at 3:03 AM Mojca Miklavec wrote:
>
>> One thing that "urgently" needs to be done is to obfuscate the email
>>
>> (if written at all). I'm not even sure whether we actually want the
>> email being displayed there. We need it to send automated emails from
>> the buildbot in case some failure happens, or to occasionally contact
>> the maintainer directly. But exposing that info on the website might
>> be too much.
>
> I had fixed it as soon as I saw the email, but couldn't reply that time.

Thank you.

>> I still need to check the code: what's your current strategy for
>> showing links to the tickets?
>> At some point we could differentiate different types of tickets (for
>> example mark bugs separately).
>
> Sorry to disappoint you here, but right now I am using web scrapping to do 
> this. I am looking for a plugin that could add public api feature to track 
> tickets, but maybe it doesn't exist right now. We can make our own for sure.

OK, thank you.

Maybe something like
https://trac-hacks.org/wiki/XmlRpcPlugin
could help. But I wouldn't concentrate on this right now. The existing
implementation does the job. Not in an efficient way, maybe not super
reliably, but there are other things with higher priority to do first.
This could be addressed after the other important stuff has been
implemented.

> I have added it to maintainer-detail and port-detail. But it still needs some 
> work on its position on the page.

Awesome. The search inside maintainer and category are really helpful.

At some point (but not now) we could add search through descriptions
and long descriptions as well, and to some extent group subports (some
experiments would be needed to figure out how to present that data in
a nice and useful way).

Regarding maintainers I left some notes in the PR.

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

On Tue, 16 Apr 2019 at 09:12, Rajdeep Bharati wrote:
>
> Hello,
> For testing a new Portfile in my local repository, the Macports guide says 
> that I need to edit the sources.conf file. However, this file is locked and 
> I'm not able to unlock it.

Did you try with root privileges?

I don't know what editor you used, but
   sudo vim /opt/local/etc/macports/sources.conf
(or some other editor of your choice) should work.

If I open the file with TextMate and modify & change it, I get a
prompt to enter the password and then I can save the file.

(The best way is probably to clone the full macports-ports repository
and point sources.conf to your clone. Or at least that's what I do.
You need to make sure to generate the PortIndex in any case.)

> Also, do I need to have a distfile and/or tarball to generate checksum.

The easiest way is to comment out the (existing) checksums and simply run
sudo port -v checksum 
or
sudo port -v extract 
and then you'll get the checksums printed, so that you can copy-paste
them. (If upstream publishes some checksums, it might be nice to
double-check, but I admit that I hardly ever do that myself.)

You may leave  out if you are in directory where the portfile lives.

> For the buidbot Portfile, would it be better to create a diff patch (ignoring 
> the npm dependencies ATM).

By far the easiest way to continue (get code review, feedback, request
for further changes, run automated tests etc.) is to create a pull
request to macports-ports repository. If the patch is not yet ready
and you just want the initial feedback, you can also open pull request
as a draft.

The port without npm dependencies is a good start.

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Hello,
For testing a new Portfile in my local repository, the Macports guide says
that I need to edit the sources.conf file. However, this file is locked and
I'm not able to unlock it. Could you tell me the right way to tackle this?
Also, do I need to have a distfile and/or tarball to generate checksum.
For the buidbot Portfile, would it be better to create a diff patch
(ignoring the npm dependencies ATM).

Thank you.
Rajdeep

On Thu, Apr 11, 2019 at 10:10 PM Rajdeep Bharati 
wrote:

> Thanks, I'll attempt this.
>
> Rajdeep
>
> On Thu, Apr 11, 2019 at 8:26 PM Mojca Miklavec  wrote:
>
>> Dear Rajdeep,
>>
>> On Thu, 11 Apr 2019 at 14:02, Rajdeep Bharati wrote:
>> >
>> > I had a doubt regarding the creation of buildbot 2.x Portfile: Would
>> writing a Portfile be sufficient or would there need to be other changes to
>> the ports tree as well?
>>
>> We would presumably move the contents of the existing buildbot /
>> buildbot-slave ports under buildbot-0.8 / buildbot-slave-0.8 and put
>> the new one under buildbot & buildbot-worker (the buildbot-slave could
>> potentially stay without being renamed since the name doesn't "clash",
>> or be declared obsolete and replaced_by buildbot-slave-0.8, just to be
>> consistent with buildbot-0.8 naming; not sure what is better).
>>
>> We would want to keep those two old ports for a while, at least until
>> we actually move our infrastructure to the new version, but
>> potentially also for the sake of other users who might be running the
>> old buildbot master that requires the old buildbot slaves (me
>> included; but I should do the transition in my other project as well,
>> I was just too lazy).
>>
>> But for the start feel free to ignore all those strange constraints.
>> The only thing we need to start with is a working port for the latest
>> version, and doing the last step should be trivial for any regular
>> contributor to ports with zero knowledge about buildbot.
>>
>> It could be that a few additional ports would be needed for
>> dependencies of buildbot (like some of the javascript stuff). I'm not
>> sure if those should be installed via npm directly (as part of
>> additional instructions) or if there's a feasible way to package them
>> nicely and depend on them, so that everything would simply work out of
>> the box after installing the port. Maybe step one could be to package
>> just buildbot (master and worker). Optional step 2 to package some
>> dependencies. We should check what other distributions are doing. Step
>> 3 to take care of transition of the old stuff (could be done by others
>> as well).
>>
>> Mojca
>>
>

Re: GSoC 2019 [Collect build statistics]

[Arjun Salyan via macports-dev]

Hi Mojca,

On Mon, Apr 15, 2019 at 9:46 PM Mojca Miklavec  wrote:

> Given the current state of the app with sufficient complexity, I
> believe that it would be wise to introduce some unit tests to be able
> to extensively test what happens with data you import, and to prevent
> / detect any breakages in the future.
>

Thank you. Since, I am currently working on parsing of maintainers I began
testing from maintainers only. It helped me make significant improvements
to the code which extracted the maintainers ( added to the pull request  :
https://github.com/macports-gsoc/macports-gsoc-2019-webapp/pull/1 ).
[update: this file has further changed since I updated the pull request,
logic remains the same, just the JSON object structure has changed]

I ran the tests and got desired results. I will show the final code and
results in around 24 hours after I get done with my viva voce and extra
classes, but below I am discussing the approach. Sorry, if this is not the
right way or the presentation is not fine.

I created five ports:

   1. portA maintainers {@github gmail.com:test1}
   2. portB maintainers {@github gmail.com:test2}SAME GITHUB,
   DIFFERENT EMAIL
   3. portC maintainers {@newgithub gmail.com:test2}SAME EMAIL,
   DIFFERENT GITHUB
   4. portD maintainers {gmail.com:test2}EMAIL REPEATED WITHOUT
   GITHUB
   5. portE maintainers {@github}GITHUB REPEATED WITHOUT EMAIL

I received 3 unique Github and Email pairs (according to the Logic[1] ) and
I am considering each as a different maintainer.
[
{
"github": "github",
"name": "test1",
"domain": "gmail.com"
},
{
"github": "github",
"name": "test2",
"domain": "gmail.com"
},
{
"name": "test2",
"domain": "gmail.com",
"github": "newgithub"
}
]

Now to each maintainer I added all those ports which had GitHub or Email or
both same as that of the unique maintainer.

[
{
"model": "ports.Maintainer",
"pk": 0,
"fields": {
"github": "github",
"name": "test1",
"domain": "gmail.com",
"ports": [
[
"portA",
"portB",
"portD"
]
}
},
{
"model": "ports.Maintainer",
"pk": 1,
"fields": {
"github": "github",
"name": "test2",
"domain": "gmail.com",
"ports": [
[
"portA",
"portB",
"portD",
"portC"
"portE"
]
}
},
{
"model": "ports.Maintainer",
"pk": 2,
"fields": {
"name": "test2",
"domain": "gmail.com",
"github": "newgithub",
"ports": [
[
"portE",
"portB",
"portC"
   ]
}
}
]


 For querying we can now use email/ GitHub and show all the ports for all
the maintainers received.

This should not break because of any inconsistency in the maintainer
details. But there is one disadvantage- On the port-detail page, we will
now show x maintainers, if the same maintainer provided x different pairs
of GitHub and email. However this disadvantage might prove to be helpful in
getting rid of the inconsistencies.

Thank You

[1]
Currently I am using the following Logic for adding maintainers (comparing
with already parsed maintainers) :

   - If neither the email nor GitHub is repeated: CREATE NEW
   - If the email and GitHub both are repeated: SKIP
   - If the email is repeated and not the GitHub handle (provided) : CREATE
   NEW with inconsistency flag
   - If the GitHub handle is repeated and not the email address (provided)
   : CREATE NEW with inconsistency flag
   - If the Github handle is repeated and email is not provided: SKIP
   - If the email address is repeated and GitHub is not provided: SKIP

Re: GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

Dear Arjun,

I'll answer individual questions & provide further suggestion separately.

Given the current state of the app with sufficient complexity, I
believe that it would be wise to introduce some unit tests to be able
to extensively test what happens with data you import, and to prevent
/ detect any breakages in the future.

Some example of a unit test: prepare a small number (for example four)
super simple ports (just enough to make them valid, they could be
simply called portA, portB, portC, portD), add various made-up
maintainers; you may use one github handle with different or missing
email addresses in different ports, or one email address with various
or missing github handles ... Then run portindex, portindex2json,
store the json file, let the unit test import the data into an empty
database, and verify that the database contains exactly the entries
you wanted there. (OK, for the first unit test just one port with
simple valid data. Then something more complicated.)

You could also try to check if you can manage to "dockerize" the
solution (lower priority).

Mojca

Re: Speed up trace mode (GSoC Project)

[Clemens Lang]

Hi,

On Fri, Apr 12, 2019 at 11:16:46PM +0530, Mihir Luthra wrote:
> I was constructing the trie data structure which has to be operated
> from the address space mapped by mmap(2). In place of “array of next
> nodes”, I am using “array of offsets”.

Sounds correct.

> the void * returned from calling mmap(2), I would type cast it as my
> struct type.

Sounds correct as well.

> When needing to write data, go to the last offset *(base + offset) and
> CAS.
> 
> I have doubts regarding “type” of offset, simply off_t or int. As here
> we are mapping file memory into process address space, I m not certain
> what “type” would be safe to use for offsets.

off_t would be a good choice, as would be size_t (although I think
they're probably aliases anyway).

-- 
Clemens

Re: GSoC 2019 [Collect build statistics]

[Arjun Salyan via macports-dev]

Hi Mojca,

On Fri, Apr 12, 2019 at 3:03 AM Mojca Miklavec  wrote:

> Awesome!


Thank You.

One thing that "urgently" needs to be done is to obfuscate the email
>
(if written at all). I'm not even sure whether we actually want the
> email being displayed there. We need it to send automated emails from
> the buildbot in case some failure happens, or to occasionally contact
> the maintainer directly. But exposing that info on the website might
> be too much.
>

I had fixed it as soon as I saw the email, but couldn't reply that time.

As far as accessing the information is concerned: at the moment you
> use email as unique identifier. I would probably use github handle by
> default / as the main entry point. I think that '@' is an allowed
> character in URL. If so, we could use
> /maintainer/@ryandesign
> to access the same page


There was a big error in my parsing script due to which the GitHub handles
of a lot of maintainers were not being parsed. And hence, I went with
email. But still, some 300 maintainers have not provided GitHub handles.


> Alternatively we could allow macports handles (for those with
> @macports.org email addresses), so
> /maintainer/ryandesign
> could work just as well.
>

Yes, this would be good.

For maintainers with a super long list of ports, or, for
> non-maintained ports in particular, we might eventually need a way to
> shorten that list (have multiple pages) or provide a similar search
> functionality as for global ports, except that here it would be
> limited to the ports by that particular maintainer.

I would put that list of ports in a table and add version & short
> description.
>

Yes, I will add pagination and show more details for both list of ports on
maintainer's page and on category page. The filter would also be amazing. I
will do this.


> I still need to check the code: what's your current strategy for
> showing links to the tickets?
> At some point we could differentiate different types of tickets (for
> example mark bugs separately).
>

Sorry to disappoint you here, but right now I am using web scrapping to do
this. I am looking for a plugin that could add public api feature to track
tickets, but maybe it doesn't exist right now. We can make our own for sure.


> One minor suggestion. I really like the "search for port" field. Could
> this be added to every page? There is "ports" in the top right corner,
> but that one is a lot less useful in itself (not saying that it should
> go, just suggesting search on each page).
>

I have added it to maintainer-detail and port-detail. But it still needs
some work on its position on the page.


> Here's what I would do, but feel free to propose an alternative and/or
> discuss further. (Actually, I have two slightly different ideas for
> implementation in my mind, I'll describe one of them first.)
>
> For the maintainers you could declare a unique keyword over the
> combination of github handle + email. Every maintainer of every port
> has a uniquely specified pair (github + email) when you import it to
> the database (neither github handles not emails would be unique on its
> own). Note that you still have index specified on both columns
> (separately on each, but the uniqueness is only defined on combination
> of the two).
>
> When you read the port, you check the pair (@github, email). If the
> pair already exists in the database, you enter the (port, author) pair
> into the database of maintainerships. If it doesn't exist, you create
> it first, and then assign the maintainership to the port. (Note that
> whenever you are updating the port, you also need to check if you need
> to remove some maintainers from that port.)
>
> When you display a particular maintainer, say @somerandomgithubhandle,
> you run a query and if you hit more that one entry with that github
> handle in the database:
>

This would solve the problem of multiple emails with same GitHub handle.
But there are cases when the maintainer has provided 'GitHub and email'
both for one port and just 'email' for other port. Sorry If I am missing
something.

Example:
for libsmf {ryandesign openmaintainer}
for penal-soft {{ryandesign @ryandesign} openmaintainer}

And while many haven't provided GitHub handles, some haven't provided
emails.

You could also have a separate page which runs different queries and
> looks for all maintainers with inconsistencies (that's for later). It
> would generally be helpful to have a collection of such pages for
> different things, like: all broken builds on buildbot, all outdated
> ports, ...
>

This for sure would be very good to add into the app once everything is
ready.

Thank You

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi,

I needed some advise regarding ctrie implementation.

I was constructing the trie data structure which has to be operated from
the address space mapped by mmap(2).
In place of “array of next nodes”, I am using “array of offsets”.
the void * returned from calling mmap(2), I would type cast it as my struct
type.
When needing to write data, go to the last offset *(base + offset) and CAS.

I have doubts regarding “type” of offset, simply off_t or int.
As here we are mapping file memory into process address space, I m not
certain what “type” would be safe to use for offsets.


Regards,
Mihir

Re: GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

Dear Arjun,

Thanks a lot for the update.

On Thu, 11 Apr 2019 at 16:15, Arjun Salyan wrote:
>
> I have added maintainer views and tables to the demo app.
>
> List of maintainers is clickable on the port-detail page.
> A maintainer-detail view that display info and list of maintained ports

Awesome!

I just noticed that short before you wrote the email as I tried to
find some info on the page :)

> Examples:
> maintainer-detail: 
> https://frozen-falls-98471.herokuapp.com/maintainer/ryandesign__macports.org/

Thank you.

One thing that "urgently" needs to be done is to obfuscate the email
(if written at all). I'm not even sure whether we actually want the
email being displayed there. We need it to send automated emails from
the buildbot in case some failure happens, or to occasionally contact
the maintainer directly. But exposing that info on the website might
be too much.

As far as accessing the information is concerned: at the moment you
use email as unique identifier. I would probably use github handle by
default / as the main entry point. I think that '@' is an allowed
character in URL. If so, we could use
/maintainer/@ryandesign
to access the same page.

Alternatively we could allow macports handles (for those with
@macports.org email addresses), so
/maintainer/ryandesign
could work just as well.

Email access could of course be left as it is, but I would not make it default.

I would show the github handle prefixed with the '@' character.

For maintainers with a super long list of ports, or, for
non-maintained ports in particular, we might eventually need a way to
shorten that list (have multiple pages) or provide a similar search
functionality as for global ports, except that here it would be
limited to the ports by that particular maintainer.

I would put that list of ports in a table and add version & short description.

Nice, in any case.

> port-detail: https://frozen-falls-98471.herokuapp.com/ports/faust/

Thanks a lot for adding the links to Trac tickets.

I still need to check the code: what's your current strategy for
showing links to the tickets?
At some point we could differentiate different types of tickets (for
example mark bugs separately).


One minor suggestion. I really like the "search for port" field. Could
this be added to every page? There is "ports" in the top right corner,
but that one is a lot less useful in itself (not saying that it should
go, just suggesting search on each page).

> But, while extracting 'maintainers' from the portindex, maintaining 
> uniqueness was very difficult.

This was totally expected :)

> There are a lot of inconsistencies
> - Same maintainer has provided GitHub details for one port and not for the 
> other.
> - Same maintainer has provided different email for different ports.
>
> I understand that it should be web-app's job to detect this and for now the 
> problem is mostly solved. But in future, one odd case and things can break. 
> What best can be done about this?

Here's what I would do, but feel free to propose an alternative and/or
discuss further. (Actually, I have two slightly different ideas for
implementation in my mind, I'll describe one of them first.)

For the maintainers you could declare a unique keyword over the
combination of github handle + email. Every maintainer of every port
has a uniquely specified pair (github + email) when you import it to
the database (neither github handles not emails would be unique on its
own). Note that you still have index specified on both columns
(separately on each, but the uniqueness is only defined on combination
of the two).

When you read the port, you check the pair (@github, email). If the
pair already exists in the database, you enter the (port, author) pair
into the database of maintainerships. If it doesn't exist, you create
it first, and then assign the maintainership to the port. (Note that
whenever you are updating the port, you also need to check if you need
to remove some maintainers from that port.)

When you display a particular maintainer, say @somerandomgithubhandle,
you run a query and if you hit more that one entry with that github
handle in the database:
- simply display ports from all the entries with that github handle
(you can potentially sort them according to email, or lack thereof, so
that it's easier to figure out which ports need a fix)
- display a red warning saying that there are inconsistencies /
problems with that maintainer which need to be fixed

You could also have a separate page which runs different queries and
looks for all maintainers with inconsistencies (that's for later). It
would generally be helpful to have a collection of such pages for
different things, like: all broken builds on buildbot, all outdated
ports, ...

The other idea would be to keep just one entry per "maintainer", but
then add an additional column with a flag saying that there's
something wrong with that entry. When you would do daily or weekly
updates of the full database, 

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Thanks, I'll attempt this.

Rajdeep

On Thu, Apr 11, 2019 at 8:26 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> On Thu, 11 Apr 2019 at 14:02, Rajdeep Bharati wrote:
> >
> > I had a doubt regarding the creation of buildbot 2.x Portfile: Would
> writing a Portfile be sufficient or would there need to be other changes to
> the ports tree as well?
>
> We would presumably move the contents of the existing buildbot /
> buildbot-slave ports under buildbot-0.8 / buildbot-slave-0.8 and put
> the new one under buildbot & buildbot-worker (the buildbot-slave could
> potentially stay without being renamed since the name doesn't "clash",
> or be declared obsolete and replaced_by buildbot-slave-0.8, just to be
> consistent with buildbot-0.8 naming; not sure what is better).
>
> We would want to keep those two old ports for a while, at least until
> we actually move our infrastructure to the new version, but
> potentially also for the sake of other users who might be running the
> old buildbot master that requires the old buildbot slaves (me
> included; but I should do the transition in my other project as well,
> I was just too lazy).
>
> But for the start feel free to ignore all those strange constraints.
> The only thing we need to start with is a working port for the latest
> version, and doing the last step should be trivial for any regular
> contributor to ports with zero knowledge about buildbot.
>
> It could be that a few additional ports would be needed for
> dependencies of buildbot (like some of the javascript stuff). I'm not
> sure if those should be installed via npm directly (as part of
> additional instructions) or if there's a feasible way to package them
> nicely and depend on them, so that everything would simply work out of
> the box after installing the port. Maybe step one could be to package
> just buildbot (master and worker). Optional step 2 to package some
> dependencies. We should check what other distributions are doing. Step
> 3 to take care of transition of the old stuff (could be done by others
> as well).
>
> Mojca
>

Re: GSoC 2019 [Collect build statistics]

[Arjun Salyan via macports-dev]

Hi,

I have added maintainer views and tables to the demo app.

   - List of maintainers is clickable on the port-detail page.
   - A maintainer-detail view that display info and list of maintained ports

Examples:
maintainer-detail:
https://frozen-falls-98471.herokuapp.com/maintainer/ryandesign__macports.org/

port-detail: https://frozen-falls-98471.herokuapp.com/ports/faust/


But, while extracting 'maintainers' from the portindex, maintaining
uniqueness was very difficult. There are a lot of inconsistencies-
- Same maintainer has provided GitHub details for one port and not for the
other.
- Same maintainer has provided different email for different ports.

I understand that it should be web-app's job to detect this and for now the
problem is mostly solved. But in future, one odd case and things can break.
What best can be done about this?

On Tue, Apr 9, 2019 at 3:21 AM Mojca Miklavec  wrote:

> A general suggestion from me would be to study in depth some good and
> exhaustive book on relational database design to fill in the holes.
> (There might also be some online courses.)
>

Thanks Mojca. I did some research for a detailed book and I found "An
Introduction to Database Systems" by C.J. Date (I also found it in the
library). I will let you know how it goes.

Thank You

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

I had a doubt regarding the creation of buildbot 2.x Portfile: Would
writing a Portfile be sufficient or would there need to be other changes to
the ports tree as well?

Rajdeep

On Tue, Apr 9, 2019 at 9:24 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> On Tue, 9 Apr 2019 at 17:27, Rajdeep Bharati wrote:
> >
> > Here's a poc of buildbot workers running on mac VM using libvirt
> (thought I'm not sure whether this is the kind of thing we need):
> https://github.com/nextgis/buildbot
>
> Cool, nice find! I don't know how hard it would be to replicate their
> work and adapt it to our needs (even with these I expect quite some
> work to set everything up), but it's definitely helpful to have a nice
> starting point. Yes, I believe that's a useful link in any case.
>
> > I have submitted a final proposal with some more changes. I'm sorry for
> being late (since there are just a couple of hours left for the deadline),
> it would be great if you could review it once.
>
> We don't see the final proposals yet, only drafts, but I already left
> the comments I had earlier and don't have much to add
> (for example: we don't want students to burn out by trying to work 60
> hours per week, it's neither realistic nor productive).
>
> Mojca
>

Re: Feedback Request for final GSoC Proposal [Speed Up Trace Mode]

[Mihir Luthra]

Hi,

Sorry for re-mailing. I know it's almost the last moment.
But if possible, please provide any possible feedback.

https://docs.google.com/document/d/14eSXwZ6N1vRcBudaJWlwO5G17dY5B1nHgfhh52tSnv0/edit#

Regards,
Mihir

Re: Wish to contribute to MacPorts as part of GSoC

[Mojca Miklavec]

Dear Shreyas,

Your references definitely sound compelling and I firmly believe that
you could make a wonderful project, but the two hours left before the
deadline are not anywhere near enough to prepare a solid proposal.
Proposal is not just about writing two paragraphs about yourself, you
need to prove that you understand very precisely what you are
undertaking, and prepare a solid plan of how to execute that.

As Umesh already replied, we hope that we will have another chance for
GSoC next year, but even outside of GSoC there are plenty
opportunities to help us build a better product and better community.
We are more than willing to help you learn and contribute to
OpenSource if you would like to proceed. Except for a small number of
students who occasionally get a stipend, the rest of us are all
volunteers who work on the project in the spare time, and at least to
me Open Source allowed getting sufficient experience in coding that I
could apply for a software engineer without having any formal
education in that field.

Regarding your enquiry about build statistics idea. We had one
proposal for that idea already. If that proposal gets selected or if
the student decides to continue despite not being funded, you are more
than welcome to help developing that project. The project will
eventually require a lot more workforce than one student can possibly
do in the course of three months, so any help is warmly appreciated.
We all benefit by learning from each other and helping each other.

The same applies to any other students who might not be the lucky
selected ones, or might have found out about GSOC too late. Also note
that those who start contributing early have a considerable time
advantage and thus bigger chances to get selected next year. (One of
our best students had a few hundred commits before starting GSoC.)

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

Dear Rajdeep,

On Tue, 9 Apr 2019 at 17:27, Rajdeep Bharati wrote:
>
> Here's a poc of buildbot workers running on mac VM using libvirt (thought I'm 
> not sure whether this is the kind of thing we need): 
> https://github.com/nextgis/buildbot

Cool, nice find! I don't know how hard it would be to replicate their
work and adapt it to our needs (even with these I expect quite some
work to set everything up), but it's definitely helpful to have a nice
starting point. Yes, I believe that's a useful link in any case.

> I have submitted a final proposal with some more changes. I'm sorry for being 
> late (since there are just a couple of hours left for the deadline), it would 
> be great if you could review it once.

We don't see the final proposals yet, only drafts, but I already left
the comments I had earlier and don't have much to add
(for example: we don't want students to burn out by trying to work 60
hours per week, it's neither realistic nor productive).

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Here's a poc of buildbot workers running on mac VM using libvirt (thought
I'm not sure whether this is the kind of thing we need):
https://github.com/nextgis/buildbot

I have submitted a final proposal with some more changes. I'm sorry for
being late (since there are just a couple of hours left for the deadline),
it would be great if you could review it once.

Thank you for providing valuable guidance!

On Tue, Apr 9, 2019 at 5:36 PM Rajdeep Bharati 
wrote:

> Ok, thanks. One more thing: You were talking about some issues with the
> Collection class in buildbot data
> which is not well compatible with Vue. Could you specify the kind of
> issues?
>
> Thanks and regards
> Rajdeep
>
> On Tue, Apr 9, 2019 at 1:44 AM Pierre Tardy  wrote:
>
>> A javascript library which you can simply call from your main.js and
>> which will implement all the boilerplate needed to connect your vue
>> component to the angularjs application
>>
>> Regards
>> Pierre
>>
>> Le lun. 8 avr. 2019 à 21:23, Rajdeep Bharati
>>  a écrit :
>> >
>> > Hi Pierre,
>> >
>> > By library do you mean a library of components and JSON interfaces?
>> >
>> > Regards
>> > Rajdeep
>> >
>> > On Tue, Apr 9, 2019 at 12:18 AM Pierre Tardy  wrote:
>> >>
>> >> Hi Rajdeep,
>> >>
>> >> We already have a yeoman generator:
>> >>
>> https://github.com/buildbot/guanlecoja/tree/master/generator-guanlecoja
>> >>
>> >> I think the current boilerplate has more than just boilerplate.
>> >> It also has some code that could be factorized.
>> >> It is better not to put too much in a generator, as it is hard to
>> >> upgrade after a first instanciation
>> >> So I would recommend a generator plus a library.
>> >>
>> >> Regards
>> >> Pierre
>> >> Pierre
>> >>
>> >> Le lun. 8 avr. 2019 à 18:44, Rajdeep Bharati
>> >>  a écrit :
>> >> >
>> >> > @Mojca Miklavec Oh, I see.
>> >> >
>> >> > @Pierre Tardy Regarding the npm package: I think it would be better
>> to create a project generator CLI (which would create the boilerplate
>> project, similar to vue-cli or create-react-app). Do you think that would
>> work? Can I use yeoman to do this?
>> >> >
>> >> > Thank you for help.
>> >> > Rajdeep
>> >> >
>> >> > On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec 
>> wrote:
>> >> >>
>> >> >> Dear Rajdeep,
>> >> >>
>> >> >> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
>> >> >> >
>> >> >> > I found something which might help us: https://bors.tech/homu-io/
>> https://github.com/barosl/homu.
>> >> >> > I will try it set it up on my local system and get back to you.
>> >> >> > Mojca
>> >> >>
>> >> >> This sounds nice, but I wonder if it would actually address any of
>> our
>> >> >> problems (I suspect not). Also, it explicitly says that the project
>> >> >> was kind of abandoned and continued elsewhere?
>> >> >>
>> >> >> What this tool is trying to solve is to check the code again after
>> >> >> potentially long code review process, and run CI again "just before
>> >> >> merging". However some of our builds would take hours, or, well, on
>> >> >> Travis you might get 2 hours of waiting time and then one hour of
>> >> >> building, just to experience a timeout at the end anyway. This makes
>> >> >> the "just before merging" kind of not-too-well-defined. Plus, half
>> of
>> >> >> our Travis failures are false negatives, and before this can be
>> useful
>> >> >> on the buildbot, we would need to be able to enable checking pull
>> >> >> requests in a "safe way" (addendum: for such checking we could
>> >> >> probably just as well engage a couple of machines which are
>> completely
>> >> >> decoupled from our main builders, which would run, say, just 10.5,
>> >> >> 10.6, 10.9, and would not upload the results anywhere). When
>> checking
>> >> >> the base ... we have so little activity that this is practically
>> never
>> >> >> an issue. And with ports there are so many distinct ports, that this
>> >> >> is also hardly ever an issue; and when it is, you would already
>> notice
>> >> >> that the PR cannot be merged.
>> >> >>
>> >> >> Mojca
>>
>

Fwd: Wish to contribute to MacPorts as part of GSoC

[Umesh Singla]

Hi Shreyas,

While I believe you have a good set of skills, it may be a bit too late if
you want to participate in GSoC with us. But we definitely welcome all
kinds of contribution outside GSoC and the program is going to be there
next year as well. Contributors are definitely given higher preference.

Our criteria for selection involves a strong proposal and a demo of the
student's skill set by either raising a PR to the existing code base or PoC
implementations of any idea you are proposing. This is for the mentors to
be confident about a student being productive as soon as the coding period
starts.

Nonetheless, I'd advise you to go through the March and April archives of
this mailing list [1], as there's an abundance of information available on
almost all the "hot" projects and problems MacPorts is facing.

To answer your questions, I don't think we have any idea "similar" to
collect-build-statistics but if you meant similar to a web application,
others on the list are more suitable to answer. I am not sure what you
meant by improvements to the previous collect-build-statistics project
either. We do not have any implementation until now.

(Also, please keep your emails on the mailing list to get responses quicker
and from the wider community. Subscribe if you haven't already.)

[1]: https://lists.macports.org/pipermail/macports-dev/

- Umesh

-- Forwarded message -
From: Shreyas H N 
Date: Tue, Apr 9, 2019 at 4:37 PM
Subject: Wish to contribute to MacPorts as part of GSoC
To: 


Hello Umesh,

I am Shreyas H N, a PG1 student at IIIT Hyderabad. I found your contact on
the MacPorts GSoC page. "Collect build statistics" project caught my eye
and I believe I am competent enough to develop significant stuff in this
domain.

A short introduction about me:
I completed my Btech at M S Ramaiah University, Bangalore. I used
throughout Linux my undergrad. Just before I joined masters at IIITH, I
switched to Mac. I am a strong believe in the Growth "Mindset".
A few projects I am proud of:
1. MapRedue framework implementation from scratch (C++). WordCount and
InvertedPage index.
2. Machine Learning intern at IISc SERC in 2017 summer.
3. "mini BitTorrent"
4. DropBox clone



 Web development experience
1. Built "DropBox clone" web application using Flask, Bootstrap.
2. Task Management web application using ASP.NET as part of my internship
at MOOG in 2016
3. Developed a responsive website for my father's firm in 2015 (
www.nagarajassociates.in). This was my very first web development project

Other info:
1. Have been using git from the past year (Projects: Mapreduce,
Bittorrrent, Terminal file explorer, Dropbox and all assignments.
https://github.com/hiteshkaushik28/MapReduce_Project )
2. I was a Machine Learning intern at IISc in 2017.
3. I am a newbie to the OpenSource world but I have already experienced
that it is filled with passionate and humble people who love collaborating.

Work I have done so far:
1. Saw the code base introduction video on youtube (
https://www.youtube.com/watch?v=46qshiDskrM)
2. Went through certain portions of the guide.
3. Read the project ideas and chose "Build Statistics"

Currently I am brain storming how I can modify/improve this "Collect build
statistics" idea to prepare my project proposal.

My questions to you:
1. If you have any idea similar to "Collect build statistics", do suggest.
2. Do you think it is a good idea for me to propose only improvements to
the previous "Collect build statistics" GSoC project?

Re: GSoC 2019 [Phase out dependency on Xcode]

[Satryaji Aulia]

Hi Mojca, Clemens, and everyone.

Thank you for the feedback and comments, I've incorporated
the suggestions and submitted my final proposal.

On Tue, Apr 9, 2019 at 3:50 AM Clemens Lang  wrote:
> I've left a couple of comments in your Google Doc, but overall yours is
> a solid proposal. Sorry for only picking up on your thread now, but
> we've been somewhat overwhelmed by GSoC proposals this year.

I understand that you're being overloaded right now. I've read
up on last year's mailing list about GSOC and it looks like
you're receiving more than 5x (?) of last year's traffic.

> don't have time to review the PR right now, but please feel free to keep
> pestering me about it weekly or biweekly until I get to it.

Will do.

On Mon, Apr 8, 2019 at 6:50 PM Mojca Miklavec  wrote:
> Amazing new feature, thank you!

You're welcome, I'm glad to help.

> Cool, thank you very much.
> I then realised that it wasn't asciidoctor we were missing as a
> package, but docbookrx (which we will need for conversion of our guide
> which is currently in docbook).
> Sorry for misguidance, but thank you for the update in any case.

I see now, I thought that I had misread your email haha.

Good luck for everyone involved.

GSoC 2019 submission deadline is today

[Umesh Singla]

Hello,

*I apologize for another email which may not be of interest to many of the
members here but till we figure out a new channel for GSoC (and more such)
activities, I believe there is no other option than using -dev.*

With only a few hours left in final submissions, I advise the students to
submit a final PDF version of their proposal as soon as possible and not
leave it to the last minute. Draft submissions do not count [1]. While
proposals are given the most weight in the decision for selection, there
are a lot of other factors which you may have noticed already, from the
replies to your queries.

To potential mentors: I'd suggest clicking on the "Want To Mentor" button
on the summer-of-code dashboard [2] against the project/proposal you'd like
to mentor if you haven't already. I am saying this because the number of
"want-to-mentor" might be one of the criteria Google considers while
deciding the number of slots to be allotted to an organization. It'd help
in assigning the mentors to students as well.

- Umesh

[1]:
https://google.github.io/gsocguides/student/writing-a-proposal#submitting-a-final-pdf-proposal
[2]: https://summerofcode.withgoogle.com/dashboard

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Ok, thanks. One more thing: You were talking about some issues with the
Collection class in buildbot data
which is not well compatible with Vue. Could you specify the kind of issues?

Thanks and regards
Rajdeep

On Tue, Apr 9, 2019 at 1:44 AM Pierre Tardy  wrote:

> A javascript library which you can simply call from your main.js and
> which will implement all the boilerplate needed to connect your vue
> component to the angularjs application
>
> Regards
> Pierre
>
> Le lun. 8 avr. 2019 à 21:23, Rajdeep Bharati
>  a écrit :
> >
> > Hi Pierre,
> >
> > By library do you mean a library of components and JSON interfaces?
> >
> > Regards
> > Rajdeep
> >
> > On Tue, Apr 9, 2019 at 12:18 AM Pierre Tardy  wrote:
> >>
> >> Hi Rajdeep,
> >>
> >> We already have a yeoman generator:
> >> https://github.com/buildbot/guanlecoja/tree/master/generator-guanlecoja
> >>
> >> I think the current boilerplate has more than just boilerplate.
> >> It also has some code that could be factorized.
> >> It is better not to put too much in a generator, as it is hard to
> >> upgrade after a first instanciation
> >> So I would recommend a generator plus a library.
> >>
> >> Regards
> >> Pierre
> >> Pierre
> >>
> >> Le lun. 8 avr. 2019 à 18:44, Rajdeep Bharati
> >>  a écrit :
> >> >
> >> > @Mojca Miklavec Oh, I see.
> >> >
> >> > @Pierre Tardy Regarding the npm package: I think it would be better
> to create a project generator CLI (which would create the boilerplate
> project, similar to vue-cli or create-react-app). Do you think that would
> work? Can I use yeoman to do this?
> >> >
> >> > Thank you for help.
> >> > Rajdeep
> >> >
> >> > On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec 
> wrote:
> >> >>
> >> >> Dear Rajdeep,
> >> >>
> >> >> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
> >> >> >
> >> >> > I found something which might help us: https://bors.tech/homu-io/
> https://github.com/barosl/homu.
> >> >> > I will try it set it up on my local system and get back to you.
> >> >> > Mojca
> >> >>
> >> >> This sounds nice, but I wonder if it would actually address any of
> our
> >> >> problems (I suspect not). Also, it explicitly says that the project
> >> >> was kind of abandoned and continued elsewhere?
> >> >>
> >> >> What this tool is trying to solve is to check the code again after
> >> >> potentially long code review process, and run CI again "just before
> >> >> merging". However some of our builds would take hours, or, well, on
> >> >> Travis you might get 2 hours of waiting time and then one hour of
> >> >> building, just to experience a timeout at the end anyway. This makes
> >> >> the "just before merging" kind of not-too-well-defined. Plus, half of
> >> >> our Travis failures are false negatives, and before this can be
> useful
> >> >> on the buildbot, we would need to be able to enable checking pull
> >> >> requests in a "safe way" (addendum: for such checking we could
> >> >> probably just as well engage a couple of machines which are
> completely
> >> >> decoupled from our main builders, which would run, say, just 10.5,
> >> >> 10.6, 10.9, and would not upload the results anywhere). When checking
> >> >> the base ... we have so little activity that this is practically
> never
> >> >> an issue. And with ports there are so many distinct ports, that this
> >> >> is also hardly ever an issue; and when it is, you would already
> notice
> >> >> that the PR cannot be merged.
> >> >>
> >> >> Mojca
>

Re: GSoC 2019 - trace mode improvements

[Davide Gerhard via macports-dev]

On Sunday, 07/04/2019 15:59 CEST, Mihir Luthra <1999mihir.lut...@gmail.com> 
wrote:

> https://docs.google.com/document/d/15cVbH6f6hBr9HryJEHUZEbRsToN1BAjY8My-oRstO9A/edit#heading=h.ng3lyvrem2d1
>
> This doc is really clumsy tbh, and more like just rough notes made by me.
> Though the links to learning resources attached in the document under the
> heading
> “surfing through the code flow”
>  are quite useful. That’s all needed to understand the code and
> And this video by Clemens [ https://www.youtube.com/watch?v=46qshiDskrM ]
> would be useful for understanding how the trace mode code gets activated
> like when and where.
>
> With 2 days left, maybe you can give this day to learning from those links
> and understanding the code and next day on project proposal.
>
> Worth a try!
> We may work together helping each other. I on speed up trace mode (well if
> I get selected) & you on auto detection of build dependencies and as you
> have contributed previously, you stand a better chance compared to me on
> getting your proposal accepted I guess.

Thank you for the links and to be so kind!
Anyway, during the weekend I had personal impediments and I could not do
more.

good luck!
/davide

Re: GSoC 2019 - trace mode improvements

[Davide Gerhard via macports-dev]

On Sunday, 07/04/2019 15:15 CEST, Mojca Miklavec  wrote:

>> > Maybe we should be doing a better job in marketing :(
>
> My question is: what would we need to have done in order for the news
> to have reached you one month (or more) ago?

It is an hard question; the first thing that arise in my mind and that can
catch a lot of macports's users attention is to introduce

$ port news

with topics/priority support, like in the venerable gentoo portage [1].

A simple news could be suffice in the case of GSoC but should be
expected that there is a way to alert the user for urgent news;
think about security issues for example.

As an example of usage:

- the user sync macports-ports with

$ port sync

- at the end the system checks if there is new news and show the
  following message:

There is N news unread. 1 urgent.

In my opinion the urgent one can be shown directly at this stage.

Obviously, the user should change the status like, read and unread.

Keep in mind that I don't have Facebook and I use generally rss/ml.
Moreover, I don't see any post on macports-announce@ and macports-users@
about GSoC.

/davide

[1] https://www.gentoo.org/glep/glep-0042.html

Re: GSoC 2019 - Feedback request

[AmirAli Mashayekhi via macports-dev]

Dear Mojca,



Thank you for merging the PR.



And Thank you for giving me some ideas to study on and decide what I would like 
to work on as this GSoC.



Here is my updated proposal : 
https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing



Can't wait to work with you folks :)



Best Regards,


On Apr 07, 2019, at 10:54 PM, Mojca Miklavec  wrote:


Dear Amir,

On Sun, 7 Apr 2019 at 18:21, AmirAli Mashayekhi wrote:



Dear Mojca (and everybody else in the mail list),


Thank you for your response, Sorry for the misundrestanding of the proposal.


As requested I also tried to expose myself to the whole workflow of Macports 
developement cycle.
Here is a link for my pull request : 
https://github.com/macports/macports-ports/pull/4039


Thank you.
Just please change the commit message, force push to the branch and
the it's good to go (the Travis failures are unrelated and Azure is
happy, so that's fine).


After spending the day to check the ports in the tree (this way I also found 
mlterm to update as I mentioned above), honestly I think I need a little bit of 
hint from the community to see and select which port right now fits my 
abilities. nevertheless I have updated my proposal's general structure, I hope 
it makes more sense for you.
The only missing part atm is to fill the port name I will work on...

You may check https://trac.macports.org/wiki/Tickets and see which
ports have more bugs, update requests etc. filed. Esp. those that have
been waiting for years.

Candidates for GSOC could be:
- KDE
- QT5 (it works, but there is a lot of work to be done with respect to
older macOS version)
- MySQL (not enough for the summer, but it needs a maintainer), Pandoc, ...
- full ruby gems spectrum, full haskell / cabal spectrum, lisp, ...
- ... plenty of others, I just listed some from the top of my head.

Since you seem to be interested in virtualisation and security,
another option would be to figure out how to set up virtualisation for
Mac OS X 10.6-10.14 (ideally for all of them, but even if just some
are done, that's still a lot), use libvirt to start and stop them, and
then figure out how to integrate this into our buildbot setup to test
pull requests in a secure way. You would need to prove to be
standalone enough to perform the research and execution.

There's another proposal for other buildbot improvements this year,
but properly doing this one task would also take a while and could be
a separate task.

(You don't have much time left until the proposal deadline though.)

Mojca


My proposal link : 
https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing



Best Regards,
Amir


On Apr 06, 2019, at 04:21 PM, Mojca Miklavec  wrote:


Dear Amir,


Nice to see you at MacPorts!


You should send your email to the macports development mailing list
and you would get the answer there. (I'm forwarding it to the mailing
list now, but you should subscribe for the future.)


On Sat, 6 Apr 2019 at 20:14, AmirAli Mashayekhi wrote:




Dear Macports GSoC admins,




I have uploaded my proposal draft on GSoC webpage. Unfortunately we at 
EPITA(mon École) got informed late about this years GSoC opportunity. But this 
doesn’t affect my true excitement about participation, open source and 
contributing on them.




Late arrival of proposals is not a problem per se, it just means that
you have less time to improve your proposal and demonstrate your
skills compared to students who started early, so you need to excel
more to catch up.


My friendly suggestion would be to not try to write three proposals in
two days, as that will more likely decrease your chances of getting
selected, rather than increase them. Not because students applying to
other orgs would sound less serious, but because you hardly have
sufficient time to write one good proposal, let alone three.


I appreciate if there is any feedbacks so I can enhance it before finalising my 
application.




You nicely answered the auxiliary questions (meant to help us
understand the student better), but the project proposal is completely
missing.


The proposal fails to show any effort to research and understand the
task. The description / plan is so vague that you could in theory
submit a three-line patch to an arbitrary outdated port (for the whole
summer) and claim that you have delivered what you promised. Not even
a single piece of software was named as an example of what you would
work with (after six years of using MacPorts you should at least be
able to name one single piece of software that you wanted to use, but
was too old or missing).


In any case we are asking all of our candidate students to create some
pull requests or small demo projects demonstrating their
problem-solving skills. This would be the perfect opportunity to
update & improve a couple of important outdated unmaintained ports of
your interest.


My draft should be available on your dashboard by

Re: GSoC 2019 [Collect build statistics]

[Mojca Miklavec]

Dear Arjun,

On Mon, 8 Apr 2019 at 20:31, Arjun Salyan wrote:
>
> Dear MacPorts Community,
>
> I have submitted my Final Proposal. I do understand that during these last 
> hours it might not be possible to give feedback on the proposals. But if I am 
> lucky enough to get more of them, I will try to get the job done (around 23 
> hours still remaining).
>
> Google Doc: 
> https://docs.google.com/document/d/198Ivygxb2NJQz_sqzDrbDPVEYZ5Ye5Yw0LV6Bt2QmG4/edit?usp=sharing

No special feedback (we still need to review the pull requests though).

A general suggestion from me would be to study in depth some good and
exhaustive book on relational database design to fill in the holes.
(There might also be some online courses.)

Mojca

Feedback Request for final GSoC Proposal [Speed Up Trace Mode]

[Mihir Luthra]

Hi everyone,

Kindly check my final proposal and provide any possible feedback.

https://docs.google.com/document/d/14eSXwZ6N1vRcBudaJWlwO5G17dY5B1nHgfhh52tSnv0/edit#


Regards,
Mihir

Re: GSoC 2019 [Phase out dependency on Xcode]

[Clemens Lang]

Hi Satryaji,

On Mon, Apr 08, 2019 at 10:44:43AM +0700, Satryaji Aulia wrote:
> Hi Clemens, I'd also like your feedback on my proposal, particularly
> about Xcode dependency. What do you think?

I've left a couple of comments in your Google Doc, but overall yours is
a solid proposal. Sorry for only picking up on your thread now, but
we've been somewhat overwhelmed by GSoC proposals this year.

I didn't comment on it in the proposal document, but I like your work on
the bump tool. Haven't looked at the code, but the screen recording
looks like a very helpful tool. We've actually wanted something like
this for a long time, but haven't found the time to implement it. I
don't have time to review the PR right now, but please feel free to keep
pestering me about it weekly or biweekly until I get to it.

> Also thanks for the 1 hour tutorial on macports-base. It helped a lot
> with understanding the MacPorts system, not just me but other
> prospective GSOC students as I read on the mailing list.

Any time, glad this MacPorts Meeting recording has been so helpful to
people over the years :)

-- 
Clemens

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

On Tue, Apr 9, 2019 at 1:59 AM Clemens Lang  wrote:

> Hi,
>
> On Sun, Apr 07, 2019 at 01:03:12AM +0530, Mihir Luthra wrote:
> > > Can you pastebin a main.log of a build that fails like this?
> >
> > https://pastebin.com/FVdp4WTw
>
> The problematic lines are
>
> :debug:archivefetch failed verification with key
> /opt/local/share/macports/macports-pubkey.pem
> :debug:archivefetch openssl output: Verified OK
>
> which is very weird, because it says the verification failed, but it
> also says the OpenSSL output is "Verified OK", which says exactly the
> opposite.
>
> The command it runs is
>
>   openssl dgst \
> -ripemd160 \
> -verify \
> -pubkey /opt/local/share/macports/macports-pubkey.pem \
> -signature ncurses-6.1_0.darwin_18.x86_64.tbz2.rmd160 \
> ncurses-6.1_0.darwin_18.x86_64.tbz2
>
> Maybe you can reproduce this command on your system manually and see if
> it works? It seems this command will return a non-zero exit code for
> reasons I don't know.
>

I will surely check it tomorrow, as today is the last date of application
submission so improvising it as much as I can :)

Also can you please give a look to the proposal:

https://docs.google.com/document/d/14eSXwZ6N1vRcBudaJWlwO5G17dY5B1nHgfhh52tSnv0/edit?usp=sharing

I was editing the schedule content to fit it into table so its not
currently done.
Rest others, i.e. methodology and phases which contain all the main
information are done. Please consider giving some feedback. :)

 regards,
Mihir

Re: GSoC 2019 proposal - Feedback request

[Clemens Lang]

Hi AmirAli,

On Sat, Apr 06, 2019 at 06:17:25PM +, AmirAli Mashayekhi via macports-dev 
wrote:
> I have uploaded my proposal draft on GSoC webpage.
>
> Unfortunately we at EPITA(mon École) got informed late about this
> years GSoC opportunity. But this doesn’t affect my true excitement
> about participation, open source and contributing on them.

Unfortunately it is quite late in the application process, which does
limit the time to get feedback and improve your proposal. Do keep in
mind though, that there might be a GSoC next year and there will be
another opportunity to participate!

> I appreciate if there is any feedbacks so I can enhance it before
> finalising my application.

> I shared a link here in case of comments.
> My draft link : 
> https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing

If I understood your proposal correctly, you want to work on a number of
orphaned and missing ports. It would be good if you could improve your
proposal with more details, i.e. which categories of ports are you
interested in? Command line tools, science-related tools, software
development tools, etc. MacPorts has a huge number of ports, and getting
to all of them is likely way too much work for a summer, so you should
choose a more focused area, and ideally a number of ports that you want
to bring up to date.

Additionally, we're always looking for a detailed timeline in proposals,
i.e. what are you planning to do when?

Note that you also haven't followed our application template:
  https://trac.macports.org/wiki/SummerOfCodeApplicationTemplate
You should correct that.

-- 
Clemens

Re: Speed up trace mode (GSoC Project)

[Clemens Lang]

Hi,

On Sun, Apr 07, 2019 at 01:03:12AM +0530, Mihir Luthra wrote:
> > Can you pastebin a main.log of a build that fails like this?
> 
> https://pastebin.com/FVdp4WTw

The problematic lines are

:debug:archivefetch failed verification with key 
/opt/local/share/macports/macports-pubkey.pem
:debug:archivefetch openssl output: Verified OK

which is very weird, because it says the verification failed, but it
also says the OpenSSL output is "Verified OK", which says exactly the
opposite.

The command it runs is

  openssl dgst \
-ripemd160 \
-verify \
-pubkey /opt/local/share/macports/macports-pubkey.pem \
-signature ncurses-6.1_0.darwin_18.x86_64.tbz2.rmd160 \
ncurses-6.1_0.darwin_18.x86_64.tbz2

Maybe you can reproduce this command on your system manually and see if
it works? It seems this command will return a non-zero exit code for
reasons I don't know.

-- 
Clemens

Re: GSoC 2019 [Buildbot ideas]

[Pierre Tardy]

A javascript library which you can simply call from your main.js and
which will implement all the boilerplate needed to connect your vue
component to the angularjs application

Regards
Pierre

Le lun. 8 avr. 2019 à 21:23, Rajdeep Bharati
 a écrit :
>
> Hi Pierre,
>
> By library do you mean a library of components and JSON interfaces?
>
> Regards
> Rajdeep
>
> On Tue, Apr 9, 2019 at 12:18 AM Pierre Tardy  wrote:
>>
>> Hi Rajdeep,
>>
>> We already have a yeoman generator:
>> https://github.com/buildbot/guanlecoja/tree/master/generator-guanlecoja
>>
>> I think the current boilerplate has more than just boilerplate.
>> It also has some code that could be factorized.
>> It is better not to put too much in a generator, as it is hard to
>> upgrade after a first instanciation
>> So I would recommend a generator plus a library.
>>
>> Regards
>> Pierre
>> Pierre
>>
>> Le lun. 8 avr. 2019 à 18:44, Rajdeep Bharati
>>  a écrit :
>> >
>> > @Mojca Miklavec Oh, I see.
>> >
>> > @Pierre Tardy Regarding the npm package: I think it would be better to 
>> > create a project generator CLI (which would create the boilerplate 
>> > project, similar to vue-cli or create-react-app). Do you think that would 
>> > work? Can I use yeoman to do this?
>> >
>> > Thank you for help.
>> > Rajdeep
>> >
>> > On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec  wrote:
>> >>
>> >> Dear Rajdeep,
>> >>
>> >> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
>> >> >
>> >> > I found something which might help us: https://bors.tech/homu-io/ 
>> >> > https://github.com/barosl/homu.
>> >> > I will try it set it up on my local system and get back to you.
>> >> > Mojca
>> >>
>> >> This sounds nice, but I wonder if it would actually address any of our
>> >> problems (I suspect not). Also, it explicitly says that the project
>> >> was kind of abandoned and continued elsewhere?
>> >>
>> >> What this tool is trying to solve is to check the code again after
>> >> potentially long code review process, and run CI again "just before
>> >> merging". However some of our builds would take hours, or, well, on
>> >> Travis you might get 2 hours of waiting time and then one hour of
>> >> building, just to experience a timeout at the end anyway. This makes
>> >> the "just before merging" kind of not-too-well-defined. Plus, half of
>> >> our Travis failures are false negatives, and before this can be useful
>> >> on the buildbot, we would need to be able to enable checking pull
>> >> requests in a "safe way" (addendum: for such checking we could
>> >> probably just as well engage a couple of machines which are completely
>> >> decoupled from our main builders, which would run, say, just 10.5,
>> >> 10.6, 10.9, and would not upload the results anywhere). When checking
>> >> the base ... we have so little activity that this is practically never
>> >> an issue. And with ports there are so many distinct ports, that this
>> >> is also hardly ever an issue; and when it is, you would already notice
>> >> that the PR cannot be merged.
>> >>
>> >> Mojca

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Hi Pierre,

By library do you mean a library of components and JSON interfaces?

Regards
Rajdeep

On Tue, Apr 9, 2019 at 12:18 AM Pierre Tardy  wrote:

> Hi Rajdeep,
>
> We already have a yeoman generator:
> https://github.com/buildbot/guanlecoja/tree/master/generator-guanlecoja
>
> I think the current boilerplate has more than just boilerplate.
> It also has some code that could be factorized.
> It is better not to put too much in a generator, as it is hard to
> upgrade after a first instanciation
> So I would recommend a generator plus a library.
>
> Regards
> Pierre
> Pierre
>
> Le lun. 8 avr. 2019 à 18:44, Rajdeep Bharati
>  a écrit :
> >
> > @Mojca Miklavec Oh, I see.
> >
> > @Pierre Tardy Regarding the npm package: I think it would be better to
> create a project generator CLI (which would create the boilerplate project,
> similar to vue-cli or create-react-app). Do you think that would work? Can
> I use yeoman to do this?
> >
> > Thank you for help.
> > Rajdeep
> >
> > On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec 
> wrote:
> >>
> >> Dear Rajdeep,
> >>
> >> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
> >> >
> >> > I found something which might help us: https://bors.tech/homu-io/
> https://github.com/barosl/homu.
> >> > I will try it set it up on my local system and get back to you.
> >> > Mojca
> >>
> >> This sounds nice, but I wonder if it would actually address any of our
> >> problems (I suspect not). Also, it explicitly says that the project
> >> was kind of abandoned and continued elsewhere?
> >>
> >> What this tool is trying to solve is to check the code again after
> >> potentially long code review process, and run CI again "just before
> >> merging". However some of our builds would take hours, or, well, on
> >> Travis you might get 2 hours of waiting time and then one hour of
> >> building, just to experience a timeout at the end anyway. This makes
> >> the "just before merging" kind of not-too-well-defined. Plus, half of
> >> our Travis failures are false negatives, and before this can be useful
> >> on the buildbot, we would need to be able to enable checking pull
> >> requests in a "safe way" (addendum: for such checking we could
> >> probably just as well engage a couple of machines which are completely
> >> decoupled from our main builders, which would run, say, just 10.5,
> >> 10.6, 10.9, and would not upload the results anywhere). When checking
> >> the base ... we have so little activity that this is practically never
> >> an issue. And with ports there are so many distinct ports, that this
> >> is also hardly ever an issue; and when it is, you would already notice
> >> that the PR cannot be merged.
> >>
> >> Mojca
>

Re: GSoC 2019 [Buildbot ideas]

[Pierre Tardy]

Hi Rajdeep,

We already have a yeoman generator:
https://github.com/buildbot/guanlecoja/tree/master/generator-guanlecoja

I think the current boilerplate has more than just boilerplate.
It also has some code that could be factorized.
It is better not to put too much in a generator, as it is hard to
upgrade after a first instanciation
So I would recommend a generator plus a library.

Regards
Pierre
Pierre

Le lun. 8 avr. 2019 à 18:44, Rajdeep Bharati
 a écrit :
>
> @Mojca Miklavec Oh, I see.
>
> @Pierre Tardy Regarding the npm package: I think it would be better to create 
> a project generator CLI (which would create the boilerplate project, similar 
> to vue-cli or create-react-app). Do you think that would work? Can I use 
> yeoman to do this?
>
> Thank you for help.
> Rajdeep
>
> On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec  wrote:
>>
>> Dear Rajdeep,
>>
>> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
>> >
>> > I found something which might help us: https://bors.tech/homu-io/ 
>> > https://github.com/barosl/homu.
>> > I will try it set it up on my local system and get back to you.
>> > Mojca
>>
>> This sounds nice, but I wonder if it would actually address any of our
>> problems (I suspect not). Also, it explicitly says that the project
>> was kind of abandoned and continued elsewhere?
>>
>> What this tool is trying to solve is to check the code again after
>> potentially long code review process, and run CI again "just before
>> merging". However some of our builds would take hours, or, well, on
>> Travis you might get 2 hours of waiting time and then one hour of
>> building, just to experience a timeout at the end anyway. This makes
>> the "just before merging" kind of not-too-well-defined. Plus, half of
>> our Travis failures are false negatives, and before this can be useful
>> on the buildbot, we would need to be able to enable checking pull
>> requests in a "safe way" (addendum: for such checking we could
>> probably just as well engage a couple of machines which are completely
>> decoupled from our main builders, which would run, say, just 10.5,
>> 10.6, 10.9, and would not upload the results anywhere). When checking
>> the base ... we have so little activity that this is practically never
>> an issue. And with ports there are so many distinct ports, that this
>> is also hardly ever an issue; and when it is, you would already notice
>> that the PR cannot be merged.
>>
>> Mojca

Re: GSoC 2019 [Collect build statistics]

[Arjun Salyan via macports-dev]

Dear MacPorts Community,

I have submitted my Final Proposal. I do understand that during these last
hours it might not be possible to give feedback on the proposals. But if I
am lucky enough to get more of them, I will try to get the job done (around
23 hours still remaining).

Google Doc:
https://docs.google.com/document/d/198Ivygxb2NJQz_sqzDrbDPVEYZ5Ye5Yw0LV6Bt2QmG4/edit?usp=sharing

Thank for being so helpful!

Re: GSoC Proposal

[KARAN SHETH via macports-dev]

Hey,

Just a gentle reminder, If I could get a reply to my previous mail as
the deadline for submission is now less than a day and I would like to
submit the proposal well before the deadline.
Sorry for remailing.

Thanks,
Karan Sheth

On Mon, Apr 8, 2019 at 12:05 AM KARAN SHETH  wrote:
>
> Hey,
>
> > There's nothing specific that I would want to comment on.
> > Well, to me it's not entirely clear how the automatic updates would be
> > supposed to work, but ...
>
> I made some changes to the update part, is it better now or am I still
> missing something?
>
> > And the stretch goals are somewhat short :)
>
> Ya I am aware of this and am waiting for some good idea to put in there.
> Should I add PortGroup for NPM as stretch goal?
>
> > The only thing (mentioned yesterday) is that you either try to support
> > Haskell, or you don't even try. If we want to try (and spend the exact
> > same amount of time as you initially planned), then it makes sense to
> > spend a bit of time earlier than in week 11. If you only start
> > exploring the options in week 11, I would say that it's nearly
> > guaranteed that you would run out of time while potentially waiting
> > for someone else to do something, or waiting for some good ideas to
> > pop up.
> > If you would want to do Haskell, you would probably need to plant a
> > seed early (not spend more than half a day or a day in any case), let
> > it ripe while working on other important features, and then see if
> > this is feasible / worth finishing or not. I still don't know what the
> > major blocks are, but just because something is not written in json
> > should not be the reason to give up. The format still seems parsable
> > to me (not sure how many complications would arise).
> >
> > http://hackage.haskell.org/package/cabal2arch
> > http://hackage.haskell.org/package/cabal2nix
> > http://hackage.haskell.org/package/cabal2spec
> > https://github.com/ddssff/cabal-debian
> >
> > Please note that I'm not implying that this is of vital importance to
> > the project. A well written support for automatic updates of
> > python/ruby/perl/npm ports would be worth a lot more than a bunch of
> > semi-broken haskell/cabal packages.
>
> I have changed the timeline for Haskell now and the basic structure of
> it starts at week3.
>
> Should I add the PR links somewhere in the proposal ?
>
> Thanks,
> Karan Sheth

-- 

           
      

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

@Mojca Miklavec  Oh, I see.

@Pierre Tardy  Regarding the npm package: I think it
would be better to create a project generator CLI (which would create the
boilerplate project, similar to vue-cli or create-react-app). Do you think
that would work? Can I use yeoman to do this?

Thank you for help.
Rajdeep

On Mon, Apr 8, 2019 at 7:05 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
> >
> > I found something which might help us: https://bors.tech/homu-io/
> https://github.com/barosl/homu.
> > I will try it set it up on my local system and get back to you.
> > Mojca
>
> This sounds nice, but I wonder if it would actually address any of our
> problems (I suspect not). Also, it explicitly says that the project
> was kind of abandoned and continued elsewhere?
>
> What this tool is trying to solve is to check the code again after
> potentially long code review process, and run CI again "just before
> merging". However some of our builds would take hours, or, well, on
> Travis you might get 2 hours of waiting time and then one hour of
> building, just to experience a timeout at the end anyway. This makes
> the "just before merging" kind of not-too-well-defined. Plus, half of
> our Travis failures are false negatives, and before this can be useful
> on the buildbot, we would need to be able to enable checking pull
> requests in a "safe way" (addendum: for such checking we could
> probably just as well engage a couple of machines which are completely
> decoupled from our main builders, which would run, say, just 10.5,
> 10.6, 10.9, and would not upload the results anywhere). When checking
> the base ... we have so little activity that this is practically never
> an issue. And with ports there are so many distinct ports, that this
> is also hardly ever an issue; and when it is, you would already notice
> that the PR cannot be merged.
>
> Mojca
>

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

Dear Rajdeep,

On Sun, 7 Apr 2019 at 19:41, Rajdeep Bharati wrote:
>
> I found something which might help us: https://bors.tech/homu-io/ 
> https://github.com/barosl/homu.
> I will try it set it up on my local system and get back to you.
> Mojca

This sounds nice, but I wonder if it would actually address any of our
problems (I suspect not). Also, it explicitly says that the project
was kind of abandoned and continued elsewhere?

What this tool is trying to solve is to check the code again after
potentially long code review process, and run CI again "just before
merging". However some of our builds would take hours, or, well, on
Travis you might get 2 hours of waiting time and then one hour of
building, just to experience a timeout at the end anyway. This makes
the "just before merging" kind of not-too-well-defined. Plus, half of
our Travis failures are false negatives, and before this can be useful
on the buildbot, we would need to be able to enable checking pull
requests in a "safe way" (addendum: for such checking we could
probably just as well engage a couple of machines which are completely
decoupled from our main builders, which would run, say, just 10.5,
10.6, 10.9, and would not upload the results anywhere). When checking
the base ... we have so little activity that this is practically never
an issue. And with ports there are so many distinct ports, that this
is also hardly ever an issue; and when it is, you would already notice
that the PR cannot be merged.

Mojca

Re: GSoC 2019 [Phase out dependency on Xcode]

[Mojca Miklavec]

On Sun, 7 Apr 2019 at 16:36, Satryaji Aulia wrote:
>
> Hi Marcus and Mojca.
>
> I’ve fleshed out my proposal (also my understanding of MacOS compilers)
> again. To disallow Xcode compilers, I added my idea to use a compiler
> blacklist which implementation already exists. I’ve also worked a lot on my
> macports-base PR and am confident that I can comfortably code in Tcl.
>
> I marked the PR ready for review as it is now already functional.
> Demo here: https://asciinema.org/a/1X9VFUxRXncOE1m7PhB9DVKcX

Amazing new feature, thank you!

> Any feedback on my proposal would be greatly appreciated. Thank you.

Please note that I have absolutely no idea how difficult it is to
implement these features (I'm not familiar with the base too much, so
it makes sense to listen to Marcus or Clemens or someone else from the
core team, don't take me as an authority in this area).

Here are some of my random ideas that could be either parts of
proposal or stretch goals:

(a) https://trac.macports.org/ticket/52575 (report xcode version in
main.log; should be simple to do)

(b) https://trac.macports.org/ticket/15712 (declarative syntax to
specify incompatible macOS versions)

(c) https://trac.macports.org/ticket/56093 (simplify blacklisting compilers)

(d) (There are probably a couple other things that could be done with
trace mode.)

(e) Probably more of a stretch goal anyway: It could be nice to
support "submit pull request" as the next step of "port bump". Maybe
from web interface, once we get there :)

>> Ruby is still an issue, whether or not that's urgent is a matter of
>> taste. Die-hard rubyist would probably turn to the other package
>> manager :). We are currently also not packaging asciidoctor, for
>> example (which would be nice to have for our documentation / guide).
>
> Of course, sorry to assume it’s not urgent.

I'm not saying it is. I just said that it's hard to say :)

> Also, I used port bump on
> asciidoctor to test it out and submitted a PR.

Cool, thank you very much.
I then realised that it wasn't asciidoctor we were missing as a
package, but docbookrx (which we will need for conversion of our guide
which is currently in docbook).
Sorry for misguidance, but thank you for the update in any case.

Mojca

Re: GSoC 2019 - Feedback request

[Mojca Miklavec]

Dear Amir,

On Sun, 7 Apr 2019 at 18:21, AmirAli Mashayekhi wrote:
>
> Dear Mojca (and everybody else in the mail list),
>
> Thank you for your response, Sorry for the misundrestanding of the proposal.
>
> As requested I also tried to expose myself to the whole workflow of Macports 
> developement cycle.
> Here is a link for my pull request : 
> https://github.com/macports/macports-ports/pull/4039

Thank you.
Just please change the commit message, force push to the branch and
the it's good to go (the Travis failures are unrelated and Azure is
happy, so that's fine).

> After spending the day to check the ports in the tree (this way I also found 
> mlterm to update as I mentioned above), honestly I think I need a little bit 
> of hint from the community to see and select which port right now fits my 
> abilities. nevertheless I have updated my proposal's general structure, I 
> hope it makes more sense for you.
> The only missing part atm is to fill the port name I will work on...

You may check https://trac.macports.org/wiki/Tickets and see which
ports have more bugs, update requests etc. filed. Esp. those that have
been waiting for years.

Candidates for GSOC could be:
- KDE
- QT5 (it works, but there is a lot of work to be done with respect to
older macOS version)
- MySQL (not enough for the summer, but it needs a maintainer), Pandoc, ...
- full ruby gems spectrum, full haskell / cabal spectrum, lisp, ...
- ... plenty of others, I just listed some from the top of my head.

Since you seem to be interested in virtualisation and security,
another option would be to figure out how to set up virtualisation for
Mac OS X 10.6-10.14 (ideally for all of them, but even if just some
are done, that's still a lot), use libvirt to start and stop them, and
then figure out how to integrate this into our buildbot setup to test
pull requests in a secure way. You would need to prove to be
standalone enough to perform the research and execution.

There's another proposal for other buildbot improvements this year,
but properly doing this one task would also take a while and could be
a separate task.

(You don't have much time left until the proposal deadline though.)

Mojca

> My proposal link : 
> https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing
>
> Best Regards,
> Amir
>
> On Apr 06, 2019, at 04:21 PM, Mojca Miklavec  wrote:
>
> Dear Amir,
>
> Nice to see you at MacPorts!
>
> You should send your email to the macports development mailing list
> and you would get the answer there. (I'm forwarding it to the mailing
> list now, but you should subscribe for the future.)
>
> On Sat, 6 Apr 2019 at 20:14, AmirAli Mashayekhi wrote:
>
>
> Dear Macports GSoC admins,
>
>
> I have uploaded my proposal draft on GSoC webpage. Unfortunately we at 
> EPITA(mon École) got informed late about this years GSoC opportunity. But 
> this doesn’t affect my true excitement about participation, open source and 
> contributing on them.
>
>
> Late arrival of proposals is not a problem per se, it just means that
> you have less time to improve your proposal and demonstrate your
> skills compared to students who started early, so you need to excel
> more to catch up.
>
> My friendly suggestion would be to not try to write three proposals in
> two days, as that will more likely decrease your chances of getting
> selected, rather than increase them. Not because students applying to
> other orgs would sound less serious, but because you hardly have
> sufficient time to write one good proposal, let alone three.
>
> I appreciate if there is any feedbacks so I can enhance it before finalising 
> my application.
>
>
> You nicely answered the auxiliary questions (meant to help us
> understand the student better), but the project proposal is completely
> missing.
>
> The proposal fails to show any effort to research and understand the
> task. The description / plan is so vague that you could in theory
> submit a three-line patch to an arbitrary outdated port (for the whole
> summer) and claim that you have delivered what you promised. Not even
> a single piece of software was named as an example of what you would
> work with (after six years of using MacPorts you should at least be
> able to name one single piece of software that you wanted to use, but
> was too old or missing).
>
> In any case we are asking all of our candidate students to create some
> pull requests or small demo projects demonstrating their
> problem-solving skills. This would be the perfect opportunity to
> update & improve a couple of important outdated unmaintained ports of
> your interest.
>
> My draft should be available on your dashboard by now, but I shared a link 
> here in case of comments.
>
> My draft link : 
> https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing
>
>
> Best Regards,
>
> Amir
>
>
> Please note that this was meant as a constructive feedback.
>
> Mojca

Re: GSoC 2019 [Phase out dependency on Xcode]

[Satryaji Aulia]

Hi Clemens, I'd also like your feedback on my proposal,
particularly about Xcode dependency. What do you think?

Also thanks for the 1 hour tutorial on macports-base. It helped
a lot with understanding the MacPorts system, not just me but
other prospective GSOC students as I read on the mailing list.

On Sun, Apr 7, 2019 at 9:36 PM Satryaji Aulia  wrote:
>
> Hi Marcus and Mojca.
>
> I’ve fleshed out my proposal (also my understanding of MacOS compilers)
> again. To disallow Xcode compilers, I added my idea to use a compiler
> blacklist which implementation already exists. I’ve also worked a lot on my
> macports-base PR and am confident that I can comfortably code in Tcl.
>
> I marked the PR ready for review as it is now already functional.
> Demo here: https://asciinema.org/a/1X9VFUxRXncOE1m7PhB9DVKcX
>
> Any feedback on my proposal would be greatly appreciated. Thank you.

Re: GSoC Proposal

[KARAN SHETH via macports-dev]

Hey,

> There's nothing specific that I would want to comment on.
> Well, to me it's not entirely clear how the automatic updates would be
> supposed to work, but ...

I made some changes to the update part, is it better now or am I still
missing something?

> And the stretch goals are somewhat short :)

Ya I am aware of this and am waiting for some good idea to put in there.
Should I add PortGroup for NPM as stretch goal?

> The only thing (mentioned yesterday) is that you either try to support
> Haskell, or you don't even try. If we want to try (and spend the exact
> same amount of time as you initially planned), then it makes sense to
> spend a bit of time earlier than in week 11. If you only start
> exploring the options in week 11, I would say that it's nearly
> guaranteed that you would run out of time while potentially waiting
> for someone else to do something, or waiting for some good ideas to
> pop up.
> If you would want to do Haskell, you would probably need to plant a
> seed early (not spend more than half a day or a day in any case), let
> it ripe while working on other important features, and then see if
> this is feasible / worth finishing or not. I still don't know what the
> major blocks are, but just because something is not written in json
> should not be the reason to give up. The format still seems parsable
> to me (not sure how many complications would arise).
>
> http://hackage.haskell.org/package/cabal2arch
> http://hackage.haskell.org/package/cabal2nix
> http://hackage.haskell.org/package/cabal2spec
> https://github.com/ddssff/cabal-debian
>
> Please note that I'm not implying that this is of vital importance to
> the project. A well written support for automatic updates of
> python/ruby/perl/npm ports would be worth a lot more than a bunch of
> semi-broken haskell/cabal packages.

I have changed the timeline for Haskell now and the basic structure of
it starts at week3.

Should I add the PR links somewhere in the proposal ?

Thanks,
Karan Sheth

-- 

           
      

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

I found something which might help us: https://bors.tech/homu-io/
https://github.com/barosl/homu.
I will try it set it up on my local system and get back to you.

Rajdeep

On Sun, Apr 7, 2019 at 6:17 PM Rajdeep Bharati 
wrote:

> I was attempting to package the plugin.
> A possible workflow may be:
>
>- The user would install the package in something like a vue-cli or
>create-react-app.
>- Write the component
>- Pass that component to the `link` function of the plugin.
>- There would be options to customize the settings(limit, order,
>adding other props, etc) (or even write the whole link function themselves)
>
> One problem that I am facing is getting the angular dependency inside the
> vue/react app which is utilizing the npm package(plugin). Here's the code:
> https://github.com/rajdeepbharati/buildbot-vue-plugin-boilerplate/tree/plugin
>
> Thank you.
> Rajdeep
>
> On Sat, Apr 6, 2019 at 9:05 PM Rajdeep Bharati 
> wrote:
>
>> Thanks for the feedback!
>>
>> On Sat, Apr 6, 2019 at 7:03 PM Mojca Miklavec  wrote:
>>
>>> Dear Rajdeep,
>>>
>>> On Sat, 6 Apr 2019 at 13:33, Rajdeep Bharati wrote:
>>> >
>>> > I was wondering what kind of features you would like to have in the
>>> new waterfall view?
>>>
>>> I don't remember whether I ever explicitly said I wanted some new
>>> features there :)
>>>
>>> Probably the only thing I really miss is the portname explicitly seen
>>> without having to click (or mouse-over) the rectangle.
>>>
>>> Way less important:
>>> - in case the build failed, it would be nice to see which step (first)
>>> failed, again without having to move the mouse around
>>> - in case the build is still running, see which step is running (I
>>> would also say "time since start" or "estimated time to finish", but I
>>> don't want a ticking timebomb that keeps changing every second, so you
>>> may safely ignore this)
>>>
>>> Generally the old waterfall had all the necessary info displayed. The
>>> new one is more compact (I would also say not too attractive design,
>>> but I'm not someone to judge that as I'm not a designer), but is
>>> basically lacking all the info. If one builds the same thing under the
>>> same builder that might be fine, but we build something else each time
>>> and without knowing what was built, those green and red squares are
>>> basically useless.
>>>
>>>
>>> Some more ideas for thinking. One of the most desperately needed
>>> features to make buildbot views usable for anything else than "what's
>>> currently being built" would be to sort (filter) by port name.
>>>
>>> Now, port name is something specific to MacPorts, but we could
>>> probably introduce filtering by any kind of keywords. We could write
>>> to our master.cfg that we want to create a new filter with name "port"
>>> and then each port would get a keyword matching its name. Then I could
>>> ask buildbot to display me history of all builds of port "clang-7.0".
>>> But then a few months later we might realize that we want to display
>>> all the failed builds from a particular maintainer. Maintainer is
>>> again something specific to MacPorts, but we could simply introduce a
>>> new category "maintainer" on the fly and use the github handles as
>>> keywords to search for, and I could ask buildbot to display me all
>>> broken builds belonging to maintainer "mojca" without any changes in
>>> the buildbot or the view itself, just by slight modification in
>>> master.cfg. One year later we could add an arbitrary new category that
>>> we have never even thought of, and be able to filter according to that
>>> one (maybe: show me all builds of python or perl modules; show me all
>>> builds with non-free licence; show me all builds from ports fetching
>>> from git, ...). Those categories could potentially be nested.
>>>
>>> Mojca
>>>
>>

Re: GSoC 2019 - Feedback request

[AmirAli Mashayekhi via macports-dev]

Dear Mojca (and everybody else in the mail list),



Thank you for your response, Sorry for the misundrestanding of the proposal.



As requested I also tried to expose myself to the whole workflow of Macports 
developement cycle.

Here is a link for my pull request : 
https://github.com/macports/macports-ports/pull/4039



After spending the day to check the ports in the tree (this way I also found 
mlterm to update as I mentioned above), honestly I think I need a little bit of 
hint from the community to see and select which port right now fits my 
abilities. nevertheless I have updated my proposal's general structure, I hope 
it makes more sense for you.

The only missing part atm is to fill the port name I will work on...



My proposal link : 
https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing



Best Regards,

Amir


On Apr 06, 2019, at 04:21 PM, Mojca Miklavec  wrote:


Dear Amir,

Nice to see you at MacPorts!

You should send your email to the macports development mailing list
and you would get the answer there. (I'm forwarding it to the mailing
list now, but you should subscribe for the future.)

On Sat, 6 Apr 2019 at 20:14, AmirAli Mashayekhi wrote:



Dear Macports GSoC admins,


I have uploaded my proposal draft on GSoC webpage. Unfortunately we at 
EPITA(mon École) got informed late about this years GSoC opportunity. But this 
doesn’t affect my true excitement about participation, open source and 
contributing on them.

Late arrival of proposals is not a problem per se, it just means that
you have less time to improve your proposal and demonstrate your
skills compared to students who started early, so you need to excel
more to catch up.

My friendly suggestion would be to not try to write three proposals in
two days, as that will more likely decrease your chances of getting
selected, rather than increase them. Not because students applying to
other orgs would sound less serious, but because you hardly have
sufficient time to write one good proposal, let alone three.


I appreciate if there is any feedbacks so I can enhance it before finalising my 
application.

You nicely answered the auxiliary questions (meant to help us
understand the student better), but the project proposal is completely
missing.

The proposal fails to show any effort to research and understand the
task. The description / plan is so vague that you could in theory
submit a three-line patch to an arbitrary outdated port (for the whole
summer) and claim that you have delivered what you promised. Not even
a single piece of software was named as an example of what you would
work with (after six years of using MacPorts you should at least be
able to name one single piece of software that you wanted to use, but
was too old or missing).

In any case we are asking all of our candidate students to create some
pull requests or small demo projects demonstrating their
problem-solving skills. This would be the perfect opportunity to
update & improve a couple of important outdated unmaintained ports of
your interest.


My draft should be available on your dashboard by now, but I shared a link here 
in case of comments.
My draft link : 
https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing



Best Regards,
Amir

Please note that this was meant as a constructive feedback.

Mojca

Re: GSoC 2019 [Phase out dependency on Xcode]

[Satryaji Aulia]

Hi Marcus and Mojca.I’ve fleshed out my proposal (also my understanding of MacOS compilers)again. To disallow Xcode compilers, I added my idea to use a compilerblacklist which implementation already exists. I’ve also worked a lot on mymacports-base PR and am confident that I can comfortably code in Tcl.I marked the PR ready for review as it is now already functional.Demo here: https://asciinema.org/a/1X9VFUxRXncOE1m7PhB9DVKcXAny feedback on my proposal would be greatly appreciated. Thank you.On 2 Apr 2019, at 03.45, Mojca Miklavec  wrote:Ruby is still an issue, whether or not that's urgent is a matter oftaste. Die-hard rubyist would probably turn to the other packagemanager :). We are currently also not packaging asciidoctor, forexample (which would be nice to have for our documentation / guide).Of course, sorry to assume it’s not urgent. Also, I used port bump onasciidoctor to test it out and submitted a PR.Also added the fact that there's already a proposal for that problem area.Project may complement each other, and students are welcome to helpeach other, in particular if one is stronger in a particular field. Ifyou are an eager rubyist, even if the upt project gets selected andthe ruby import gets implemented, the expertise and/or passion toimprove this area would still help.Alright, got it. Thank you for your help.

Re: GSoC 2019 - trace mode improvements

[Mihir Luthra]

>
>
> read [1] and related thread and probably I am too late to contribute
> through
> GSoC. So I give up.
>
>
> /davide
>
> [1]
> https://lists.macports.org/pipermail/macports-dev/2019-April/040495.html


Hi ,

https://docs.google.com/document/d/15cVbH6f6hBr9HryJEHUZEbRsToN1BAjY8My-oRstO9A/edit#heading=h.ng3lyvrem2d1

This doc is really clumsy tbh, and more like just rough notes made by me.
Though the links to learning resources attached in the document under the
heading
“surfing through the code flow”
 are quite useful. That’s all needed to understand the code and
And this video by Clemens [ https://www.youtube.com/watch?v=46qshiDskrM ]
would be useful for understanding how the trace mode code gets activated
like when and where.

With 2 days left, maybe you can give this day to learning from those links
and understanding the code and next day on project proposal.

Worth a try!
We may work together helping each other. I on speed up trace mode (well if
I get selected) & you on auto detection of build dependencies and as you
have contributed previously, you stand a better chance compared to me on
getting your proposal accepted I guess.


Regards,
Mihir

Re: GSoC 2019 - trace mode improvements

[Mojca Miklavec]

Dear Davide,

On Sun, 7 Apr 2019 at 12:55, Davide Gerhard wrote:
> On Sunday, 07/04/2019 11:55 CEST, Mojca Miklavec wrote:
> > On Sun, 7 Apr 2019 at 09:40, Davide Gerhard wrote:
> >>
> >> Sorry, I am late but I saw only the other day that MacPorts
> >> participate to the Google Summer of Code.
> >
> > Maybe we should be doing a better job in marketing :(
> > Any advice on how the message could have reached you (provided that
> > you have contributed before) would be greatly appreciated.
>
> it was you on github issue about mercurial.

Yes, I'm aware of that, but even in that case it was only you who has
been notified (a bit late), there could still be other contributors
who are / were not aware about GSOC. (Those subscribed to the mailing
list couldn't have missed the news, and we received request to move
the discussions elsewhere. We might need to set up something like
Slack in fact, but we have never in history received so many proposals
from students, so we were kind of caught by surprise this year.)

My question is: what would we need to have done in order for the news
to have reached you one month (or more) ago?

We have absolutely no idea which contributors are qualifying students,
so that we could notify them individually.

We had a notification on the first page of the website (which is
admittedly too boring to ever visit) and it was impossible to miss it
for anyone subscribed to the development mailing list, but there are
certainly other ways to reach the target population.

> > Previous contribution to the project definitely brings you "extra
> > points", but the time left is super scarce, so you need to use it
> > well.
> >
> >> Any comment will be appreciate.
> >
> > I'm not familiar with the topic, so I cannot really advise you, but:
> >
> > (1) Please get subscribed to the development mailing list.
> >
> > (2) Please carefully read the archives at
> > https://lists.macports.org/pipermail/macports-dev/
> > in particular those from March and April this year.
> >
> > (2.a) There is quite a bit of extra information about trace mode on
> > the list already.
>
> read [1] and related thread and probably I am too late to contribute through
> GSoC. So I give up.

It's entirely up to you whether you read and treat the response as a
discouragement or as a challenge. One definite way to not get selected
is by not even trying.

As I said, being a previous contributor is certainly an advantage you
have over those who have never even used MacPorts before, but not
enough on its own. You still need to know what exactly you are
proposing to implement and prove in some way that you can do it. The
proposal needs to contain sufficient details that it's clear to the
mentors that you know what you are talking about. What you sent now
doesn't contain anywhere near sufficient details.

If you only start studying the topic after you send the final
proposal, it could happen that you realise that what you propose is
not even feasible to do in the given timeframe ;)

Mojca

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

I was attempting to package the plugin.
A possible workflow may be:

   - The user would install the package in something like a vue-cli or
   create-react-app.
   - Write the component
   - Pass that component to the `link` function of the plugin.
   - There would be options to customize the settings(limit, order, adding
   other props, etc) (or even write the whole link function themselves)

One problem that I am facing is getting the angular dependency inside the
vue/react app which is utilizing the npm package(plugin). Here's the code:
https://github.com/rajdeepbharati/buildbot-vue-plugin-boilerplate/tree/plugin

Thank you.
Rajdeep

On Sat, Apr 6, 2019 at 9:05 PM Rajdeep Bharati 
wrote:

> Thanks for the feedback!
>
> On Sat, Apr 6, 2019 at 7:03 PM Mojca Miklavec  wrote:
>
>> Dear Rajdeep,
>>
>> On Sat, 6 Apr 2019 at 13:33, Rajdeep Bharati wrote:
>> >
>> > I was wondering what kind of features you would like to have in the new
>> waterfall view?
>>
>> I don't remember whether I ever explicitly said I wanted some new
>> features there :)
>>
>> Probably the only thing I really miss is the portname explicitly seen
>> without having to click (or mouse-over) the rectangle.
>>
>> Way less important:
>> - in case the build failed, it would be nice to see which step (first)
>> failed, again without having to move the mouse around
>> - in case the build is still running, see which step is running (I
>> would also say "time since start" or "estimated time to finish", but I
>> don't want a ticking timebomb that keeps changing every second, so you
>> may safely ignore this)
>>
>> Generally the old waterfall had all the necessary info displayed. The
>> new one is more compact (I would also say not too attractive design,
>> but I'm not someone to judge that as I'm not a designer), but is
>> basically lacking all the info. If one builds the same thing under the
>> same builder that might be fine, but we build something else each time
>> and without knowing what was built, those green and red squares are
>> basically useless.
>>
>>
>> Some more ideas for thinking. One of the most desperately needed
>> features to make buildbot views usable for anything else than "what's
>> currently being built" would be to sort (filter) by port name.
>>
>> Now, port name is something specific to MacPorts, but we could
>> probably introduce filtering by any kind of keywords. We could write
>> to our master.cfg that we want to create a new filter with name "port"
>> and then each port would get a keyword matching its name. Then I could
>> ask buildbot to display me history of all builds of port "clang-7.0".
>> But then a few months later we might realize that we want to display
>> all the failed builds from a particular maintainer. Maintainer is
>> again something specific to MacPorts, but we could simply introduce a
>> new category "maintainer" on the fly and use the github handles as
>> keywords to search for, and I could ask buildbot to display me all
>> broken builds belonging to maintainer "mojca" without any changes in
>> the buildbot or the view itself, just by slight modification in
>> master.cfg. One year later we could add an arbitrary new category that
>> we have never even thought of, and be able to filter according to that
>> one (maybe: show me all builds of python or perl modules; show me all
>> builds with non-free licence; show me all builds from ports fetching
>> from git, ...). Those categories could potentially be nested.
>>
>> Mojca
>>
>

Re: GSoC 2019 [Collect build statistics]

[Arjun Salyan via macports-dev]

>
> Hi,

On Fri, Apr 5, 2019 at 8:58 AM Umesh Singla 
> wrote:
>
>> It’s always to good to show your work and get feedback. It’s difficult to
>> comment on the quality otherwise. Please do not forget to make a PR.
>>
>
I have submitted the PR for adding an option to portindex (which would
generate a separate file with Changed Ports)

https://github.com/macports/macports-base/pull/121

I am still working on handling deletions.

Thank You

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

> That's unfortunately not the same, since while you're swapping the
> second value, a different thread could swap the first for a different
> value again.
>

I get your point.
Generally what comes to the mind in such circumstances is blocking the
other thread some way like mostly by spin locks.
But at the same time, if I use spin locks here, there is no point of using
compare and swap.
So need to figure out a better solution to change variables staying with
cas.

>
> That should be possible using atomic fetch-and-add [1]. This may end up
> increasing the storage by more than we need, but that's not a problem.
>
> The problem I see is that we actually need to do the truncation and
> remapping at some point. We should end up in a situation where the size
> is actually larger than the file/memory block.
>

I understand. Many checks can be put to avoid this but at last they will
make the main reason of this code which is optimisation useless.


> Unfortunately ftruncate(2) is all that we have, and without writing
> kernel code I don't see a way to do this.
>

Actually I had kind of interpose to  __dt_ftruncate() and __dt_mmap in mind
that time. I should be more precise in words, sorry for that.
These would allow us to  make the checks beforehand but the same thing
again, doing so much would just erase the optimisation benefit we are
planning.
Although I am not sure about that because memory expansion would be
required very less if we predict to a good extent the memory required.


>
> Can you pastebin a main.log of a build that fails like this?
>

https://pastebin.com/FVdp4WTw

I guess now this was for some similar reason as path or co-existence.


>
>
> [1] https://en.wikipedia.org/wiki/Fetch-and-add
>
> Thanks.


Regards,
Mihir

GSoC 2019 proposal - Feedback request

[AmirAli Mashayekhi via macports-dev]

Dear Macports Developers,


I have uploaded my proposal draft on GSoC webpage.
Unfortunately we at EPITA(mon École) got informed late about this years GSoC 
opportunity. But this doesn’t affect my true excitement about  participation, 
open source and contributing on them.


I appreciate if there is any feedbacks so I can enhance it before finalising my 
application.


I shared a link here in case of comments.
My draft link : 
https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing



Best Regards,
Amir

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

Thanks for the feedback!

On Sat, Apr 6, 2019 at 7:03 PM Mojca Miklavec  wrote:

> Dear Rajdeep,
>
> On Sat, 6 Apr 2019 at 13:33, Rajdeep Bharati wrote:
> >
> > I was wondering what kind of features you would like to have in the new
> waterfall view?
>
> I don't remember whether I ever explicitly said I wanted some new
> features there :)
>
> Probably the only thing I really miss is the portname explicitly seen
> without having to click (or mouse-over) the rectangle.
>
> Way less important:
> - in case the build failed, it would be nice to see which step (first)
> failed, again without having to move the mouse around
> - in case the build is still running, see which step is running (I
> would also say "time since start" or "estimated time to finish", but I
> don't want a ticking timebomb that keeps changing every second, so you
> may safely ignore this)
>
> Generally the old waterfall had all the necessary info displayed. The
> new one is more compact (I would also say not too attractive design,
> but I'm not someone to judge that as I'm not a designer), but is
> basically lacking all the info. If one builds the same thing under the
> same builder that might be fine, but we build something else each time
> and without knowing what was built, those green and red squares are
> basically useless.
>
>
> Some more ideas for thinking. One of the most desperately needed
> features to make buildbot views usable for anything else than "what's
> currently being built" would be to sort (filter) by port name.
>
> Now, port name is something specific to MacPorts, but we could
> probably introduce filtering by any kind of keywords. We could write
> to our master.cfg that we want to create a new filter with name "port"
> and then each port would get a keyword matching its name. Then I could
> ask buildbot to display me history of all builds of port "clang-7.0".
> But then a few months later we might realize that we want to display
> all the failed builds from a particular maintainer. Maintainer is
> again something specific to MacPorts, but we could simply introduce a
> new category "maintainer" on the fly and use the github handles as
> keywords to search for, and I could ask buildbot to display me all
> broken builds belonging to maintainer "mojca" without any changes in
> the buildbot or the view itself, just by slight modification in
> master.cfg. One year later we could add an arbitrary new category that
> we have never even thought of, and be able to filter according to that
> one (maybe: show me all builds of python or perl modules; show me all
> builds with non-free licence; show me all builds from ports fetching
> from git, ...). Those categories could potentially be nested.
>
> Mojca
>

Re: Speed up trace mode (GSoC Project)

[Clemens Lang]

Hi,

On Sat, Apr 06, 2019 at 01:05:08AM +0530, Mihir Luthra wrote:
> > You have the right ideas to solve the problem. Do keep in mind
> > though that CAS will only work up to a word size or a double word
> > size at most, i.e. swapping more than 64 bit with CAS atomically is
> > probably not going to work.
> 
> Got it. I may swap one by one instead if swapping the complete
> structure and its just 4 variables.

That's unfortunately not the same, since while you're swapping the
second value, a different thread could swap the first for a different
value again.

> > I'm not sure whether we will actually need a separate process to
> > increase the allocation size. Consider this proposal:
> >
> >   struct shared_block_mgmt {
> > size_t last_known_size;
> > size_t refcnt;
> > int blockfd;
> > struct shared_block* block;
> >   };
> >
> >   struct shared_block_mgmt block_mgmt;
> >
> >   struct shared_block {
> > size_t size;
> > size_t used;
> > char memory[];
> >   };
> >
> > This struct would mean that the first `used` bytes of `memory` are
> > in-use by our data structure and everything after that up to `size`
> > bytes is free[1].
> >
> > Any allocation request where used + request_size <= size would succeed
> > and we could change `used` using CAS to ensure that this allocation
> > operation is atomic.
> >
> > Any allocation request where used + request_size > size would trigger
> > growing the memory area. Growing would calculate the size we want to
> > grow to, let's call this `target_size`. Then, it would attempt to grow
> > atomically using:
> >
> >   size_t local_size;
> >   size_t local_used;
> >   size_t target_used;
> >   size_t target_size;
> >   bool needs_resize;
> >   do {
> > local_used = block_mgmt.block->used;
> > local_size = block_mgmt.block->size;
> > target_used = local_used + request_size;
> > needs_resize = target_used < local_size;
> > if (needs_resize) {
> >   // growing required
> >   target_size = local_size + BLOCK_GROWTH;
> >   ftruncate(block_mgmt.blockfd, target_size);
> >
> 
> What I was doing is to keep a file created beforehand and append it
> whenever needed to the existing file.
> ftruncate() is much better & cleaner approach I guess.
> 
> 
> > }
> >   } while (
> >   (needs_resize &&
> > !CAS(_mgmt.block->size, local_size, target_size) &&
> > !CAS(_mgmt.block->used, local_used, target_used)) ||
> >   (!needs_resize &&
> > !CAS(_mgmt.block->used, local_used, target_used)));
> >
> >   // At this point, either resizing was not needed and block->used is
> >   // what we expect it to be (i.e. allocation succeeded), or resizing
> >   // was needed, and we did successfully resize, and after resizing we
> >   // did update block->used (i.e. allocation also succeeded).
> >
> > This will oportunistically call ftruncate with the bigger size on the
> > mmap'd file descriptor, but calling ftruncate in multiple processes at
> > the same time should not be a problem unless the truncation would ever
> > shrink the file (which we can not completely avoid, but make very
> > unlikely by making BLOCK_GROWTH
> 
> 
> Maybe, we can call ftruncate outside the loop and just set size first.
> I dunno if it is possible but if it is we can modify compare and swap to
> behave in the following way:
> If the process detects that used + request_size > size and area needs
> to be grown, instead of swapping the size with its new value, we can
> just increment the old value without caring what is the old value by
> ‘requested_size’. Now in this type of swapping, we just don’t need to
> care what is old value, all we do is increment requested size to it.
> So if 2 or more process do this simultaneously, they just end up
> incrementing the memory correctly. And to keep a margin we can add
> some extra_growth to each.

That should be possible using atomic fetch-and-add [1]. This may end up
increasing the storage by more than we need, but that's not a problem.

The problem I see is that we actually need to do the truncation and
remapping at some point. We should end up in a situation where the size
is actually larger than the file/memory block.

> Although I m not sure if atomic operations allow this, a better
> approach maybe to just share a variable called maxFileMemory which can
> only be swapped if the new value being provided is greater. Before
> mmaping for write or read we may just ensure if current file size
> matches maxFileMemory. This maybe would prevent shrinking.
> 
> Or maybe we somehow can block shrinking by implementing our own
> ftruncate or some other way maybe.

Unfortunately ftruncate(2) is all that we have, and without writing
kernel code I don't see a way to do this.


On Sat, Apr 06, 2019 at 05:50:40PM +0530, Mihir Luthra wrote:
> I was trying to test the code by making changes but I am stuck with
> one issue. If I install from git and set it up I always receive this
> error message. (Please 

Re: Speed up trace mode (GSoC Project)

[Mihir Luthra]

Hi,

I was trying to test the code by making changes but I am stuck with one
issue.
If I install from git and set it up I always receive this error message. [1]

1) I tried `make` on base code taken as it is from git without checking out
to latest version, it showed this error.
2) I tried `make` on v2.5.4, it still shows same error.
3) I tried source install, here the installation completes successfully and
mostly everything works fine except trace mode.( I didn’t make any changes
to source code)
I tried installing 7-8 ports in this case and whenever I used trace
mode, it said unable to fetch archive.


In case of git install, it seems to be some error of that macro
HAVE_DECL_RL_USERNAME_COMPLETION_FUNCTION.
Because only if it is false, it should move to next macro and choose to use
username_completion_function.
But in make errors, it says rl_username_completion_function is declared
somewhere and exists. So I guess  HAVE_DECL_RL_USERNAME_COMPLETION_FUNCTION
should have been defined. [2]


Regards,
Mihir


https://drive.google.com/file/d/0Bz1h_hHcxZVEbmdBa2Y3dzdNUVltaUV5b3ZURFJrT3VWQXZR/view?usp=sharing
  [1]
https://drive.google.com/file/d/0Bz1h_hHcxZVEZEdNWVdlVjd6djhVOWljYVVHSlE4azVjVTRZ/view?usp=sharing
   [2]

Re: GSoC 2019 - Feedback request

[Mojca Miklavec]

Dear Amir,

Nice to see you at MacPorts!

You should send your email to the macports development mailing list
and you would get the answer there. (I'm forwarding it to the mailing
list now, but you should subscribe for the future.)

On Sat, 6 Apr 2019 at 20:14, AmirAli Mashayekhi wrote:
>
> Dear Macports GSoC admins,
>
> I have uploaded my proposal draft on GSoC webpage. Unfortunately we at 
> EPITA(mon École) got informed late about this years GSoC opportunity. But 
> this doesn’t affect my true excitement about  participation, open source and 
> contributing on them.

Late arrival of proposals is not a problem per se, it just means that
you have less time to improve your proposal and demonstrate your
skills compared to students who started early, so you need to excel
more to catch up.

My friendly suggestion would be to not try to write three proposals in
two days, as that will more likely decrease your chances of getting
selected, rather than increase them. Not because students applying to
other orgs would sound less serious, but because you hardly have
sufficient time to write one good proposal, let alone three.

> I appreciate if there is any feedbacks so I can enhance it before finalising 
> my application.

You nicely answered the auxiliary questions (meant to help us
understand the student better), but the project proposal is completely
missing.

The proposal fails to show any effort to research and understand the
task. The description / plan is so vague that you could in theory
submit a three-line patch to an arbitrary outdated port (for the whole
summer) and claim that you have delivered what you promised. Not even
a single piece of software was named as an example of what you would
work with (after six years of using MacPorts you should at least be
able to name one single piece of software that you wanted to use, but
was too old or missing).

In any case we are asking all of our candidate students to create some
pull requests or small demo projects demonstrating their
problem-solving skills. This would be the perfect opportunity to
update & improve a couple of important outdated unmaintained ports of
your interest.

> My draft should be available on your dashboard by now, but I shared a link 
> here in case of comments.
> My draft link : 
> https://docs.google.com/document/d/1RUmFN7PSPqbGzH4h8Hqq-qkTFgS3XJ_bIrsq7GhikYE/edit?usp=sharing
>
> Best Regards,
> Amir

Please note that this was meant as a constructive feedback.

Mojca

GSoC 2019 - trace mode improvements

[Davide Gerhard via macports-dev]

Sorry, I am late but I saw only the other day that MacPorts
participate to the Google Summer of Code.

Any comment will be appreciate.


* Personal Details

Name: Davide Gerhard
Email: rain...@irh.it
IRC: ra1nb0w at freenode
Github: @ra1nb0w
University: University of Trento (Italy)
Location: Italy
Short Bio:

I have always been interested in Informatics, Security and Networking
in particularly developed by the BSD/free-software community.
This passion has lead me to study information security.
I took a Bachelor degree in Security of Computer Systems and Networks
and now I am taking a Master degree on Computer Science.

In the last few months I contributed to macports-ports, mainly in the
following areas:

- SDR, mainly with @michaelld and @mf2k
- mercurial and related, mainly with @mojca


* Project Idea

** Goal

The main goal of my GSoC will be to:

- implement a function that auto-detect build dependencies of a new port
  and print out which are needed; especially useful when the developer
  is using his everyday installation.

- speed up trace mode: improve performance of the sandbox particularly
  during the build process, with so many syscalls and I/O.

** Abstract (with) Methodology

The main argument of my GSoC will be understanding and improving the
trace mode used on macports and learn the related low level
functionality of macOS.  This will be done mainly in three steps: first
understand how it works and which functionality already implements and
how. Second, trace which files a configuration script read(2)/stat(2) or
try to access, like during automake/autoconfig or cmake phase, and build
a complete and minimal dependency tree that will be shown to the
developer. In this stage, should be pay attention to present only direct
build dependencies and avoiding which one are non pertinent to macOS
configuration. Third, analyze the trace mode with dynamic tracing tools,
like dtrace and flame graph, to identify which functions are slow down
the process; identify which ones are more important or more slow and
implement a solution. Re-iterate this path until the execution is
acceptable for the normal usage. At the end of the process I expect a
dylib that could be used easily and fast from many other parts, like
permit fakeroot or phasing out XCode.

* Technical Details

** Schedule

At this stage, I generally don't trace a strict timetable because some
steps are more longer than others and I am not conscious of every single
effort.

Phase 1:

- deeply understand macports-base and its architecture;
- learn darwintracelib1.0 code and try to hack some functionality to
  better understand which section I should change.

Phase 2:

- implement the functions that trace the configuration phase and build
  an approximate dependency tree;
- try that code with many different ports and languages to detect
  glitches or cases not considered at the implementation time; verify
  manually that the build dependency tree is complete and minimal;
- review the implementation: adding new changes, if they are little, or
  try new approaches;
- do many tests with real packages and verify that everything is
  correct.

Phase 3:

- after understanding the tracelib, I can start to analyze the
  performance of the injected library; as I already know dtrace and how
  to plot the results as flame graph, this will be my first way to
  understand which function/call is slow. Considering that many low
  level things of macOS are unknown to me, I will be happy to find new
  way to analyze problem like this;
- define which functions could be speed-up and which kind of
  algorithm/structure could be used to improve the performance;
  generally these changes require caches or new approaches to the
  problem;
- implement one improvement at time and create some border line tests to
  verify that the new code works; after this, identify few packages
  that use that function a lot and verify the improvement with the tool
  used during the analysis;
- reiterate the above statement for every function that was source of
  bottleneck until acceptable performance is reached; this aspect must
  be considered reached only with real ports and not with test cases;
- test if the changes works well with the goal of phase 2 and see if
  need changes to generalize the usage, like fakeroot or phasing out
  XCode.


* Additional Questions

** What are your experiences with macOS so far? (How long do you use
   it, did you switch from Windows/Linux, etc.)

I used for fifteen years only OpenBSD/FreeBSD/Linux (Gentoo, Archlinux
and Debian).  Now it is 9 months that I am using macOS as main laptop.

** How long have you been using MacPorts?

7 months

** Do you have experience with other package management systems?

Yes

** How much experience do you have with Tcl and C?

Many years of C but very little with Tcl

** Will you be available after the project ends?

Yes


* Availability

** Do you plan to go on vacations, have exams, internship or be
   otherwise

Re: GSoC 2019 [Buildbot ideas]

[Mojca Miklavec]

Dear Rajdeep,

On Sat, 6 Apr 2019 at 13:33, Rajdeep Bharati wrote:
>
> I was wondering what kind of features you would like to have in the new 
> waterfall view?

I don't remember whether I ever explicitly said I wanted some new
features there :)

Probably the only thing I really miss is the portname explicitly seen
without having to click (or mouse-over) the rectangle.

Way less important:
- in case the build failed, it would be nice to see which step (first)
failed, again without having to move the mouse around
- in case the build is still running, see which step is running (I
would also say "time since start" or "estimated time to finish", but I
don't want a ticking timebomb that keeps changing every second, so you
may safely ignore this)

Generally the old waterfall had all the necessary info displayed. The
new one is more compact (I would also say not too attractive design,
but I'm not someone to judge that as I'm not a designer), but is
basically lacking all the info. If one builds the same thing under the
same builder that might be fine, but we build something else each time
and without knowing what was built, those green and red squares are
basically useless.


Some more ideas for thinking. One of the most desperately needed
features to make buildbot views usable for anything else than "what's
currently being built" would be to sort (filter) by port name.

Now, port name is something specific to MacPorts, but we could
probably introduce filtering by any kind of keywords. We could write
to our master.cfg that we want to create a new filter with name "port"
and then each port would get a keyword matching its name. Then I could
ask buildbot to display me history of all builds of port "clang-7.0".
But then a few months later we might realize that we want to display
all the failed builds from a particular maintainer. Maintainer is
again something specific to MacPorts, but we could simply introduce a
new category "maintainer" on the fly and use the github handles as
keywords to search for, and I could ask buildbot to display me all
broken builds belonging to maintainer "mojca" without any changes in
the buildbot or the view itself, just by slight modification in
master.cfg. One year later we could add an arbitrary new category that
we have never even thought of, and be able to filter according to that
one (maybe: show me all builds of python or perl modules; show me all
builds with non-free licence; show me all builds from ports fetching
from git, ...). Those categories could potentially be nested.

Mojca

Re: GSoC Proposal

[Mojca Miklavec]

On Sat, 6 Apr 2019 at 04:06, Cyril Roelandt wrote:
>
> Upt gives you three kinds of dependencies: build, runtime and test. This
> should be done in the backend.

MacPorts also splits "runtime" dependencies into two types. One are
proper runtime dependencies (not needed for building the package
itself), and others are needed all the time (say, libpng): both during
build and while running the software.

This allows optimisation to the extent that the package builder
doesn't really need to install that dependency when creating the
binaries.

> > 5) please add "supported_archs noarch” after “platforms"; since there 
> > are no architecture dependent files installed (as is often the case for 
> > Python scripts)
>
> This could probably be added to the backend as well!

However it might be non-trivial to figure it out whether a python
module needs to build some sources or not.
Figuring this out might be a standalone task on its own (I'm not
saying that this would take the full summer to implement, but it's
certainly not five minutes work). The question there is whether the
installed files depend on architecture (x86_64 vs. i386 etc.) or if a
package is installing just plain text files.

Mojca

Re: GSoC Proposal

[KARAN SHETH via macports-dev]

Hey,

> Check your dependencies; this should only happen if your port does not
> depend on the port that provides this file, i.e. python37. You might be
> overwriting the dependency by setting depends_lib rather than appending
> to it.

Changing from this :

if {${name} ne ${subport}} {
depends_build-append \
port:py${python.version}-setuptools

depends_lib-append  port:py${python.version}-spdx

livecheck.type  none
}


to this:

depends_build-append \
port:py${python.version}-setuptools

depends_lib-append  port:py${python.version}-spdx

livecheck.type  none


solved the issue.

I have submitted a PR[1] for spdx, spdx-lookup, upt, upt-pypi,
upt-cpan and upt-macports.

Please review the same.

I would also like to ask if anything else needs to be added or updated
in the proposal as the deadline is nearing.

Thanks,
Karan Sheth

[1] https://github.com/macports/macports-ports/pull/4032

-- 

           
      

Re: GSoC 2019 [Buildbot ideas]

[Rajdeep Bharati]

All right, thank you so much (sorry for the late reply).

I was wondering what kind of features you would like to have in the new
waterfall view?

Rajdeep

On Tue, Apr 2, 2019 at 7:46 PM Mojca Miklavec  wrote:

> On Tue, 2 Apr 2019 at 16:01, Pierre Tardy wrote:
> >
> > Not sure if this is applicable, but there is no import method from
> > buildbot 0.8.x to buildbot 0.9+
>
> I don't believe we really need that.
>
> It would be nice to "export" all the relevant (summary) data once
> before decommissioning the old master (or rather import it from an
> external script using the existing json api and store it into an
> independent database, for example like shown on this demo page:
> https://frozen-falls-98471.herokuapp.com/ports/SoapyAirspy/), but I
> don't have absolutely any expectations about keeping the old data
> around in the new interface. You already loose all the data even by
> trivial actions such as renaming the builder, so ...
>
> (An expected downside is that we sometimes link to buildbot logs from
> our bug tracker, but we keep loosing those logs all the time already
> since the buildbot master only keeps the last 10k builds.)
>
> Maybe I didn't understand what Rajdeep was asking. If the "old
> database" referred to our existing 0.8 setup: no, we don't have any
> expectations to keep that one around after the upgrade.
>
> Mojca
>