Re: [macports-mgr] Question about MacPorts project membership application

[Ryan Schmidt]

On May 14, 2018, at 00:46, Joshua Root wrote:

> On 2018-5-14 15:31 , Ryan Schmidt wrote:
>> 
>> On May 13, 2018, at 23:55, Joshua Root wrote:
>> 
>>> On 2018-5-14 14:30 , Ryan Schmidt wrote:
 
 On May 13, 2018, at 22:39, Joshua Root wrote:
 
> Perhaps by recommending the use
> of return receipts
 
 Email has no such feature.
>>> 
>>> And yet, you know what I meant.
>> 
>> Well no, I don't. "Return receipt" to me means that the sender receives 
>> automatic notification that their message was received by the recipient. 
>> Standard email systems do not have this capability, so what are you 
>> suggesting?
> 
> Sending a Disposition-Notification-To: header. Which is labelled "Return
> Receipt" in the UI for many MUAs.

Ah. I've never heard of that before.

Re: [macports-mgr] Question about MacPorts project membership application

[Joshua Root]

On 2018-5-14 15:31 , Ryan Schmidt wrote:
> 
> On May 13, 2018, at 23:55, Joshua Root wrote:
> 
>> On 2018-5-14 14:30 , Ryan Schmidt wrote:
>>>
>>> On May 13, 2018, at 22:39, Joshua Root wrote:
>>>
 Perhaps by recommending the use
 of return receipts
>>>
>>> Email has no such feature.
>>
>> And yet, you know what I meant.
> 
> Well no, I don't. "Return receipt" to me means that the sender receives 
> automatic notification that their message was received by the recipient. 
> Standard email systems do not have this capability, so what are you 
> suggesting?

Sending a Disposition-Notification-To: header. Which is labelled "Return
Receipt" in the UI for many MUAs.

- JOsh

Re: Gsoc 18 Project | Collect build statistics

[Mojca Miklavec]

Dear Vishnu,

On 13 May 2018 at 09:37, Vishnu wrote:
> Hey
>
> Mojca you said once that you have lot to comment on the current database.
> So it would be great if you could do that.

I left a comment on
https://github.com/macports/macports-webapp/issues/2
and made a bunch of changes and suggestions to

https://github.com/macports/macports-webapp/blob/master/docs/Database_Design.md
(I would still like to go through installation statistics and build
statistics in slightly more detail.)

Since the idea was to spend the whole week heavily concentrated on the
database design and implementation (which also includes planning,
documenting, changing, testing etc.), my suggestion would be to do the
following (but feel free to adapt it):

(0) Given the limitation in heroku, what you can do is take
approximately 1000 ports for the initial tests. A quick trick could be
to loop through the file with something like "while
lowercase(first_letter_of_portname) < 'f'". No need to spend any extra
time of this, just a simple suggestion for a quick workaround. You can
of course also cut the portindex file manually for now.

(1) So far you have implemented one table with portindex. Before you
end up implementing the full database scheme from the ground up, you
can probably try the first and the easiest many-to-many relationship
"categories". Change the current app to add a new table "categories"
and while you are reading the database of ports:
- create or update categories as ports list them
- create entries in port_category
- the port page should list all categories to which this port belongs
- these categories should be hyperlinks to a page listing all ports
belonging to that particular category
so basically something similar to what https://www.macports.org/ports.php does.

(2) I would strongly suggest to try to figure out how to do unit tests
and start writing some tests. A random URL:
https://realpython.com/testing-in-django-part-1-best-practices-and-examples/
but don't necessarily stick to this URL/suggestion, it's just one of
the first few hits, you might find better resources elsewhere.

What would be nice to test at this stage is for example the following scenario:
- prepare a simple, stripped-down version of portindex.json (maybe
just 5 ports or so)
- import them to a clean database (you should have a different
database for running unit tests than "production" code)
- check that you have the correct number of entries in all tables,
check for some values that are potentially tricky to get right
- prepare another simple portindex.json with one new port, one deleted
port, one updated port, apply that one and make sure that you are
still getting the expected number of entries in all tables, and that
what you are getting is still correct

(3) I didn't check the recent changes in details yet, but you should
make sure that the application keeps working properly when you read
portindex.json multiple times. Last time the database would create a
new entry for the same port. If that port exists, you should update
the entry instead of creating a new one (there's a function
create_or_update, I think).

Of course feel free to continue implementing tables from the design
document, but instead of rushing into creation of all tables at once,
it make most sense to test them incrementally and have a good coverage
with unit tests to avoid breaking stuff as you keep developing.

Since you started working on portindex anyway, it's probably a lot
more important to have a well implemented and tested this part of
database (+ of course having the database design document ready for
further steps) than to rush into setting up all the tables at once.


See also
https://github.com/umeshksingla/macports-stats
which has been created during / immediately after the MacPorts meeting
in Slovenia two months ago as we were brainstorming on this idea. It's
just a quick proof-of-concept repository, but you might find some
ideas there.

Mojca

Re: [macports-mgr] Question about MacPorts project membership application

[Ryan Schmidt]

On May 13, 2018, at 23:55, Joshua Root wrote:

> On 2018-5-14 14:30 , Ryan Schmidt wrote:
>> 
>> On May 13, 2018, at 22:39, Joshua Root wrote:
>> 
>>> Perhaps by recommending the use
>>> of return receipts
>> 
>> Email has no such feature.
> 
> And yet, you know what I meant.

Well no, I don't. "Return receipt" to me means that the sender receives 
automatic notification that their message was received by the recipient. 
Standard email systems do not have this capability, so what are you suggesting?

Re: [macports-mgr] Question about MacPorts project membership application

[Joshua Root]

On 2018-5-14 14:30 , Ryan Schmidt wrote:
> 
> On May 13, 2018, at 22:39, Joshua Root wrote:
> 
>> Perhaps by recommending the use
>> of return receipts
> 
> Email has no such feature.

And yet, you know what I meant.

Re: [macports-mgr] Question about MacPorts project membership application

[Ryan Schmidt]

On May 13, 2018, at 22:39, Joshua Root wrote:

> Perhaps by recommending the use
> of return receipts

Email has no such feature.

Re: [macports-mgr] Question about MacPorts project membership application

[Joshua Root]

On 2018-5-14 07:03 , Mojca Miklavec wrote:
> On 13 May 2018 at 21:32, Rainer Müller wrote:
>> On 2018-05-13 18:30, Mojca Miklavec wrote:
>>> The portmgr team might need some time to check the contributions and
>>> verify candidate's suitability for commit rights, but maybe they
>>> forgot to send you a preliminary confirmation of receiving your
>>> request and were simply waiting for the votes to arrive.
>>
>> We never did anything like that... Why would we expected to confirm the
>> reception of every single mail?
> 
> Probably not confirming reception of every single email, but I believe
> it makes an enormous difference to the user asking for commit rights
> if he gets a simple, totally generic email saying something like
> "Thank you for contacting us and for volunteering to help us shape our
> package manager into an even better product. Please be patient while
> we are evaluating your recent contributions and don't hesitate to ping
> us again in case you don't get an answer within two weeks." ... as
> compared to not even knowing whether the email was sent to the correct
> email address and not lost on the way due to spam filters.

This sounds reasonable. It's much less work to reply with some
boilerplate than to evaluate the applicant's contributions. As Rainer
said, this has not been part of the process to date. I fully understand
the concerns about not having enough time as it is, but it's important
to communicate well with contributors.

I wonder if we could even automate it. Perhaps by recommending the use
of return receipts, or a bot that can recognise requests for commit access?

The idea of having a secretary is an interesting one too, though I
wonder if we have anyone who would volunteer for the job.

- Josh

Re: [macports-mgr] Question about MacPorts project membership application

[Jan Stary]

On May 13 23:38:35, rai...@macports.org wrote:
> On 2018-05-13 23:03, Mojca Miklavec wrote:
> > On 13 May 2018 at 21:32, Rainer Müller wrote:
> >> On 2018-05-13 18:30, Mojca Miklavec wrote:
> >>> The portmgr team might need some time to check the contributions and
> >>> verify candidate's suitability for commit rights, but maybe they
> >>> forgot to send you a preliminary confirmation of receiving your
> >>> request and were simply waiting for the votes to arrive.
> >>
> >> We never did anything like that... Why would we expected to confirm the
> >> reception of every single mail?
> > 
> > Probably not confirming reception of every single email, but I believe
> > it makes an enormous difference to the user asking for commit rights
> > if he gets a simple, totally generic email saying something like
> > "Thank you for contacting us and for volunteering to help us shape our
> > package manager into an even better product. Please be patient while
> > we are evaluating your recent contributions and don't hesitate to ping
> > us again in case you don't get an answer within two weeks." ... as
> > compared to not even knowing whether the email was sent to the correct
> > email address and not lost on the way due to spam filters.

"Dear valued commiter" is what this is missing.

Re: [macports-mgr] Question about MacPorts project membership application

[Mojca Miklavec]

On 13 May 2018 at 21:32, Rainer Müller wrote:
> On 2018-05-13 18:30, Mojca Miklavec wrote:
>> The portmgr team might need some time to check the contributions and
>> verify candidate's suitability for commit rights, but maybe they
>> forgot to send you a preliminary confirmation of receiving your
>> request and were simply waiting for the votes to arrive.
>
> We never did anything like that... Why would we expected to confirm the
> reception of every single mail?

Probably not confirming reception of every single email, but I believe
it makes an enormous difference to the user asking for commit rights
if he gets a simple, totally generic email saying something like
"Thank you for contacting us and for volunteering to help us shape our
package manager into an even better product. Please be patient while
we are evaluating your recent contributions and don't hesitate to ping
us again in case you don't get an answer within two weeks." ... as
compared to not even knowing whether the email was sent to the correct
email address and not lost on the way due to spam filters.

Mojca

Re: [macports-mgr] Question about MacPorts project membership application

[Rainer Müller]

On 2018-05-13 18:30, Mojca Miklavec wrote:
> The portmgr team might need some time to check the contributions and
> verify candidate's suitability for commit rights, but maybe they
> forgot to send you a preliminary confirmation of receiving your
> request and were simply waiting for the votes to arrive.

We never did anything like that... Why would we expected to confirm the
reception of every single mail?

Rainer

Re: Question about MacPorts project membership application

[Eitan Adler]

On 13 May 2018 at 09:30, Mojca Miklavec  wrote:
> Dear Chih-Hsuan Yen,
>
> On 13 May 2018 at 16:37, Chih-Hsuan Yen wrote:
>> Hello everyone,
>>
>> Perry Metzger suggests me to apply for MacPorts project membership on two of
>> my pull requests [1][2], so I wrote a letter "Commit access: Chih-Hsuan Yen"
>> to macports-...@lists.macports.org 5 days ago, following tips listed on the
>> official guide [3]. However, there has not been any reply since then. Am I
>> doing things right?
>
> I don't have access to emails sent to portmgr, but this ticket might
> be relevant (since very recently):
> https://trac.macports.org/ticket/56052
>
> The portmgr team might need some time to check the contributions and
> verify candidate's suitability for commit rights, but maybe they
> forgot to send you a preliminary confirmation of receiving your
> request and were simply waiting for the votes to arrive.

The timeline for my request was as follows.

2017-12-16 - Initial request
2017-12-21 - ping
2018-01-02 - ping
2018-01-07 - ping
2018-01-08 - first response / confirmation of receipt
2018-01-29 - acceptance (woot!)

Note that it was during the new-years so it might be slower than
typical. This is reconstructed from my email so apologies to the team
if I missed anything.

If there is one thing I wish that would change is a confirmation of
receipt earlier on. In the FreeBSD project we solve this issue by
having a dedicated (typically non-voting) secretary.


-- 
Eitan Adler

Re: Question about MacPorts project membership application

[Mojca Miklavec]

Dear Chih-Hsuan Yen,

On 13 May 2018 at 16:37, Chih-Hsuan Yen wrote:
> Hello everyone,
>
> Perry Metzger suggests me to apply for MacPorts project membership on two of
> my pull requests [1][2], so I wrote a letter "Commit access: Chih-Hsuan Yen"
> to macports-...@lists.macports.org 5 days ago, following tips listed on the
> official guide [3]. However, there has not been any reply since then. Am I
> doing things right?

I don't have access to emails sent to portmgr, but this ticket might
be relevant (since very recently):
https://trac.macports.org/ticket/56052

The portmgr team might need some time to check the contributions and
verify candidate's suitability for commit rights, but maybe they
forgot to send you a preliminary confirmation of receiving your
request and were simply waiting for the votes to arrive.

Mojca

Question about MacPorts project membership application

[Chih-Hsuan Yen]

Hello everyone,


Perry Metzger suggests me to apply for MacPorts project membership on
two of my pull requests [1][2], so I wrote a letter "Commit access:
Chih-Hsuan Yen" to macports-...@lists.macports.org 5 days ago, following
tips listed on the official guide [3]. However, there has not been any
reply since then. Am I doing things right?


Thanks,


Chih-Hsuan Yen


[1]
https://github.com/macports/macports-ports/pull/1749#issuecomment-387110414

[2]
https://github.com/macports/macports-ports/pull/1771#issuecomment-388044421

[3] https://guide.macports.org/#project.membership



signature.asc
Description: OpenPGP digital signature

Re: Gsoc 18 Project | Collect build statistics

[Vishnu]

Hey

Also can i know the status of your getting access to your VM ?

Thanks

On 13 May 2018 at 13:07, Vishnu  wrote:

> Hey
>
> Mojca you said once that you have lot to comment on the current database.
> So it would be great if you could do that.
> As today is the last day of Community bonding period.
> From tommorrow coding starts.
>
> Thanks
>
> On 12 May 2018 at 23:14, Mojca Miklavec  wrote:
>
>> Dear Vishnu,
>>
>> Sorry for being slightly out-of-sync (sending emails I wrote quite a
>> while ago and in the meantime no longer being up-to-date with other
>> responses).
>>
>> On 12 May 2018 at 17:26, Vishnu wrote:
>> > Hi
>> >
>> > "The PortIndex file itself is generated with the 'portindex' command,
>> > which only updates the data for Portfiles that have a modification time
>> > that is newer than the existing PortIndex file."
>> >
>> > So where can i see that code that checks the modification time?
>>
>> https://github.com/macports/macports-base/blob/master/src/po
>> rt/portindex.tcl
>>
>> > So suppose
>> > Our ports tree contains three ports A, B, C last modified time : 1:00
>> UTC
>> > present port index:
>> > [ A
>> >   B
>> >   C
>> > ] @ 1:00 UTC
>> >
>> > There was a edit in Port A @ 1:30 UTC
>> > so something will check all the port files modification time and if
>> > relatively newer it will update the port index.
>> >
>> > so new  port index:
>> > [ A
>> >   B
>> >   C
>> > ] @ 1:30 UTC
>> >
>> >
>> > "The portindex2postgres.sql script merely converts the PortIndex file
>> > from the custom format based on Tcl lists to SQL. The output will always
>> > contain SQL statements with the full data for every port."
>> >
>> > Then after the port index is updated this script runs and flushes the db
>> > then fills it with new portindex data?
>> > or just updates the db with the modified port data?
>>
>> I don't know the answer without looking at it first (you can check
>> what the code does yourself), but from the perspective of your app
>> development this is irrelevant.
>>
>> > So after every commit the entire 15mb portindex being processed and data
>> > being uploaded to the db.
>> > I think this is very inefficient.
>> > rather it would be best if after every commit the buildbot/ webhook
>> updates
>> > the database itself.
>>
>> I totally agree that it might make a lot more sense to submit just
>> differential data, but my suggestion would be to first implement the
>> functionality and optimise it at the end if there will be time. No
>> user will suffer from suboptimal data transfer (even if that database
>> only gets updated once per day, it will still be perfectly
>> acceptable), while nobody will benefit from spending a lot of time
>> figuring out how to properly implement a super efficient solution and
>> then running out of time to actually finish the app.
>>
>> Let's just work with what we have for now.
>>
>> Mojca
>>
>
>

Re: Gsoc 18 Project | Collect build statistics

[Vishnu]

Hey

Mojca you said once that you have lot to comment on the current database.
So it would be great if you could do that.
As today is the last day of Community bonding period.
>From tommorrow coding starts.

Thanks

On 12 May 2018 at 23:14, Mojca Miklavec  wrote:

> Dear Vishnu,
>
> Sorry for being slightly out-of-sync (sending emails I wrote quite a
> while ago and in the meantime no longer being up-to-date with other
> responses).
>
> On 12 May 2018 at 17:26, Vishnu wrote:
> > Hi
> >
> > "The PortIndex file itself is generated with the 'portindex' command,
> > which only updates the data for Portfiles that have a modification time
> > that is newer than the existing PortIndex file."
> >
> > So where can i see that code that checks the modification time?
>
> https://github.com/macports/macports-base/blob/master/src/
> port/portindex.tcl
>
> > So suppose
> > Our ports tree contains three ports A, B, C last modified time : 1:00 UTC
> > present port index:
> > [ A
> >   B
> >   C
> > ] @ 1:00 UTC
> >
> > There was a edit in Port A @ 1:30 UTC
> > so something will check all the port files modification time and if
> > relatively newer it will update the port index.
> >
> > so new  port index:
> > [ A
> >   B
> >   C
> > ] @ 1:30 UTC
> >
> >
> > "The portindex2postgres.sql script merely converts the PortIndex file
> > from the custom format based on Tcl lists to SQL. The output will always
> > contain SQL statements with the full data for every port."
> >
> > Then after the port index is updated this script runs and flushes the db
> > then fills it with new portindex data?
> > or just updates the db with the modified port data?
>
> I don't know the answer without looking at it first (you can check
> what the code does yourself), but from the perspective of your app
> development this is irrelevant.
>
> > So after every commit the entire 15mb portindex being processed and data
> > being uploaded to the db.
> > I think this is very inefficient.
> > rather it would be best if after every commit the buildbot/ webhook
> updates
> > the database itself.
>
> I totally agree that it might make a lot more sense to submit just
> differential data, but my suggestion would be to first implement the
> functionality and optimise it at the end if there will be time. No
> user will suffer from suboptimal data transfer (even if that database
> only gets updated once per day, it will still be perfectly
> acceptable), while nobody will benefit from spending a lot of time
> figuring out how to properly implement a super efficient solution and
> then running out of time to actually finish the app.
>
> Let's just work with what we have for now.
>
> Mojca
>