Re: GitHub migration complete

[Marko Käning]

On 01 Nov 2016, at 04:00 , Ryan Schmidt  wrote:
> We suggest that you move your user repository to your own GitHub account 
> where you can continue to use it as you see fit. Instructions for how to move 
> it are forthcoming. You should not use the Fork button to do so.

Can I simply remove it?
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Ryan Schmidt]

If you had a personal directory in the users directory of the Subversion 
repository, that has now been converted to a separate git repository in 
https://github.com/macports with a name starting with "macports-user-".

We suggest that you move your user repository to your own GitHub account where 
you can continue to use it as you see fit. Instructions for how to move it are 
forthcoming. You should not use the Fork button to do so.



___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Marko Käning]

On 31 Oct 2016, at 21:14 , Lawrence Velázquez  wrote:

>> On Oct 31, 2016, at 5:23 AM, Marko Käning  wrote:
>> 
>> a post-commit-hook checking whether the GitHub pull request ID #123
>> actually exists for the main repository seems like a valuable feature,
>> especially in the transition phase. Shall I file a ticket on trac for it?
> 
> Sure, if you like. Feel free to assign me as owner; I'll try to look into it 
> this week.

Thanks, done in https://trac.macports.org/ticket/52763#ticket .
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 31, 2016, at 5:23 AM, Marko Käning  wrote:
> 
> a post-commit-hook checking whether the GitHub pull request ID #123
> actually exists for the main repository seems like a valuable feature,
> especially in the transition phase. Shall I file a ticket on trac for it?

Sure, if you like. Feel free to assign me as owner; I'll try to look into it 
this week.

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 31, 2016, at 1:16 PM, Thibaut Paumard  wrote:
> 
> Le 31/10/2016 à 17:23, Lawrence Velázquez a écrit :
>>> On Oct 31, 2016, at 12:16 PM, Thibaut Paumard  wrote:
>>> 
 Le 31/10/2016 à 17:01, René J.V. Bertin a écrit :
> On Monday October 31 2016 10:00:05 Ryan Schmidt wrote:
> 
> This issue only affects the very small percentage of the MacPorts user 
> population (including developers and maintainers) that clones the git 
> repository. Most users will use the rsync server, on which we do generate 
> portindexes for each macOS version.
 
 Of course, but note how I used the word collective, which was supposed to 
 include the idea that portindex has to be run each time by every user. :)
 
>>> 
>>> I would actually believe the number of affected users should be between
>>> very small and zero.
>> 
>> Pretty much.
>> 
>> In any case, between the capabilities of Git itself and the facilities 
>> GitHub provides, I do not believe it is possible to do what you suggest.
> 
> You mean what René is suggesting?

Yes.

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Thibaut Paumard]

Le 31/10/2016 à 17:23, Lawrence Velázquez a écrit :
>> On Oct 31, 2016, at 12:16 PM, Thibaut Paumard  wrote:
>>
>>> Le 31/10/2016 à 17:01, René J.V. Bertin a écrit :
 On Monday October 31 2016 10:00:05 Ryan Schmidt wrote:

 This issue only affects the very small percentage of the MacPorts user 
 population (including developers and maintainers) that clones the git 
 repository. Most users will use the rsync server, on which we do generate 
 portindexes for each macOS version.
>>>
>>> Of course, but note how I used the word collective, which was supposed to 
>>> include the idea that portindex has to be run each time by every user. :)
>>>
>>
>> I would actually believe the number of affected users should be between
>> very small and zero.
> 
> Pretty much.
> 
> In any case, between the capabilities of Git itself and the facilities GitHub 
> provides, I do not believe it is possible to do what you suggest.

You mean what René is suggesting?

What *I* am suggesting (and that is not in the text you quoted) has very
little to do with git. Running portindex somewhere after pulling from
git is easily done with a post-merge hook, but that's barely relevant.

Regards, Thibaut.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 31, 2016, at 12:16 PM, Thibaut Paumard  wrote:
> 
>> Le 31/10/2016 à 17:01, René J.V. Bertin a écrit :
>>> On Monday October 31 2016 10:00:05 Ryan Schmidt wrote:
>>> 
>>> This issue only affects the very small percentage of the MacPorts user 
>>> population (including developers and maintainers) that clones the git 
>>> repository. Most users will use the rsync server, on which we do generate 
>>> portindexes for each macOS version.
>> 
>> Of course, but note how I used the word collective, which was supposed to 
>> include the idea that portindex has to be run each time by every user. :)
>> 
> 
> I would actually believe the number of affected users should be between
> very small and zero.

Pretty much.

In any case, between the capabilities of Git itself and the facilities GitHub 
provides, I do not believe it is possible to do what you suggest.

vq
Sent from my iPhone
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Thibaut Paumard]

Le 31/10/2016 à 17:01, René J.V. Bertin a écrit :
> On Monday October 31 2016 10:00:05 Ryan Schmidt wrote:
> 
>> This issue only affects the very small percentage of the MacPorts user 
>> population (including developers and maintainers) that clones the git 
>> repository. Most users will use the rsync server, on which we do generate 
>> portindexes for each macOS version.
> 
> Of course, but note how I used the word collective, which was supposed to 
> include the idea that portindex has to be run each time by every user. :)
> 

Hi,

I would actually believe the number of affected users should be between
very small and zero.

I personally never run portindex on the full port tree: I have a very
small development tree in which I keep symlinks only to the ports that I
am currently developing, and I run portindex on this tiny subset. For
the rest, I trust rsync. I'm wondering whether I got the hint from some
macports cookbook, but I think not.

Such a workflow should work for most maintainers, and should scale
fairly well also for very active developers.

Kind regards, Thibaut.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[René J . V . Bertin]

On Monday October 31 2016 10:00:05 Ryan Schmidt wrote:

> This issue only affects the very small percentage of the MacPorts user 
> population (including developers and maintainers) that clones the git 
> repository. Most users will use the rsync server, on which we do generate 
> portindexes for each macOS version.

Of course, but note how I used the word collective, which was supposed to 
include the idea that portindex has to be run each time by every user. :)

R.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Ryan Schmidt]

> On Oct 31, 2016, at 4:18 AM, René J. V. Bertin  wrote:
> 
> Clemens Lang wrote:
> 
>> If your question is not yet answered, ask on the mailing lists so it can
>> be added.
> 
> I may have overlooked this, but does github have any provisions that would 
> allow 
> the PortIndex files to be generated on the server and served with the actual 
> repo 
> contents? That would probably give a very significant reduction in the 
> resources 
> spent collectively to regenerate those files...

This issue only affects the very small percentage of the MacPorts user 
population (including developers and maintainers) that clones the git 
repository. Most users will use the rsync server, on which we do generate 
portindexes for each macOS version.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 31, 2016, at 7:12 AM, René J.V. Bertin  wrote:
> 
>> On Monday October 31 2016 11:52:28 Rainer Müller wrote:
>> 
>> rsync -vt 
>> rsync://rsync.macports.org/macports/release/ports/PortIndex_darwin_16_i386/PortIndex*
>>  $ports
> 
> Thanks for the suggestion, I might do that.
> 
> (Are you running 10.12 on a 32bit host? Is that even possible?)

The portindices are based on platform, not architecture. There are
indices for Intel and (for Tiger and Leopard) PowerPC.

-

% rsync 'rsync://rsync.macports.org/macports/release/ports/PortIndex*'

Willkommen auf dem RSYNC-server auf ftp.fau.de.
Nicht all unsere Mirror sind per rsync verfuegbar.

Welcome to the RSYNC daemon on ftp.fau.de.
Not all of our mirrors are available through rsync.


-rw-r--r-- 11,985,357 2016/10/31 08:31:49 PortIndex
-rw-r--r--509,450 2016/10/31 08:31:49 PortIndex.quick
drwxr-xr-x  4,096 2016/10/31 09:01:42 PortIndex_darwin_10_i386
drwxr-xr-x  4,096 2016/10/31 09:01:45 PortIndex_darwin_11_i386
drwxr-xr-x  4,096 2016/10/31 09:01:48 PortIndex_darwin_12_i386
drwxr-xr-x  4,096 2016/10/31 09:01:50 PortIndex_darwin_13_i386
drwxr-xr-x  4,096 2016/10/31 09:01:53 PortIndex_darwin_14_i386
drwxr-xr-x  4,096 2016/10/31 09:01:55 PortIndex_darwin_15_i386
drwxr-xr-x  4,096 2016/10/31 09:01:57 PortIndex_darwin_16_i386
drwxr-xr-x  4,096 2016/10/31 09:01:35 PortIndex_darwin_8_i386
drwxr-xr-x  4,096 2016/10/31 09:01:32 PortIndex_darwin_8_powerpc
drwxr-xr-x  4,096 2016/10/31 09:01:40 PortIndex_darwin_9_i386
drwxr-xr-x  4,096 2016/10/31 09:01:37 PortIndex_darwin_9_powerpc

-

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[René J . V . Bertin]

On Monday October 31 2016 11:52:28 Rainer Müller wrote:

>Just as with Subversion.

Yeah, I wouldn't expect that the SVC type had any influence on this.

>  rsync -vt 
> rsync://rsync.macports.org/macports/release/ports/PortIndex_darwin_16_i386/PortIndex*
>  $ports

Thanks for the suggestion, I might do that.

(Are you running 10.12 on a 32bit host? Is that even possible?)

>Just make sure you use the correct OS version in the rsync path. There
>is also no guarantee that this really matches the git ports tree, which
>might already contain newer Portfile changes.

That should just mean more ports will be indexed, no?

A related question: what happens if you run portindex on a fresh clone on one 
host, and then rsync the whole tree to another host with a different OS 
version? I just did that transfer; using rsync with compression (-z) it took 
much less time (48s ...) than even simply checking out the tree from git on the 
other host. Just running portindex on the transferred tree without -f didn't do 
anything so apparently it doesn't check for OS version or platform mismatches?

R.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Rainer Müller]

On 2016-10-31 11:41, René J.V. Bertin wrote:
> Pity though, the first-run portindex of a fresh git clone just took about 5 
> quarters of an hour on one of my machines (a good 5s/port).

Just as with Subversion.

To speed it up, you could seed it with the latest generated version
from rsync:

  rsync -vt 
rsync://rsync.macports.org/macports/release/ports/PortIndex_darwin_16_i386/PortIndex*
 $ports

Just make sure you use the correct OS version in the rsync path. There
is also no guarantee that this really matches the git ports tree, which
might already contain newer Portfile changes.

Rainer

PS: On purpose not sending this to macports-users, as it should not be
used in the general case.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[René J . V . Bertin]

On Monday October 31 2016 10:49:55 Clemens Lang wrote:

Hi,

>Just as with Subversion, the answer is no. Remember that the PortIndex
>is specific to the macOS version you are running, so a server-generated

Ah, of course. I didn't actually know this but indeed port versions could be 
specific to OS version or platform even if no other specific information is 
stored.
Sorry for the noise.

Pity though, the first-run portindex of a fresh git clone just took about 5 
quarters of an hour on one of my machines (a good 5s/port).

>Additionally, git does not preserve timestamps from the repository on
>checkout, so you might actually end up re-generating the index locally
>anyway.

I think that wouldn't (or shouldn't) happen as the timestamp would be newer. 
And of course the auto-regeneration could be deactivated if the server always 
serves an up-to-date index.

R.
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Clemens Lang]

Hi,

On Mon, Oct 31, 2016 at 10:18:42AM +0100, René J.  V. Bertin wrote:
> I may have overlooked this, but does github have any provisions that
> would allow the PortIndex files to be generated on the server and
> served with the actual repo contents? That would probably give a very
> significant reduction in the resources spent collectively to
> regenerate those files...

Just as with Subversion, the answer is no. Remember that the PortIndex
is specific to the macOS version you are running, so a server-generated
PortIndex could only generate all of them into different files.

Additionally, git does not preserve timestamps from the repository on
checkout, so you might actually end up re-generating the index locally
anyway.

-- 
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Marko Käning]

Hi Larry,

On 31 Oct 2016, at 05:38 , Lawrence Velázquez  wrote:
> Old habits die hard, but from now on do NOT refer to Trac tickets as
> "#12345" in your commit messages; GitHub's website interprets those as
> pull request numbers. Copy and paste the full Trac URL instead.

a post-commit-hook checking whether the GitHub pull request ID #123
actually exists for the main repository seems like a valuable feature,
especially in the transition phase. Shall I file a ticket on trac for it?

Greets,
Marko
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[René J . V . Bertin]

Lawrence Velázquez wrote:

...
> $ git gc --aggressive

FWIW, while theoretically very space-efficient, git's .git directories tend to 
grow to considerable size for active repositories. I find it useful to run the 
attached script from time to time. It runs the garbage collector, but also 
(re)packs the remaining data.

R
#!/bin/sh
# NB: should support repos checked out with --separate-git-dir!
# those have .git as a file containing something like
# gitdir: /path/to/git-dir

if [ ! -d ./.git/ ] ;then
echo "Not a git repository (./.git/ is not a directory)"
exit 0
fi

# use a bit of a hack to determine if our stamp exists and is the newest entry 
in .git
# (using grep to filter out the . and .. entries; this is still faster than 
running the whole
# operation for nothing).
LASTFILE="`ls -1tra ./.git/ | grep -v '^\.[\.]*$' | tail -1`"
if [ "${LASTFILE}" = ".eradicated" ] ;then
echo "Nothing changed since last `basename $0` - skipping"
exit 0
fi

gfilter () {
echo git filter-branch -f  --index-filter "git rm --force --cached 
--ignore-unmatch \"$1\"" -- --all
git filter-branch -f  --index-filter "git rm --force --cached 
--ignore-unmatch \"$1\"" -- --all
}

for f in $@ ;do
gfilter "$f"
done

rm -Rf .git/refs/original && \
git reflog expire --expire=now --all && \
git gc --aggressive && \
git prune

date > .git/.eradicated
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Brandon Allbery]

On Mon, Oct 31, 2016 at 12:38 AM, Lawrence Velázquez 
wrote:

> Old habits die hard, but from now on do NOT refer to Trac tickets as
> "#12345" in your commit messages; GitHub's website interprets those as
> pull request numbers. Copy and paste the full Trac URL instead.
>

Sounds like a job for a commit-msg hook (sadly it can't rewrite them, just
see #xxx and suggest replacements).

-- 
brandon s allbery kf8nh   sine nomine associates
allber...@gmail.com  ballb...@sinenomine.net
unix, openafs, kerberos, infrastructure, xmonadhttp://sinenomine.net
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 30, 2016, at 9:54 PM, Clemens Lang  wrote:
> 
> MacPorts developers should now have commit access to the GitHub
> repositories.

A quick reminder about commit messages:

Old habits die hard, but from now on do NOT refer to Trac tickets as
"#12345" in your commit messages; GitHub's website interprets those as
pull request numbers. Copy and paste the full Trac URL instead.

vq

P.S. If you want to see something nifty, use this line in a commit that
resolves a ticket:

Closes: https://trac.macports.org/ticket/[NUMBER]
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Lawrence Velázquez]

> On Oct 30, 2016, at 9:54 PM, Clemens Lang  wrote:
> 
> Our Subversion repository has been split into several repositories on
> GitHub.

Please note that Ryan ran the svn2git conversion several times this
weekend, so any clones made previously will have nothing in common with
the final repositories, and a naïve "git pull" will try to produce
a merge commit. This is not desirable.

You should instead press the proverbial reset button. Assuming you made
a straightforward clone and only checked out a local master branch:

$ git fetch
$ git reset --hard origin/master
$ git gc --aggressive

This fetches the new history, forces your local master branch to match
ours, and garbage-collects the old objects.

vq
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: GitHub migration complete

[Ryan Schmidt]

> On Oct 30, 2016, at 10:28 PM, Carlo Tambuatco  wrote:
> 
> Is that mailto link macports-us...@lists.macosforge.org in the signature 
> still valid…?

Yes, our previous mailing lists have not moved yet. And even when they do, the 
old addresses will remain valid.

___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev