Re: [gentoo-user] git serving portage tree.
On 03/11/15 06:46, Michael Orlitzky wrote:
> On 11/02/2015 05:42 PM, Bill Kenworthy wrote:
>>>
>>
>> Thanks. Yes I have a few gentoo machines and around 12 gentoo VM's - I
>> currently use a rsync/httpreplicator to a master machine architecture
>> which could be replaced by git.
>>
Replace your rsync infrastructure with a git one. After set up, there
is faster syncing than rsync, both externally to the gentoo
infrastructure (reasonable ADSL link), and really fast compared to rsync
internally. Not simple, but its a one off hit on just a few machines
for me. Works for me, no warranty etc. :)
Thanks to Michael and Hasufell for the hints needed to get this working.
Apologies for the line wrap.
Replace with your own values!
repo name: olympus
repo master host: olympus.lan.localdomain
On the master:
Download https://github.com/hasufell/portage-gentoo-git-config
unzip
cp -v portage-gentoo-git-config-master/etc/portage/repo.postsync.d/*
/etc/portage/repo.postsync.d/
cp portage-gentoo-git-config-master/etc/portage/repos.conf/gentoo.conf
/etc/portage/repos.conf/
if needed: chmod +x /etc/portage/repo.postsync.d/sync_overlay_cache
cd /usr/
mv portage portage.old
emerge --sync
To use as the master for your own mirror system:
Uncomment the lines in /etc/rsyncd.conf to set up your own
gentoo-portage rsync server - still needed for now
rc-update add rsyncd
/etc/init.d/rsyncd start
On the internal clients wanting to use the mirror:
reccomended: set up passwordless ssh between clients and the master
emerge dev-vcs/git
create /etc/portage/repos.conf/olympus.conf as follows
[DEFAULT]
main-repo = olympus
[olympus]
location = /usr/portage
sync-type = git
sync-uri = git://olympus.lan.localdomain:/usr/portage
auto-sync = yes
create /etc/portage/repo.postsync.d/sync_olympus
#!/bin/bash
repository_name="${1}"
repository_path="${3}"
[[ ${repository_name} == "olympus" ]] || exit 0
source /lib/gentoo/functions.sh
# Number of jobs for egencache, default is number or processors.
parallel_jobs="$(nproc)"
# caches
ebegin "Fetching pre-generated metadata cache for ${repository_name}"
rsync -a
rsync://olympus.lan.localdomain/gentoo-portage/metadata/md5-cache/
"${repository_path}"/metadata/md5-cache/
eend $?
ebegin "Updating metadata cache for ${repository_name}"
egencache --jobs="${parallel_jobs}" --repo="${repository_name}"
--update --update-use-local-desc
eend $?
#DTD's
DTDDIR="${repository_path}"/metadata/dtd
ebegin "Updating DTDs"
if [[ -e ${DTDDIR} ]]; then
git -C "${DTDDIR}" pull -q --ff
else
git clone olympus.lan.localdomain:/usr/portage/metadata/dtd
"${DTDDIR}"
fi
eend "$?"
#GLSA's
GLSADIR="${repository_path}"/metadata/glsa
ebegin "Updating GLSAs"
if [[ -e ${GLSADIR} ]]; then
git -C "${GLSADIR}" pull -q --ff
else
git clone olympus.lan.localdomain:/usr/portage/metadata/glsa
"${GLSADIR}"
fi
eend "$?"
# herds
ebegin "Updating herds.xml"
rsync -a
rsync://olympus.lan.localdomain/gentoo-portage/metadata/herds.xml
"${repository_path}"/metadata/md5-cache/
eend $?
#news
NEWSDIR="${repository_path}"/metadata/news
ebegin "Updating news items"
if [[ -e ${NEWSDIR} ]]; then
git -C "${NEWSDIR}" pull -q --ff
else
git clone olympus.lan.localdomain:/usr/portage/metadata/news
"${NEWSDIR}"
fi
eend $? "Try to remove ${NEWSDIR}"
mv /usr/portage /usr/portage.old
git clone olympus.lan.localdomain:/usr/portage /usr/portage
echo 'olympus' > /usr/portage/profiles/repo_name
echo 'masters = olympus' > /usr/local/portage/metadata/layout.conf
echo 'thin-manifests = true' >> /usr/local/portage/metadata/layout.conf
cp /usr/portage/metadata/layout.conf
/usr/local/portage/metadata/layout.conf
cp /etc/portage/repos.conf/olympus.conf /usr/share/portage/config/repos.conf
emerge --sync
emerge --sync
* the first sync will complete the process, the second is to check there are
no errors.
Enjoy!
Re: [gentoo-user] git serving portage tree.
On 11/02/2015 04:52 PM, Bill Kenworthy wrote: > I have been playing with using git to sync portage while watching the > conversation on gentoo-dev. Works, and after the initial sync it is > actually faster than rsync and you get changelogs of a sort. > > Is there a guide to set up git as a local server for portage? > What do you mean by local server? You want to share the git repo on one of your machines with some other machines of yours? If so, the git repo is just a directory, and can be copied around with scp or anything else. But to do it efficiently, git knows how to operate over SSH. So on the client, you can do something like, $ git clone my-server.example.com:/path/to/repo.git Enter the SSH password for your account on my-server.example.com, and git will clone the whole repo. Later on, running `git pull` on the client (from within the repo.git directory) will sync it.
[gentoo-user] git serving portage tree.
I have been playing with using git to sync portage while watching the conversation on gentoo-dev. Works, and after the initial sync it is actually faster than rsync and you get changelogs of a sort. Is there a guide to set up git as a local server for portage? BillK
Re: [gentoo-user] git serving portage tree.
On 03/11/15 06:08, Michael Orlitzky wrote: > On 11/02/2015 04:52 PM, Bill Kenworthy wrote: >> I have been playing with using git to sync portage while watching the >> conversation on gentoo-dev. Works, and after the initial sync it is >> actually faster than rsync and you get changelogs of a sort. >> >> Is there a guide to set up git as a local server for portage? >> > > What do you mean by local server? You want to share the git repo on one > of your machines with some other machines of yours? > > If so, the git repo is just a directory, and can be copied around with > scp or anything else. But to do it efficiently, git knows how to operate > over SSH. So on the client, you can do something like, > > $ git clone my-server.example.com:/path/to/repo.git > > Enter the SSH password for your account on my-server.example.com, and > git will clone the whole repo. Later on, running `git pull` on the > client (from within the repo.git directory) will sync it. > > Thanks. Yes I have a few gentoo machines and around 12 gentoo VM's - I currently use a rsync/httpreplicator to a master machine architecture which could be replaced by git. BillK
Re: [gentoo-user] git serving portage tree.
On 11/02/2015 05:42 PM, Bill Kenworthy wrote: >> > > Thanks. Yes I have a few gentoo machines and around 12 gentoo VM's - I > currently use a rsync/httpreplicator to a master machine architecture > which could be replaced by git. > If you don't want to share credentials between the machines, git understands HTTP, too. So you could serve up the gentoo.git directory on your main box, and then $ git clone http://w.x.y.z/gentoo.git on the VMs. Afterwards `git pull` works normally and neither of those operations require any credentials.
