Re: Best VCS layout for our packages?
Hi, I checked out two random tags and compared it with the contents of the file on snap.debian.org; up to whitespace differences, they are equal. Good! Am Dienstag, den 07.07.2015, 07:54 +0300 schrieb Dmitry Bogatov: I added them, but some of them have conflicting names, in particular haskell-dbus. You can see what errors throws ~kaction-guest/haskell-darcs-mirror/relink.sh In such cases, non-old is preferred. Sounds reasonable. * All patches have me as a committer. Can you change that so that the author is the committer? Possible, I believe. Change user.name and user.email before applying every patch. Does anyone know tool for splitting From: field into name and email? Seems to work, but I did not checked all 22k patches. with the patches that I checked, I found a strange committer, e.g.: committer a19760978ffb9c876a8d2c792533fa9b50fb52c4 Mon Sep 17 00:00:00 2001 m...@joachim-breitner.de 1435798802 +0300 If that is fixed, I believe we can make the switch. What’s a good repository name? all-packages.git? DHG.git? Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: Best VCS layout for our packages?
Another iteration, with 22k patches now in
~kaction-guest/bare-united
A few minor comments:
* It would be nice to also have the repos from
/darcs/pkg-haskell/old/.
I added them, but some of them have conflicting names, in particular
haskell-dbus. You can see what errors throws
~kaction-guest/haskell-darcs-mirror/relink.sh
In such cases, non-old is preferred.
* These two repos should be excluded:
mrconfig policy
Done.
* The haskell-platform/ repository already contains a debian/
directory, so here you need to skip moving it into a subdirectory.
Done.
* https://github.com/kaction/darcs-to-git does not exist, it seems
Is it private or something?
It exists. Just
$ git clone git://github.com/kaction/darcs-to-git
Sometimes github does not display stuff in web interface, but `git(1)'
always works to me.
* All patches have me as a committer. Can you change that so that
the author is the committer?
Possible, I believe. Change user.name and user.email before applying every
patch.
Does anyone know tool for splitting From: field into name and email?
Seems to work, but I did not checked all 22k patches.
--
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
#!/bin/bash -eu
# Script converts all darcs repositories, used for individual packages by
# debian haskell group into single git repository. It has following
# implementation steps:
#
# * Download list of repositories. See `list_darcs_repos' function and
#`DARCS_REPOS_LISTFILE' variable.
# * Clone every darcs repository. If it is already cloned, up it to date.
#See `clone_or_update_darcs_repos' function and `DARCS_LOCAL_REPOS_DIR'
#variable.
# * With `darcs-to-git' script convert every darcs repository into git one.
#See functions `convert_darcs_to_git_repo' and `convert_darcs_to_git_repos'
#functions and `GIT_LOCAL_REPOS_DIR' variable.
# * Convert every git repository into set of patches, renaming patches to
#make them be sorted by date. Every patch contains extra information
#about it's tag and repository.
# * Unite them with `git-am`. See `assemble_united_git_repository' function.
#
# Script performs many separated calls to `ssh' command. The following lines
# in your `~/.ssh/config' will improve perfomance.
#
# Host *
# ControlPath ~/.ssh/master-%l-%r@%h:%p
# ControlMaster auto
# ControlPersist 100
#
# All in all, I would suggest to run this script nightly. There should not be
any
# surprise, but I can't prove it.
#
# If global variable is exported, it means it is used in some function
# called by GNU Parallel.
## This file contains list of darcs repositories we are going to
## incorporate into big git repository. This list is generated by
## `list_darcs_repos' function.
DARCS_REPOS_LISTFILE=repolist.darcs
## Host where darcs repositories are stored. You are assumed to
## have paswordless ssh access to this host.
export DARCS_REPOS_HOST=darcs.debian.org
## Directory on `DARCS_REPOS_HOST', containing all darcs repositories
## we are interested in.
export
DARCS_REPOS_DIR=/var/lib/gforge/chroot/home/users/kaction-guest/haskell-darcs-mirror
## Directory where all cloned darcs repositories are stored. It greatly
## improve perfomance.
export DARCS_LOCAL_REPOS_DIR=darcs
## Directory where all converted git repositories will be stored.
export GIT_LOCAL_REPOS_DIR=git
## Directory, where all patches, representing git repositories will be stored.
## It contains subdirectories for every repository, to make convering from
## git repository to patches parallelable.
export PATCHES_DIR=patches
## Git repository, that will absorb all patches.
export UNITED_GIT_REPO=united-git-repository
### list_darcs_repos
## Print darcs repositories in `DARCS_REPOS_DIR' directory on `DARCS_REPOS_HOST'
## host to the stdout. Directory assumed to be valid darcs repository, if it
## contains `_darcs/inventories' subdirectory and does not contains `DELETE' in
its name.
## As another exception, repositories `mrconfig' and `policy' are excluded.
list_darcs_repos () {
tmpfile=$(mktemp)
cat EOF $tmpfile
#!/bin/bash
for dir in \$(ls -1A $DARCS_REPOS_DIR) ; do
if [[ -d $DARCS_REPOS_DIR/\$dir/_darcs/inventories ]] ; then
echo \$dir
fi
done
rm -f \$0
EOF
scp $tmpfile $DARCS_REPOS_HOST:$tmpfile
ssh $DARCS_REPOS_HOST /bin/bash $tmpfile \
| grep -v DELETE | grep -v '^mrconfig$' | grep -v '^policy$'
rm -f $tmpfile
}
### clone_or_update_darcs_repo NAME
## Clone repository with NAME from host `DARCS_REPOS_HOST'
## in directory `DARCS_REPOS_DIR' into `DARCS_LOCAL_REPOS_DIR'.
## If it is already there, update it.
clone_or_update_darcs_repo () {
local repo=$1
local cwd=$PWD
cd $DARCS_LOCAL_REPOS_DIR
if [[ -d $repo ]] ; then
darcs pull -q --all --repodir=$repo
else
darcs clone -q $DARCS_REPOS_HOST:$DARCS_REPOS_DIR/$repo --complete
fi
## This is strange hack to workaround mysterios
Re: Best VCS layout for our packages?
On Sat, Jul 04, 2015 at 05:41:36AM +0300, Dmitry Bogatov wrote: * Joachim Breitner nome...@debian.org [2015-07-03 22:02:14+0200] * All patches have me as a committer. Can you change that so that the author is the committer? Possible, I believe. Change user.name and user.email before applying every patch. Git will honor environment variables to control those, which should be helpful. See git-commit-tree(1) for details on what's available and the format of the data it recognizes. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@debian.org -- To UNSUBSCRIBE, email to debian-haskell-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150706021342.gc16...@freya.jamessan.com
Re: Best VCS layout for our packages?
* Joachim Breitner nome...@debian.org [2015-07-03 22:02:14+0200] Am Mittwoch, den 01.07.2015, 17:08 +0300 schrieb Dmitry Bogatov: Take a look in /var/lib/gforge/chroot/home/users/kaction-guest/bare-united [...] A few minor comments: * It would be nice to also have the repos from /darcs/pkg-haskell/old/. * These two repos should be excluded: mrconfig policy * The haskell-platform/ repository already contains a debian/ directory, so here you need to skip moving it into a subdirectory. * https://github.com/kaction/darcs-to-git does not exist, it seems Is it private or something? I hope to look at it till next weekend. * All patches have me as a committer. Can you change that so that the author is the committer? Possible, I believe. Change user.name and user.email before applying every patch. Does anyone know tool for splitting From: field into name and email? Also, if anyone knows natural way, similar to `git am --committer-date-is-author-date', it would be even greater. -- Accept: text/plain, text/x-diff Accept-Language: eo,en,ru X-Keep-In-CC: yes pgpmKRhMxmSKJ.pgp Description: PGP signature
Re: Best VCS layout for our packages?
Quoting Gianfranco Costamagna (2015-07-03 15:56:48) ./ghc/debian/rules:GHC=$(firstword $(shell bash -c type -p ghc)) this is something really obscure to me... why can't we just do GHC=$(which ghc) Variable expansion in debian/rules is not shell but make. It can be simplified to eliminate bash like this: ./ghc/debian/rules:GHC := $($(shell which ghc) (the change from = to := is to resolve once when read rahter than postpone and resolve each time the variable is used). - Jonas -- * Jonas Smedegaard - idealist Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
Re: Best VCS layout for our packages?
On Wed, 1 Jul 2015 17:08:24 +0300 Dmitry Bogatov kact...@gnu.org wrote: Take a look in /var/lib/gforge/chroot/home/users/kaction-guest/bare-united What do you think? It was generated by attached script (takes several hours and a lot of heat) Looks good! Thanks for the work! Why is everything under a subdirectory named p? Does it serve any special purpose? Regards Sven pgptjK0sNQXq1.pgp Description: Digitale Signatur von OpenPGP
Re: Best VCS layout for our packages?
On Fri, Jul 03, 2015 at 08:56:48PM +, Gianfranco Costamagna wrote: Hi, #!/bin/bash -eu I do not know where it does come from, but why people still use bash instead of sh? The conversion script is using bash-specific functionality. From what I see at a quick glance all of it can trivially be replaced with sh-compatible code, but as it currently stands the #!/bin/bash is accurate. I did some bin/bash search on that git repo […] they all seems to be replaceable That does seem to be the case, but that can be handled independently from the process of actually converting to git. Cheers, -- James GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy james...@debian.org -- To UNSUBSCRIBE, email to debian-haskell-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150704001743.ga16...@freya.jamessan.com
Re: Best VCS layout for our packages?
Hi, Am Mittwoch, den 01.07.2015, 17:08 +0300 schrieb Dmitry Bogatov: Take a look in /var/lib/gforge/chroot/home/users/kaction-guest/bare-united What do you think? It was generated by attached script (takes several hours and a lot of heat) very good job! I like it. A few minor comments: * It would be nice to also have the repos from /darcs/pkg-haskell/old/. * These two repos should be excluded: mrconfig policy * The haskell-platform/ repository already contains a debian/ directory, so here you need to skip moving it into a subdirectory. * All patches have me as a committer. Can you change that so that the author is the committer? * https://github.com/kaction/darcs-to-git does not exist, it seems Is it private or something? I’m surprised that you did not use any of the git-merging-tools that I mentioned on the list, but if the result is good, then that’s of course fine! I’d appreciate another pair of eyeballs. So especially those who are happy to switch to git: Can you review this repository as well? Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: Best VCS layout for our packages?
On 2015-07-01 17:08:24, Dmitry Bogatov wrote: * Joachim Breitner nome...@debian.org [2015-06-28 10:29:23+0200] Hi, Am Sonntag, den 28.06.2015, 05:26 +0300 schrieb Dmitry Bogatov: * Joachim Breitner nome...@debian.org [2015-06-15 20:38:25+0200] C. One big repo, haskell-foo’s debian/ files in haskell -foo/debian/ Having one repo has various advantages: [..] But there are also disadvantages: [..] Seems that most of us are prefering this variant. I am going to play with it during next 48 hours. Is it something to base on around or someone who already started hacking on it? I haven’t, if you want to give it a shot, that’d be great. I assume you are talking about converting the darcs repos to a git repository. Take a look in /var/lib/gforge/chroot/home/users/kaction-guest/bare-united What do you think? It was generated by attached script (takes several hours and a lot of heat) Nitpick (feel free to ignore): #!/bin/bash -eu I much prefer moving the -eu to from shebang 'set -eu', since that works even when someone calls the script via 'bash -x'. The shebang options take effect only when some calls the script directly, not via bash. thanks, iustin signature.asc Description: Digital signature
Re: Best VCS layout for our packages?
* Joachim Breitner nome...@debian.org [2015-06-28 10:29:23+0200]
Hi,
Am Sonntag, den 28.06.2015, 05:26 +0300 schrieb Dmitry Bogatov:
* Joachim Breitner nome...@debian.org [2015-06-15 20:38:25+0200]
C. One big repo, haskell-foo’s debian/ files in haskell
-foo/debian/
Having one repo has various advantages:
[..]
But there are also disadvantages:
[..]
Seems that most of us are prefering this variant. I am going to play
with it during next 48 hours. Is it something to base on around or
someone who already started hacking on it?
I haven’t, if you want to give it a shot, that’d be great. I assume you
are talking about converting the darcs repos to a git repository.
Take a look in
/var/lib/gforge/chroot/home/users/kaction-guest/bare-united
What do you think? It was generated by attached script (takes several
hours and a lot of heat)
--
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Keep-In-CC: yes
#!/bin/bash -eu
# Script converts all darcs repositories, used for individual packages by
# debian haskell group into single git repository. It has following
# implementation steps:
#
# * Download list of repositories. See `list_darcs_repos' function and
#`DARCS_REPOS_LISTFILE' variable.
# * Clone every darcs repository. If it is already cloned, up it to date.
#See `clone_or_update_darcs_repos' function and `DARCS_LOCAL_REPOS_DIR'
#variable.
# * With `darcs-to-git' script convert every darcs repository into git one.
#See functions `convert_darcs_to_git_repo' and `convert_darcs_to_git_repos'
#functions and `GIT_LOCAL_REPOS_DIR' variable.
# * Convert every git repository into set of patches, renaming patches to
#make them be sorted by date. Every patch contains extra information
#about it's tag and repository.
# * Unite them with `git-am`. See `assemble_united_git_repository' function.
#
# Script performs many separated calls to `ssh' command. The following lines
# in your `~/.ssh/config' will improve perfomance.
#
# Host *
# ControlPath ~/.ssh/master-%l-%r@%h:%p
# ControlMaster auto
# ControlPersist 100
#
# All in all, I would suggest to run this script nightly. There should not be
any
# surprise, but I can't prove it.
#
# If global variable is exported, it means it is used in some function
# called by GNU Parallel.
## This file contains list of darcs repositories we are going to
## incorporate into big git repository. This list is generated by
## `list_darcs_repos' function.
DARCS_REPOS_LISTFILE=repolist.darcs
## Host where darcs repositories are stored. You are assumed to
## have paswordless ssh access to this host.
export DARCS_REPOS_HOST=darcs.debian.org
## Directory on `DARCS_REPOS_HOST', containing all darcs repositories
## we are interested in.
export DARCS_REPOS_DIR=/darcs/pkg-haskell
## Directory where all cloned darcs repositories are stored. It greatly
## improve perfomance.
export DARCS_LOCAL_REPOS_DIR=darcs
## Directory where all converted git repositories will be stored.
export GIT_LOCAL_REPOS_DIR=git
## Directory, where all patches, representing git repositories will be stored.
## It contains subdirectories for every repository, to make convering from
## git repository to patches parallelable.
export PATCHES_DIR=patches
## Git repository, that will absorb all patches.
export UNITED_GIT_REPO=united-git-repository
### list_darcs_repos
## Print darcs repositories in `DARCS_REPOS_DIR' directory on `DARCS_REPOS_HOST'
## host to the stdout. Directory assumed to be valid darcs repository, if it
## contains `_darcs/inventories' subdirectory and does not contains `DELETE' in
its name.
list_darcs_repos () {
tmpfile=$(mktemp)
cat EOF $tmpfile
#!/bin/bash
for dir in \$(ls -1A $DARCS_REPOS_DIR) ; do
if [[ -d $DARCS_REPOS_DIR/\$dir/_darcs/inventories ]] ; then
echo \$dir
fi
done
rm -f \$0
EOF
scp $tmpfile $DARCS_REPOS_HOST:$tmpfile
ssh $DARCS_REPOS_HOST /bin/bash $tmpfile \
| grep -v DELETE
rm -f $tmpfile
}
### clone_or_update_darcs_repo NAME
## Clone repository with NAME from host `DARCS_REPOS_HOST'
## in directory `DARCS_REPOS_DIR' into `DARCS_LOCAL_REPOS_DIR'.
## If it is already there, update it.
clone_or_update_darcs_repo () {
local repo=$1
local cwd=$PWD
cd $DARCS_LOCAL_REPOS_DIR
if [[ -d $repo ]] ; then
darcs pull -q --all --repodir=$repo
else
darcs clone -q $DARCS_REPOS_HOST:$DARCS_REPOS_DIR/$repo --complete
fi
## This is strange hack to workaround mysterios files, appearing in some
repositories.
## FIXME: Remove this hack and try again.
cd $repo
find ! -regex .*/_darcs.* -delete
darcs revert -a /dev/null
cd $cwd
}
## This way we can pass it to GNU Parallel.
export -f clone_or_update_darcs_repo
### clone_or_update_darcs_repos
## Call `clone_or_update_darcs_repo' function for every repository
## name in file `DARCS_REPOS_LISTFILE'
##
## It will take a lot of time.
Re: Best VCS layout for our packages?
Hi, Am Sonntag, den 28.06.2015, 05:26 +0300 schrieb Dmitry Bogatov: * Joachim Breitner nome...@debian.org [2015-06-15 20:38:25+0200] C. One big repo, haskell-foo’s debian/ files in haskell -foo/debian/ Having one repo has various advantages: [..] But there are also disadvantages: [..] Seems that most of us are prefering this variant. I am going to play with it during next 48 hours. Is it something to base on around or someone who already started hacking on it? I haven’t, if you want to give it a shot, that’d be great. I assume you are talking about converting the darcs repos to a git repository. I suggest to do it in a script, in case the result is not yet fully what we we want and you need to change something early, so it is easy to re-trace the steps. Here is what I would do: 1. Fetch all darcs repositories from alioth, including those in /darcs/pkg-haskell/old 2. Convert all of them to git. The latest darcs has a darcs export command to do that. 3. For each repository, use git filter-branch to move all old commits everything to a debian/ subdirectory. 4. Then use one of the tools mentioned in my mail from Jun 18 to combine the repositories into one. 5. Use git filter-branch to put everything into a subdirectory p/. I can imagine that we will later want to put non-package-things into that repository as well (e.g. the package plan, or simply tools required to work with the repository, or documentation) and it is nice to have space for that. Note that dpkg-buildpackage will spew out its result into that p/ directory. 6. Add a commit that removes all packages that you took from old/ 7. Add a .gitignore to p/ that makes sure that everything is ignored in p/* and p/*/* with the exception of p/*/debian. 8. Put it on alioth and let us see! Enjoy the hacking! Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: Best VCS layout for our packages?
* Joachim Breitner nome...@debian.org [2015-06-15 20:38:25+0200] C. One big repo, haskell-foo’s debian/ files in haskell-foo/debian/ Having one repo has various advantages: [..] But there are also disadvantages: [..] Seems that most of us are prefering this variant. I am going to play with it during next 48 hours. Is it something to base on around or someone who already started hacking on it? -- Accept: text/plain, text/x-diff Accept-Language: eo,en,ru X-Keep-In-CC: yes pgpDvxkJa2ttQ.pgp Description: PGP signature
Re: Best VCS layout for our packages?
C B A, but I don't have that strong an opinion on it. Darcs is quaint, but the community seems to favor git more than ever, so I'll follow group consensus either way. Greetings Marcel pgpPgfy53Qnp7.pgp Description: OpenPGP digital signature
Re: Best VCS layout for our packages?
On 2015-06-17 09:19:56, Joachim Breitner wrote: Hi, Am Dienstag, den 16.06.2015, 20:39 -0700 schrieb Iustin Pop: In general, I believe that storing upstream code in git is better for reasons beyond packaging ease (e.g. authoritative deltas between upstream orig.tar.gz as they were uploaded in Debian, reproduction of arbitrary past versions, etc.), but I kind of agree that for mass packaging it's better to not do that. That said, I'd prefer if at least the repository records a checksum of the upstream archive, since I feel this checkout git signed tags but use unsigned upstream archive from hackage is not safe enough. Anyway, another discussion. yes, this is a drawback of that approach, as also noted by Thomas Koch. For tarballs that are already in the archive, the problem is not so big (the authoritative file is there, and one cannot accidentally upload another), but that does not help until the first version is uploaded. I believe that even for existing versions, recording the checksum somewhere in the git repo (potentially on another branch) is helpful, as it ties together the debian packaging and upstream correspondence in a single place, rather than having to dig in other places. But this is really beside Haskell packaging, which is the point under discussion here :) hackage-security will eventually help here, but that does not exist yet. ack. thanks! iustin signature.asc Description: Digital signature
Re: Best VCS layout for our packages?
Hi, Am Dienstag, den 16.06.2015, 20:39 -0700 schrieb Iustin Pop: In general, I believe that storing upstream code in git is better for reasons beyond packaging ease (e.g. authoritative deltas between upstream orig.tar.gz as they were uploaded in Debian, reproduction of arbitrary past versions, etc.), but I kind of agree that for mass packaging it's better to not do that. That said, I'd prefer if at least the repository records a checksum of the upstream archive, since I feel this checkout git signed tags but use unsigned upstream archive from hackage is not safe enough. Anyway, another discussion. yes, this is a drawback of that approach, as also noted by Thomas Koch. For tarballs that are already in the archive, the problem is not so big (the authoritative file is there, and one cannot accidentally upload another), but that does not help until the first version is uploaded. hackage-security will eventually help here, but that does not exist yet. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
Re: Best VCS layout for our packages?
Hi, Joachim Breitner nome...@debian.org writes: A. One repo per package, debian/ files in repo root. B. One repo per package, debian directory in repo. C. One big repo, haskell-foo’s debian/ files in haskell-foo/debian/ I've never liked the redundancy of having a debian/ directory as a repo root, so for me B A. I prefer C, but we should think in advance what things will break after implementing this. One thing that comes to my mind is, for example, vcswatch CGI used by the QA team. Cheers, Rul signature.asc Description: PGP signature
Re: Best VCS layout for our packages?
Joachim Breitner a écrit : C. One big repo, haskell-foo’s debian/ files in haskell-foo/debian/ [...] But there are also disadvantages: * Slightly unusual, so needs explanation. * It is not possible to check out just one package. But maybe that is a good thing, for such a tightly coupled package set. * Some tools might need to be extended to support that well. For example debcheckout would need to support a subdirectory parameter in the url specified in Vcs-Git, similar to the branch parameter. * Tags needs to be qualified by the package name (haskell -foo/1.0-1 instead of 1.0-1). This would need support in debcommit, similar to how gbp supports --git-debian-tag which can be set to %(pkg)/%(version). It seems that some teams rely on mr/myrepos to manage individual packages (with standalone git repositories) within a single meta repository which only includes the mrconfig. E.g.: http://anonscm.debian.org/cgit/pkg-go/meta.git/ http://anonscm.debian.org/cgit/pkg-perl/meta.git/ Wouldn't this help to circumvent these issues? -- To UNSUBSCRIBE, email to debian-haskell-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/558080c8.2030...@laxalde.org
Re: Best VCS layout for our packages?
On 2015-06-15 20:38:25, Joachim Breitner wrote: Hi, eventually, we will have to migrate away from Darcs, I’m afraid. If only that nobody can use his ignorance of darcs as an excuse to do work here :-). I’m sure some of you will be happy to hear that. Indeed! Thanks for starting this discussion. But this opens a new can of worms: What is the best git-based packaging approach for us? I would argue that we have uncommon needs. Most of our packages are very simple (and can be made even simpler with the help of cabal -debian/debdry), but often changes affect multiple packages (bumping of haskell-devscripts dependency) and may only make sense together (upgrading of dependency chains). So we should discuss what that means for us. In particular, I wonder if we find a scheme where all development happens in one git repository. Also, there is the question of what should be in a packaging git repository. I am a firm believer that the upstream code should _not_ be tracked in a packaging repository, on the grounds that we create and maintain the debian/ directory, and the debian directory only: The version in debian/changelog file and the debian/patches/ directory are sufficient to produce the source package. In general, I believe that storing upstream code in git is better for reasons beyond packaging ease (e.g. authoritative deltas between upstream orig.tar.gz as they were uploaded in Debian, reproduction of arbitrary past versions, etc.), but I kind of agree that for mass packaging it's better to not do that. That said, I'd prefer if at least the repository records a checksum of the upstream archive, since I feel this checkout git signed tags but use unsigned upstream archive from hackage is not safe enough. Anyway, another discussion. So I currently see three sensible variants: A. One repo per package, debian/ files in repo root. This is basically the status quo, simply converted to git. B. One repo per package, debian directory in repo. A small variation. This has the advantage that you can run $ git clone /haskell-foo.git $ cd haskell-foo $ origtargz and have a standard debian source directory, ready to run debuild or quilt or whatever. A .gitignore file can be crafted that makes sure that everything but debian/ is ignored. Note that debcheckout supports this as well. C. One big repo, haskell-foo’s debian/ files in haskell-foo/debian/ Having one repo has various advantages: * Most importantly, there is one commit per logical change, even if it affects multiple packages. * This also reduces noise on the commit mailing list. * It is immediately clear which packages we have packaged. * No need to annoyingly create remote repositories when you package something new. * In order to compare the repo state with the package plan, only one repository has to be looked at, not 700. * It becomes easier to enforce certain things with git hooks, as they do not have to be set up for every repository. * Git hooks (and maybe jenkins jobs) have a global view of the packaging. But there are also disadvantages: * Slightly unusual, so needs explanation. * It is not possible to check out just one package. But maybe that is a good thing, for such a tightly coupled package set. * Some tools might need to be extended to support that well. For example debcheckout would need to support a subdirectory parameter in the url specified in Vcs-Git, similar to the branch parameter. * Tags needs to be qualified by the package name (haskell -foo/1.0-1 instead of 1.0-1). This would need support in debcommit, similar to how gbp supports --git-debian-tag which can be set to %(pkg)/%(version). Personally, for a long while I'm using fully qualified tag names, after being burned during project merges due to tag conflicts. So I would welcome this change, actually :) I’d prefer C over B over A. How about you? While the myrepos suggestion is interesting, I also prefer C B A. The rationale is that without upstream sources in git, each individual repository is small enough that the overhead of many repositories will be much higher than the network/disk saving of being able to checkout only one repository. Plus, git partial checkouts. And furthermore, C is the logical way if we want to scale our packaging efforts. thanks again, iustin signature.asc Description: Digital signature
Re: Best VCS layout for our packages?
I’d prefer C over B over A. How about you? C B A -- To UNSUBSCRIBE, email to debian-haskell-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150615184620.ga28...@scru.org
Re: Best VCS layout for our packages?
Hi Joachim, you wrote: It is not possible to check out just one package. But maybe that is a good thing, for such a tightly coupled package set. http://stackoverflow.com/questions/600079/is-there-any-way-to-clone-a-git-repositorys-sub-directory-only with git 1.7 and sparse checkout this seems to be possible. Note: I don't care and I never tried it, but since you listed it as a disavantage, I wrote it there. When you start from scratch it might be useful to enable such features (if users needs them of course) cheers, Gianfranco Sent from Yahoo Mail on Android From:Joachim Breitner nome...@debian.org Date:Mon, 15 Jun, 2015 at 20:38 Subject:Best VCS layout for our packages? Hi, eventually, we will have to migrate away from Darcs, I’m afraid. If only that nobody can use his ignorance of darcs as an excuse to do work here :-). I’m sure some of you will be happy to hear that. But this opens a new can of worms: What is the best git-based packaging approach for us? I would argue that we have uncommon needs. Most of our packages are very simple (and can be made even simpler with the help of cabal -debian/debdry), but often changes affect multiple packages (bumping of haskell-devscripts dependency) and may only make sense together (upgrading of dependency chains). So we should discuss what that means for us. In particular, I wonder if we find a scheme where all development happens in one git repository. Also, there is the question of what should be in a packaging git repository. I am a firm believer that the upstream code should _not_ be tracked in a packaging repository, on the grounds that we create and maintain the debian/ directory, and the debian directory only: The version in debian/changelog file and the debian/patches/ directory are sufficient to produce the source package. So I currently see three sensible variants: A. One repo per package, debian/ files in repo root. This is basically the status quo, simply converted to git. B. One repo per package, debian directory in repo. A small variation. This has the advantage that you can run $ git clone /haskell-foo.git $ cd haskell-foo $ origtargz and have a standard debian source directory, ready to run debuild or quilt or whatever. A .gitignore file can be crafted that makes sure that everything but debian/ is ignored. Note that debcheckout supports this as well. C. One big repo, haskell-foo’s debian/ files in haskell-foo/debian/ Having one repo has various advantages: * Most importantly, there is one commit per logical change, even if it affects multiple packages. * This also reduces noise on the commit mailing list. * It is immediately clear which packages we have packaged. * No need to annoyingly create remote repositories when you package something new. * In order to compare the repo state with the package plan, only one repository has to be looked at, not 700. * It becomes easier to enforce certain things with git hooks, as they do not have to be set up for every repository. * Git hooks (and maybe jenkins jobs) have a global view of the packaging. But there are also disadvantages: * Slightly unusual, so needs explanation. * It is not possible to check out just one package. But maybe that is a good thing, for such a tightly coupled package set. * Some tools might need to be extended to support that well. For example debcheckout would need to support a subdirectory parameter in the url specified in Vcs-Git, similar to the branch parameter. * Tags needs to be qualified by the package name (haskell -foo/1.0-1 instead of 1.0-1). This would need support in debcommit, similar to how gbp supports --git-debian-tag which can be set to %(pkg)/%(version). I’d prefer C over B over A. How about you? Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata
Re: Best VCS layout for our packages?
Hi, Am Montag, den 15.06.2015, 20:30 +0100 schrieb Gianfranco Costamagna: Hi Joachim, you wrote:It is not possible to check out just one package. But maybethat is a good thing, for such a tightly coupled package set. http://stackoverflow.com/questions/600079/is-there-any-way-to-clone-a -git-repositorys-sub-directory-only with git 1.7 and sparse checkout this seems to be possible. thanks. That’s new to me, but it also looks like it needs some more porcelain support before it is nicely useful. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: F0FBF51F JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part
