Re: Introducing dgit - git integration with the Debian archive

2013-09-03 Thread Barry Warsaw
On Aug 31, 2013, at 01:37 AM, Jelmer Vernooij wrote:

In the end, this meant that using UDD had some minor benefits (a richer
history, and theoretical improved merge support)

I'd argue that they were more than minor benefits!

but there were a number of extra things you had to watch out for that made it
frustrating to use: manually checking sure that the branch was in sync with
the archive before using it (and giving up on UDD or prodding us if it
wasn't),

It was really great when you guys added the feature to bzr to actually do this
check and tell you if the branch was up-to-date or not.

making sure you pushed the right changes to the UDD branches (otherwise they
would be overwritten by the automatic importer).

I learned never to push a UDD branch.  I almost never had problems when I let
the importer update the branch on upload, rather than try to push a branch.
The downside is that there could be a lag between upload and branch import,
but it hasn't really been much of a problem in practice.

Cloning one of the bzr branches can also be quite slow because of performance
issues in bzr that took a long time to fix, and because Launchpad is in the
UK, whereas there are a lot of mirrors of the Ubuntu archive.

I never found the performance issues to be a hindrance, but I have a good
internet connection.  Besides, over time this was ameliorated by using bzr
shared repos.

I always considered having the .pc/ directory checked in and the patches
applied by default for the quilt format a mistake, as it lead to tons of
spurious conflicts during merges.

It's nice that a source branch gives you a fully unpacked and patched tree
which you can start hacking on immediately, but it was tricky to get quilt
interaction right, especially when doing merges.  Another important lesson is
to be sure that when committing a branch (say for a merge proposal), you do it
at the same quilt push level as the original branch.

-Barry


signature.asc
Description: PGP signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-30 Thread Jelmer Vernooij
On Sun, Aug 25, 2013 at 01:04:32PM +0200, Raphael Hertzog wrote:
 Hello,
 
 On Thu, 22 Aug 2013, Ian Jackson wrote:
  I'm pleased to announce that dgit 0.7, which is a version of dgit
  suitable for alpha and beta testers, is available in unstable.
  
  From the manpage:
  
 dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
 dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
 dgit [dgit-opts] build|sbuild [build-opts]
 dgit [dgit-opts] push [dgit-opts] [suite]
  
 dgit treats the Debian archive as a version control system, and
 bidirectionally gateways between the archive and git.  The git
 
 Basically, this is Ubuntu Distributed Development (UDD) but for Debian 
 Git (instead of Ubuntu  bzr).
 
 Have you looked at UDD? They have been doing this for multiple years and
 have much more experience than us here. I'm sure there a quite a few
 things to learn from what they did to not redo the same mistakes.
 
 https://wiki.ubuntu.com/DistributedDevelopment
 http://developer.ubuntu.com/packaging/html/udd-intro.html
 https://launchpad.net/udd
 
 I'm putting James Westby in CC (as I believe he's one of the core UDD
 developers, and also a Debian contributor). He might want to review dgit
 and share his hindsights.
 
 Among notable differences there's that dgit contrary to UDD decentralizes
 the creation of the branch with all the archive uploads. But I never used
 UDD and don't know it well enough to comment much more than that.

The automatic importer was one of the main problems in UDD. There was a small
team responsible for importing all of the archive into bzr.

The goal was to import the entire archive and as much history as we could find,
including all of the odd packages that live in it. We spent a lot of time
dealing with odd source packages - packages that were huge like nexuiz, source
packages with multiple upstream tarballs, tarballs with unusual file names that
bzr didn't like (e.g. \), etc.

This centralized approach meant that Ubuntu developers depended on
the team building and running the importer to fix bugs and deal with
out of date branches when the importer failed. Any errors from the importer
wouldn't be visible to them - it ran as a separate cronjob elsewhere,
with its output only visible on the package-importer web site.

Users weren't empowered to fix bugs. There were a couple of people
who contributed fixes to the UDD importer, but this was far harder than it
should be, as setting up the importer locally was a pain.

In the end, this meant that using UDD had some minor benefits (a
richer history, and theoretical improved merge support) but there were a number
of extra things you had to watch out for that made it frustrating to use:
manually checking sure that the branch was in sync with the archive before
using it (and giving up on UDD or prodding us if it wasn't), making sure you
pushed the right changes to the UDD branches (otherwise they would be
overwritten by the automatic importer). Cloning one of the bzr branches can
also be quite slow because of performance issues in bzr that took a long time
to fix, and because Launchpad is in the UK, whereas there are a lot of mirrors
of the Ubuntu archive.

I always considered having the .pc/ directory checked in and the patches applied
by default for the quilt format a mistake, as it lead to tons of spurious
conflicts during merges.

Cheers,

Jelmer


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130830233732.ga10...@vernstok.nl



Re: Introducing dgit - git integration with the Debian archive

2013-08-26 Thread Philipp Kern
On Sun, Aug 25, 2013 at 08:10:14PM -0400, Joey Hess wrote:
 Steve Langasek wrote:
  I would have expected dgit to support pristine-tar
  directly/automatically/unconditionally.  Any system that requires me to
  download the same information (== the upstream source) both from a VCS
  repository and the archive in order to get a fully-formed source package for
  upload is a non-starter.
 AFAIK you don't actually need the bit-identical .orig.tar.gz to build
 the  -1 uploads of a package. You only need a tarball that contains the
 identical files as the .orig.tar.gz. Which can be generated from a
 tag or branch with the same contents. Perhaps dgit should arrange for
 that to be checked in automatically the same way it checks in the
 debinized source tree?

I'd hope that the hashes of the .orig are part of the  -1 .dsc, even if
not of the .changes (only with -sa).

Kind regards
Philipp Kern


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-26 Thread Joey Hess
Philipp Kern wrote:
 I'd hope that the hashes of the .orig are part of the  -1 .dsc, even if
 not of the .changes (only with -sa).

They are, but dgit builds the dsc.

-- 
see shy jo


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Raphael Hertzog
Hello,

On Thu, 22 Aug 2013, Ian Jackson wrote:
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.
 
 From the manpage:
 
dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]
 
dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git

Basically, this is Ubuntu Distributed Development (UDD) but for Debian 
Git (instead of Ubuntu  bzr).

Have you looked at UDD? They have been doing this for multiple years and
have much more experience than us here. I'm sure there a quite a few
things to learn from what they did to not redo the same mistakes.

https://wiki.ubuntu.com/DistributedDevelopment
http://developer.ubuntu.com/packaging/html/udd-intro.html
https://launchpad.net/udd

I'm putting James Westby in CC (as I believe he's one of the core UDD
developers, and also a Debian contributor). He might want to review dgit
and share his hindsights.

Among notable differences there's that dgit contrary to UDD decentralizes
the creation of the branch with all the archive uploads. But I never used
UDD and don't know it well enough to comment much more than that.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130825110432.gb20...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Dmitrijs Ledkovs
On 25 August 2013 12:04, Raphael Hertzog hert...@debian.org wrote:
 Hello,

 On Thu, 22 Aug 2013, Ian Jackson wrote:
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.

 From the manpage:

dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]

dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git

 Basically, this is Ubuntu Distributed Development (UDD) but for Debian 
 Git (instead of Ubuntu  bzr).

 Have you looked at UDD? They have been doing this for multiple years and
 have much more experience than us here. I'm sure there a quite a few
 things to learn from what they did to not redo the same mistakes.


Things to learn from UDD:
1) the fact that debian didn't have a _standartised_ VCS repository
format, for UDD workflow all debian packages had to be imported, such
that lp:debian/package can be merged into lp:ubuntu/package.
2) 3.0 (quilt) causes problems:
- we had to go with committing .pc directory in the unpacked tree. As
otherwise, new patch end up at the start of the quilt series and can
cause the rest of series to fail to apply
- debian/patches/series.$vendor is evil, often series.ubuntu were not
updated/refreshed/rebased causing dpkg-source -x to fail with
DEB_VENDOR=Ubuntu
- Merging two quilt series is a pain, as there is no $ quilt merge. We
end up unapplying both quilt series, merging the branches and throw
conflicts in debian/patches/series at the developer and asking them to
figure out what patches to apply and refresh quilt series themselves.
- versioning .pc directory is a pain, especially when quilt is
updated. E.g. newer versions of quilt added pointless .timestamp files
in the .pc directory which where not present in the automatic
lp:ubuntu/* and lp:debian/* branches which used older quilt
- a valid git/bzr patch may not be a valid quilt patch, and it turn
may not be a valid patch as considered by dpkg. It's getting better
with patch(1) starting to support git format-patch style patches. Thus
cherry-picking from upstream becomes a pain, I have multiple times
applied upstream cherry-picked patch, only later find out that e.g. +x
flag was not preserved, or fuzz is generated, or files are not
renamed.
- tarball inside tarball packaging is evil  must die
3) Automatic importer is part of the UDD workflow, only because there
was no standartised developer created rich-VCS history on Debian side
which fully matched the archive state. And basing ubuntu branches, on
something that doesn't match debian uploads into the archive was a
no-go.
4) automatic importer was necessory to import Debian history and well
it was not perfect: http://package-import.ubuntu.com/status/,
pristine-tar used to fail (importer was running on stable, now
upgraded and much better), dpkg-source -x sometimes fails, operational
issues (timeouts, OOM, etc), unreconcilable history (developer rebases
old tags, and importer can no longer reconcile it's state),
- history can be odd: UDD discovered where referenced uploads didn't
happen, or experimetal got ahead  then behind sid and has a really
hard time figuring out when, if ever, experimental got merged into
sid. (sometimes it's just abandoned)
I think james can give more examples.

The best UDD workflow seems to work with native packages:
As a highlight I can give example debian-installer. All
debian-installer git repositories are homogeneous and follow the same
layout.
All of d-i projects are imported into bzr branches
https://code.launchpad.net/d-i
And then Ubuntu Installer team maintains branches where Ubuntu diverges, e.g.:
lp:~ubuntu-core-dev/apt-setup/ubuntu which frequently merges in debian
changes from lp:apt-setup

In a similar manner packages which use 1.0 format without a patch
system work really well with lp:ubuntu/* and lp:debian/* branches.

I have maintenance access to UDD  have filed a few bugs about it, and
all I can say is that dgit so far is getting a lot of things right:

1) round-trip tree guarantee
same is required for UDD, and automatic importer can fail to get the
state right when developers push different tree in the VCS vs what
dpkg-source -x produces.
Don't forget, e.g. git doesn't commit empty directories. I have seen a
case where bzr-git was used to push commits without empty directories
into lp:ubuntu/$pkg branches  then dpkg-source -x not matching the
state of the vcs, resulting in the automatic importer failing.

2) removing automatic importer
forcing all the checks on the developer side  forcing VCS commit to
match the src upload is a massive win. It means that one can actually
trust the archive  VCS commits. And they will always match. (Well one
can even verify that by unpacking the .dsc and comparing 

Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Dmitrijs Ledkovs
On 22 August 2013 20:52, Ian Jackson ijack...@chiark.greenend.org.uk wrote:
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.


I have now started daily PPA builds for dgit, for all supported Ubuntu releases.

add-apt-repository ppa:xnox/dgit

Since dgit dependencies are so simple, it should also be safe to use
that PPA on any debian release as well, e.g.:

deb http://ppa.launchpad.net/xnox/dgit/ubuntu precise main
deb-src http://ppa.launchpad.net/xnox/dgit/ubuntu precise main

With the following archive key 1024R/4031D287
2D9DF1E22F3416238D46F49F157951FE4031D287
http://keyserver.ubuntu.com:11371/pks/lookup?op=getsearch=0x157951FE4031D287

Regards,

Dmitrijs.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANBHLUjGfENtc3ne+Cupwvgc4DtN3+_8kcJ=21oxxwzksd8...@mail.gmail.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Ian Jackson
Raphael Hertzog writes (Re: Introducing dgit - git integration with the Debian 
archive):
 What you mean is that on the server side the dgit-branches containing the
 uploads are in refs/dgit/ and not in refs/heads/. But there will also
 be refs/heads/dgit/suite branches on the server side since that's where
 people will naturally push stuff:

Yes.  That's fine.

 So there's a refs/head/dgit/suite and it can be pushed to the server.
 But it will be different to the refs/dgit/suite that gets merged by
 dgit pull...

Indeed, if you do that kind of git push.  The remote that
corresponds to that is origin though.

 Speaking of braindamage... :-)

I'll see if it confuses people.  If so it should probably be mentioned
in the manpage.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21018.5937.766774.259...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
archive):
 Unfortunately there is a problem that means it's difficult for others
 to test.  The alioth git directory is not writeable by the right
 group.  [...]

I have fixed this by using a subdirectory instead, whose permissions I
have set appropriately.  (Thanks to Joey for the suggestion.)

This means that previous versions of dgit won't work any more.  Also,
trees previosly made with dgit clone will have to have their
origin remotes adjusted with something like this:

  git-remote set-url origin 
git+ssh://i...@git.debian.org//git/dgit-repos/repos/PACKAGE.git

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21018.11167.530395.993...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Steve Langasek
On Sun, Aug 25, 2013 at 12:51:31PM +0100, Dmitrijs Ledkovs wrote:
 I have maintenance access to UDD  have filed a few bugs about it, and
 all I can say is that dgit so far is getting a lot of things right:

snip 

 2) removing automatic importer
 forcing all the checks on the developer side  forcing VCS commit to
 match the src upload is a massive win. It means that one can actually
 trust the archive  VCS commits. And they will always match. (Well one
 can even verify that by unpacking the .dsc and comparing it to the
 Dgit: commit id) After all the archive will always be authoritative,
 as that's that gets GPG signature, is mirrored and gets deployed to
 the users.

I don't think removing the automatic importer is an improvement at all. 
If we want VCS branches for the whole of Debian that we can rely on,
something / someone needs to update them automatically when a package is
uploaded.

The problems with the UDD automatic importer have all been around its
failing to cope with any kind of manual changes to the bzr branch.  I.e.,
if even once the importer sees an upload before it sees the corresponding
commit on the bzr branch - because a maintainer did a bzr push --overwrite,
or because of a race between the upload and the branch propagation, or
because of a bug on the server - then that package is forever after in
manual import mode until someone with admin privileges can kick the
machine.  This is a pretty bad failure mode; but it's a failure because the
importer can't cope with changes to the branch, not because automatic
importing was being done.

 And one is free to push pristine-tar (if makes sense/easy to
 generate), and/or any other branches into the repository (git-dpm,
 git-quilt, etc)

I would have expected dgit to support pristine-tar
directly/automatically/unconditionally.  Any system that requires me to
download the same information (== the upstream source) both from a VCS
repository and the archive in order to get a fully-formed source package for
upload is a non-starter.

 I am exited about dgit, as for the first time it will be possible for
 derivatives to centrally share history with Debian.

I am as well!

 In practice one doesn't actually care how far back the history goes,
 as the history that is interesting is where developers get to do
 intermediate commits between the two uploads to granulise the
 changes

I don't agree with this at all.  I have regularly made use of the UDD
branches to examine the history of packages (and not just the Ubuntu
history, but also the Debian history).  Being upload-level granularity makes
it less useful than if it were at the granularity of a VCS branch being
committed to natively by the developer, but it's still *very* useful for
understanding the uploader's thought process at the time a change was made.

The fact that git will allow us to graft the developer's own VCS on to these
dgit repositories in a way that UDD never did is an important improvement,
but as this is *optional*, not importing the package history from the
archive would make dgit much less useful for the common case than UDD is
today.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Dmitrijs Ledkovs
On 25 August 2013 17:31, Steve Langasek vor...@debian.org wrote:
 On Sun, Aug 25, 2013 at 12:51:31PM +0100, Dmitrijs Ledkovs wrote:
 I have maintenance access to UDD  have filed a few bugs about it, and
 all I can say is that dgit so far is getting a lot of things right:

 snip

 2) removing automatic importer
 forcing all the checks on the developer side  forcing VCS commit to
 match the src upload is a massive win. It means that one can actually
 trust the archive  VCS commits. And they will always match. (Well one
 can even verify that by unpacking the .dsc and comparing it to the
 Dgit: commit id) After all the archive will always be authoritative,
 as that's that gets GPG signature, is mirrored and gets deployed to
 the users.

 I don't think removing the automatic importer is an improvement at all.
 If we want VCS branches for the whole of Debian that we can rely on,
 something / someone needs to update them automatically when a package is
 uploaded.


Under dgit: push = (git push + dput *.changes). Thus the failure mode
is much sooner and in general it's more strict.

For uploads done without using dgit, it can synthesise upload
granulaty commits with reproducible / stable commit ids.

Thus the branches are maintained up to date, without the need to rely
on automatic importer.
One can trivially add automatic importer for uploads done without dgit.


 The problems with the UDD automatic importer have all been around its
 failing to cope with any kind of manual changes to the bzr branch.  I.e.,
 if even once the importer sees an upload before it sees the corresponding
 commit on the bzr branch - because a maintainer did a bzr push --overwrite,
 or because of a race between the upload and the branch propagation, or
 because of a bug on the server - then that package is forever after in
 manual import mode until someone with admin privileges can kick the
 machine.  This is a pretty bad failure mode; but it's a failure because the
 importer can't cope with changes to the branch, not because automatic
 importing was being done.

 And one is free to push pristine-tar (if makes sense/easy to
 generate), and/or any other branches into the repository (git-dpm,
 git-quilt, etc)

 I would have expected dgit to support pristine-tar
 directly/automatically/unconditionally.  Any system that requires me to
 download the same information (== the upstream source) both from a VCS
 repository and the archive in order to get a fully-formed source package for
 upload is a non-starter.


if pristine-tar can recreate the tarball, without size penalties.
Since it's just a git repository, one can do $ pristine-tar commit and
push that into the dgit repository.
At the moment it's not a requirement.

 I am exited about dgit, as for the first time it will be possible for
 derivatives to centrally share history with Debian.

 I am as well!

 In practice one doesn't actually care how far back the history goes,
 as the history that is interesting is where developers get to do
 intermediate commits between the two uploads to granulise the
 changes

 I don't agree with this at all.  I have regularly made use of the UDD
 branches to examine the history of packages (and not just the Ubuntu
 history, but also the Debian history).  Being upload-level granularity makes
 it less useful than if it were at the granularity of a VCS branch being
 committed to natively by the developer, but it's still *very* useful for
 understanding the uploader's thought process at the time a change was made.

 The fact that git will allow us to graft the developer's own VCS on to these
 dgit repositories in a way that UDD never did is an important improvement,
 but as this is *optional*, not importing the package history from the
 archive would make dgit much less useful for the common case than UDD is
 today.


Ok. Given that we have snapshots.debian.org  graft points we can
create import level history in retrospect.
But I find merging existing history more interesting: either upstream,
or existing git/svn packaging repositories.
For new packages with dgit, I start my repository on top of upstream
git reposity, which gives full history of the project in the dgit
repository.
Imho shared history with upstream projects is more interesting than
debian packaging upload history. Ideally one would have both =)

Regards,

Dmitrijs.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/canbhluhypp-aaq-k1oihougwok+itswxkq2fsjl0qdk9gmf...@mail.gmail.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Ian Jackson
Steve Langasek writes (Re: Introducing dgit - git integration with the Debian 
archive):
 I don't think removing the automatic importer is an improvement at all. 

I agree.  Except that I wanted something I could deploy and use
immediately.  Providing an automatic importer would involve
suddenly dumping the whole archive into alioth, which is not something
I should do without consultation.  (It looks like the alioth admins
are very busy or away right now.)

However, dgit's design certainly doesn't forbid having an automatic
importer.  Indeed there's a nice hole where the automatic importer
would sit, and most of the code necessary is already present.

 If we want VCS branches for the whole of Debian that we can rely on,
 something / someone needs to update them automatically when a package is
 uploaded.

Quite so.

 The problems with the UDD automatic importer have all been around its
 failing to cope with any kind of manual changes to the bzr branch.  I.e.,
 if even once the importer sees an upload before it sees the corresponding
 commit on the bzr branch - because a maintainer did a bzr push --overwrite,
 or because of a race between the upload and the branch propagation, or
 because of a bug on the server - then that package is forever after in
 manual import mode until someone with admin privileges can kick the
 machine.  This is a pretty bad failure mode; but it's a failure because the
 importer can't cope with changes to the branch, not because automatic
 importing was being done.

dgit suites can have an analogous error state.  I don't expect it to
arise in practice.

  And one is free to push pristine-tar (if makes sense/easy to
  generate), and/or any other branches into the repository (git-dpm,
  git-quilt, etc)
 
 I would have expected dgit to support pristine-tar
 directly/automatically/unconditionally.  Any system that requires me to
 download the same information (== the upstream source) both from a VCS
 repository and the archive in order to get a fully-formed source package for
 upload is a non-starter.

I'm afraid you'll have to wait for dgit to be enhanced to treat
.orig tarballs specially, then.

I agree that having do download what amounts to the same data twice is
suboptimal.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21018.22268.542723.672...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Szalay Attila
Hi All,

On szo, 2013-08-24 at 17:40 +0100, Ian Jackson wrote:
 Raphael Hertzog writes (Re: Introducing dgit - git integration with the 
 Debian archive):
  This is wrong on so many levels...
 
 I don't think we are going to agree.  I stand by the description in
 the dgit manpage.

I agree with you, that quilt format is not as intuitive and
straightforward as it should be.

But I'm afraid that if we try to do it in any better way we will invite
the quilt format. :(

I see the following goals of the quilt format:

1. The orig.tar.gz should not be touched in any way.
2. ANY patch that touch the upstream files (ie. files outside of debian
directory) should be easily reproducible, recognisable and well
described
3. This patches ideally will be merged upstream, so drooping one of it
should be easy.
4. Of course if upstream change something we should follow. So this
patches should be easily changeable too.


Anyway.

I like this format very much because with this format the debian
directory in the orig.tar.gz is uninteresting. And with this mode I can
easily follow the upstream changes and merge it to the master branch
without fearing of conflicts. And I'm able to update all my patches in a
separate branch and merge it back to the master branch when it needed.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1377463197.3161.14.camel@mochrul.balabit



Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Steve Langasek
On Sun, Aug 25, 2013 at 08:11:56PM +0100, Ian Jackson wrote:
 Steve Langasek writes (Re: Introducing dgit - git integration with the 
 Debian archive):
  I don't think removing the automatic importer is an improvement at all. 

 I agree.  Except that I wanted something I could deploy and use
 immediately.  Providing an automatic importer would involve
 suddenly dumping the whole archive into alioth, which is not something
 I should do without consultation.  (It looks like the alioth admins
 are very busy or away right now.)

 However, dgit's design certainly doesn't forbid having an automatic
 importer.  Indeed there's a nice hole where the automatic importer
 would sit, and most of the code necessary is already present.

Right.  Getting this going manually is an important first stepping stone on
the path; I just wanted to make sure we weren't stopping short of the real
goal.

If anything, the problems with the UDD automatic importer can be summarized
as: the importer is /not automatic enough/.

   And one is free to push pristine-tar (if makes sense/easy to
   generate), and/or any other branches into the repository (git-dpm,
   git-quilt, etc)

  I would have expected dgit to support pristine-tar
  directly/automatically/unconditionally.  Any system that requires me to
  download the same information (== the upstream source) both from a VCS
  repository and the archive in order to get a fully-formed source package for
  upload is a non-starter.

 I'm afraid you'll have to wait for dgit to be enhanced to treat
 .orig tarballs specially, then.

That's fine, I don't mind waiting, just as long as this is on the roadmap.
:-)

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Steve Langasek
On Sun, Aug 25, 2013 at 05:56:00PM +0100, Dmitrijs Ledkovs wrote:
  In practice one doesn't actually care how far back the history goes,
  as the history that is interesting is where developers get to do
  intermediate commits between the two uploads to granulise the
  changes

  I don't agree with this at all.  I have regularly made use of the UDD
  branches to examine the history of packages (and not just the Ubuntu
  history, but also the Debian history).  Being upload-level granularity
  makes it less useful than if it were at the granularity of a VCS branch
  being committed to natively by the developer, but it's still *very*
  useful for understanding the uploader's thought process at the time a
  change was made.

  The fact that git will allow us to graft the developer's own VCS on to
  these dgit repositories in a way that UDD never did is an important
  improvement, but as this is *optional*, not importing the package
  history from the archive would make dgit much less useful for the common
  case than UDD is today.

 Ok. Given that we have snapshots.debian.org  graft points we can
 create import level history in retrospect.

Or perhaps it would be better to actually import into dgit directly from
UDD?  Since UDD has a more or less complete history of all packages in
Debian, probably more extensive than even what we can assemble from
snapshots.debian.org (thinking back to a snapshots hardware failure a few
years ago).  This won't get all the newer package history due to UDD
importer failures, but it might be a good starting point.

 But I find merging existing history more interesting: either upstream,
 or existing git/svn packaging repositories.

Absolutely; but that can only be done on a per-package basis, and will only
happen for those packages where the maintainers are already bought into the
model and are willing to do the work themselves.

 Imho shared history with upstream projects is more interesting than
 debian packaging upload history. Ideally one would have both =)

Yes.  This was always the ultimate target for UDD, the implementation just
fell short of the ideal.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Joey Hess
Steve Langasek wrote:
 I would have expected dgit to support pristine-tar
 directly/automatically/unconditionally.  Any system that requires me to
 download the same information (== the upstream source) both from a VCS
 repository and the archive in order to get a fully-formed source package for
 upload is a non-starter.

AFAIK you don't actually need the bit-identical .orig.tar.gz to build
the  -1 uploads of a package. You only need a tarball that contains the
identical files as the .orig.tar.gz. Which can be generated from a
tag or branch with the same contents. Perhaps dgit should arrange for
that to be checked in automatically the same way it checks in the
debinized source tree?

(Having such a ref in git would be more or less a prerequisite for using
pristine-tar in an efficient manner anyway.)

-- 
see shy jo


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-25 Thread Scott Kitterman
On Sunday, August 25, 2013 14:08:31 Steve Langasek wrote:
 Or perhaps it would be better to actually import into dgit directly from
 UDD?  Since UDD has a more or less complete history of all packages in
 Debian, probably more extensive than even what we can assemble from
 snapshots.debian.org (thinking back to a snapshots hardware failure a few
 years ago).  This won't get all the newer package history due to UDD
 importer failures, but it might be a good starting point.

UDD (the Ubuntu one) is outside the trust boundary for the Debian project.  
I'm not sure how much effort would be saved once the repository contents were 
bounced against a trusted source for correctness.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Raphael Hertzog
Hi,

On Fri, 23 Aug 2013, Ian Jackson wrote:
  Well, by default step 3 fails if you have upstream changes.
 
 I'm not sure what you mean by upstream changes.  Do you mean changes
 outside the Debian directory ?

Yes. This was discussed on debian-devel and it was considered to be a
better default behaviour.

Discussion starts at:
http://lists.debian.org/20110529085303.ga17...@rivendell.home.ouaza.com

 If so I wasn't aware of it.  Perhaps I
 didn't read the manpage clearly enough.  I think that is also broken.
 It seems to me that the most basic requirement of an archive source
 format is that the four operations I describe above should work and
 DTRT.

Well doing the right things with 3.0 (quilt) means (for most people)
to keep the source package tree actually usable with quilt, so actually
recording non-quilt managed changes in a new quilt patch.

 Ie:
  (i) dpkg-source -b should be able to run on any reasonable tree
  (ii) dpkg-source -b should not modify the directory it is run in
  (iii) dpkg-source -x should always produce the same tree as
was fed to dpkg-source -b
 But as I understand you, `3.0 (quilt)' violates these and this 
 deliberate.
 
 If I have understood you, dgit won't work properly if you make the
 wrong kind of change, so I need to either have this fixed, or (more
 likely) to work around it (and bitch some more in the manpage).  Does
 dpkg-source --commit make any changes other than to quilt metadata ?
 Perhaps dgit push should do that automatically.

dpkg-source --commit adds a new patch in debian/patches/, it adds the
corresponding quilt metadata, and it might also modify
debian/source/include-binaries in case you have (modified/new) binary
files that can't be represented in a quilt patch.

I agree that it should be called automatically before trying to build
the source package. At least for formats 2.0 and 3.0 (quilt), it
doesn't fail with 1.0 but does nothing except spitting out a small
message “'dpkg-source --commit' is not supported by the source format
'1.0'”.

But you should be aware that it's an interactive command, it will ask for
the patch name (thought this can be avoided by feeding it on the command
line) and it will run sensible-editor to let the user edit the DEP-3
metadata on top of the generated patch.

OTOH, if you use --auto-commit, you get back to the old behaviour and
dpkg-source -b will work and create the new patch non-interactively.
(aka dpkg-buildpackage --source-option=--auto-commit -S)

  You put single-debian-patch in debian/source/local-options (or …/options
  if you want to keep it in the generated source package).
 
  (= with local-options this is another case of something that you can
  have in your tree and that you won't have in the unpack of the generated
  source package)
 
 It seems to me that it is the maintainer who determines the source
 package format.  Therefore putting this in local-options makes no
 sense.

On the contrary, the maintainer uses a VCS based workflow and generates a
single patch on every upload. But then if someones does an NMU, you'd
rather have dpkg-source generate a separate patch instead of updating the
maintainer's monolothic patch. Putting it in local-options (in the git
repository) achieves exactly that.

  Are those commits replacing all the files or are they actually a merge of
  changes from upload_n-1 to upload_n in the git repository?
 
 Each upload is represented as a new root commit whose tree contains
 the files which are the result of dpkg-source -x.  This is then
 merged into the existing head of the suite's branch in dgit-repos with
 a pseudo-merge (ie, a merge which takes content only from one of the
 parents).  The dgit-repos suite branches are fast-forwarding.

And the content is taken from the parent which represents the uploads? So
basically you overwrite any other changes that might have been done since
the last upload? Otherwise there's no point in that pseudo-merge if it
doesn't grab the changes made by this upload.

So basically the difference with git-buildpackage is:
- the uploads are represented by pseudo-merges from root commits
  (whereas with git-import-dsc you get non-deterministic commits
  because they are made on top of the upstream branch with a varying
  history)
- you have a centralized repository for all packages
- you define some default branches that track the centralized repository
- you track in the source package the commit from which it has been
  generated

Are there any other important differences?

It seems to me that the value is really in the first feature because it
allows merges to stop at the first common upload available in the history
and thus work even though the repositories might sensibly differ.

The centralization is a nice bonus but the fact that it's required is
somewhat problematic, I don't see the point in duplicating data in another
git repository when all my packages are already using git (and often in
collab-maint so all DD have write access). It would 

Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Raphael Hertzog
On Fri, 23 Aug 2013, Ian Jackson wrote:
 Bastien ROUCARIES writes (Re: Introducing dgit - git integration with the 
 Debian archive):
  Le 22 août 2013 21:52, Ian Jackson ijack...@chiark.greenend.org.uk a
  écrit :
   I'm pleased to announce that dgit 0.7, which is a version of dgit
   suitable for alpha and beta testers, is available in unstable.
  
  What are the pro/con and différence compared to gitpkg ?
 
 gitpkg seems to be something entirely different.  dgit tries to make
 the Debian archive look like a git repository.

Well, technically, git-buildpackage and gitpkg both allow you to import
existing source packages in a Git repository so that you can always
do that first, work in it and then generate the source package to upload.

Obviously dgit add some value to make this less of a pain and to make
actual collaboration among multiple persons possible, but it wasn't
obvious to me either, that's why I asked you some more questions
in the other parts of this thread.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130824084211.gd5...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
(reordered slightly)

Thanks for the information.

Raphael Hertzog writes (Re: Introducing dgit - git integration with the Debian 
archive):
 On Fri, 23 Aug 2013, Ian Jackson wrote:
  Ie:
   (i) dpkg-source -b should be able to run on any reasonable tree
   (ii) dpkg-source -b should not modify the directory it is run in
   (iii) dpkg-source -x should always produce the same tree as
 was fed to dpkg-source -b
  But as I understand you, `3.0 (quilt)' violates these and this 
  deliberate.
...
 Well doing the right things with 3.0 (quilt) means (for most people)
 to keep the source package tree actually usable with quilt, so actually
 recording non-quilt managed changes in a new quilt patch.

IMO the design should have been that the quilt user would do this
after having unpacked the package.  I still think it's fair to
describe the failure to maintain the properties (i)..(iii) above as
braindamage.

But never mind, I will work around it.

  If I have understood you, dgit won't work properly if you make the
  wrong kind of change, so I need to either have this fixed, or (more
  likely) to work around it (and bitch some more in the manpage).  Does
  dpkg-source --commit make any changes other than to quilt metadata ?
  Perhaps dgit push should do that automatically.
 
 dpkg-source --commit adds a new patch in debian/patches/, it adds the
 corresponding quilt metadata, and it might also modify
 debian/source/include-binaries in case you have (modified/new) binary
 files that can't be represented in a quilt patch.

I will hope that the user who doesn't care about the innards of
the `3.0 (quilt)' format doesn't care about all of that.

 But you should be aware that it's an interactive command, it will ask for
 the patch name (thought this can be avoided by feeding it on the command
 line) and it will run sensible-editor to let the user edit the DEP-3
 metadata on top of the generated patch.

I will RTFM to use it in some noninteractive way by generating the
arguments automatically.

 And the content is taken from the parent which represents the uploads? So
 basically you overwrite any other changes that might have been done since
 the last upload? Otherwise there's no point in that pseudo-merge if it
 doesn't grab the changes made by this upload.

I think you're confused.  We are talking here about the import from
the archive, not the upload to it.

But it is true that if you make an upload you overwrite the changes
made in any recent uploads you haven't explicitly merged into what
you're uploading.  I think this is a fundamental misfeature of the
archive.

But dgit users won't overwrite each others' changes, because they are
pushed to the fast-forwarding dgit suite branch before upload.

 So basically the difference with git-buildpackage is:
...
 Are there any other important differences?

dgit is a lot simpler than git-buildpackage.  I think the manpage
should answer this questions.  I'm not really qualified to talk about
git-buildpackage in detail.

 The centralization is a nice bonus but the fact that it's required is
 somewhat problematic, I don't see the point in duplicating data in another
 git repository when all my packages are already using git (and often in
 collab-maint so all DD have write access). It would be nice if it could
 rely on the Vcs-Git fields available in the archive and then optionally
 work on those repositories instead of dgit.debian.net.

The central repositories are fundamental to the design.

I chose not to use collab-maint because that would have involved
getting agreement from everyone who uses collab-maint that it was OK
for dgit to create its branches and tags there.  If you want to
combine dgit-repos and collab-maint, that would certainly be
technically possible.  Please go away and get the required consensus
and get back to me and I will make it so :-).

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21016.51167.494885.568...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the
Debian archive):
 [Raphael:]
  dpkg-source --commit adds a new patch in debian/patches/, it adds the
  corresponding quilt metadata, and it might also modify
  debian/source/include-binaries in case you have (modified/new) binary
  files that can't be represented in a quilt patch.
...
  But you should be aware that it's an interactive command, it will ask for
  the patch name (thought this can be avoided by feeding it on the command
  line) and it will run sensible-editor to let the user edit the DEP-3
  metadata on top of the generated patch.
 
 I will RTFM to use it in some noninteractive way by generating the
 arguments automatically.

stable's dpkg-source doesn't even have this damned option!

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21016.51836.226784.436...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
archive):
 stable's dpkg-source doesn't even have this damned option!

Sorry, my mistake.  I was tricked by #708163.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21016.51913.62734.580...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Niels Thykier writes (Re: Introducing dgit - git integration with the Debian 
archive):
 If you promise to add autopkgtest for it, you can use any of my
 packages[1]. :P
...
 [1] Any of which I am the sole maintainer.  E.g. mscgen.

I took you up on this.  I'll send the autopkgtest diff to the BTS.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21016.52040.32108.182...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Raphael Hertzog
On Sat, 24 Aug 2013, Ian Jackson wrote:
  Well doing the right things with 3.0 (quilt) means (for most people)
  to keep the source package tree actually usable with quilt, so actually
  recording non-quilt managed changes in a new quilt patch.
 
 IMO the design should have been that the quilt user would do this
 after having unpacked the package.

This is wrong on so many levels...

Where would we record the changes made outside of the debian directory?
If you don't want to feed them back in the debian directory as new quilt
patch, then you're forced to treat them like we do with binary files aka
add them in the .debian.tar.gz even though they're outside of the debian
directory.  The goal of 3.0 (quilt) was to always the represent the
source package as a debian directory + a stack of patches to apply on the
.orig.tar. Refusing to create a new quilt patch for unmanaged changes
would break this principle.

It's not only about convenience for Debian, it's also about doing the
right thing wrt upstream (and anybody who peeks at our source packages) by
always having all our changes as a set of patches that do apply on the
latest upstream releases.

  But you should be aware that it's an interactive command, it will ask for
  the patch name (thought this can be avoided by feeding it on the command
  line) and it will run sensible-editor to let the user edit the DEP-3
  metadata on top of the generated patch.
 
 I will RTFM to use it in some noninteractive way by generating the
 arguments automatically.

You can't do that with dpkg-source --commit. But if you don't want the
interactive process, just use --auto-commit when you build the source
package.

That said we could enhance dpkg-source --commit to not run the editor when
stdin is not a tty.

  And the content is taken from the parent which represents the uploads? So
  basically you overwrite any other changes that might have been done since
  the last upload? Otherwise there's no point in that pseudo-merge if it
  doesn't grab the changes made by this upload.
 
 I think you're confused.  We are talking here about the import from
 the archive, not the upload to it.

That's what I meant, anything that is in the archive has been uploaded
before so it's also an upload, just not the same ;-)

 But it is true that if you make an upload you overwrite the changes
 made in any recent uploads you haven't explicitly merged into what
 you're uploading.  I think this is a fundamental misfeature of the
 archive.

You could avoid this enforcing a dgit pull before dgit push?

I was more thinking of this scenario:

1/ I clone the repository at a time where the archive has 1.0-1
2/ I add some commit/changes
3/ Someone else uploads 1.0-2
4/ I import the 1.0-2 source archive (with dgit pull)

But I guess that in fact this scenario is fine since the branch where that
pseudo-merge happens is the (remoted) dgit one, not the one where I'm working.

And I guess that you're not allowed to push anything in the dgit/*
branches (on dgit.debian.net) except when you do a real upload. Thus if
multiple people have to collaborate to prepare the next revision, they
must do so on another branch. The dgit branch cannot be used to accumulate
changes in preparation of the next upload.

Is that correct?

 The central repositories are fundamental to the design.

I understand that, it's still a pity that a maintainer can't indicate that
his usual repository should be used as the central dgit repository for
this package.

In particular when moving to dgit-repos (which is the only solution if he
wants to avoid the duplication) implies loosing the possibility to
hand out access to non-DD (which collab-maint makes possible).

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130824161644.ga10...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Raphael Hertzog writes (Re: Introducing dgit - git integration with the Debian 
archive):
 This is wrong on so many levels...

I don't think we are going to agree.  I stand by the description in
the dgit manpage.

  I will RTFM to use it in some noninteractive way by generating the
  arguments automatically.
 
 You can't do that with dpkg-source --commit. But if you don't want the
 interactive process, just use --auto-commit when you build the source

I have found that by setting EDITOR this can be made to work.  I wrote
a shonky thing to edit the patch description into something halfway
reasonable.  (I was tempted by EDITOR=true but maintainers would hate
the resulting patches.)

Did you know that dpkg-source doesn't notice if sensible-editor exits
nonzero ?

  But it is true that if you make an upload you overwrite the changes
  made in any recent uploads you haven't explicitly merged into what
  you're uploading.  I think this is a fundamental misfeature of the
  archive.
 
 You could avoid this enforcing a dgit pull before dgit push?

No, not really.  I do that already, but there is still a race.

 I was more thinking of this scenario:
 
 1/ I clone the repository at a time where the archive has 1.0-1
 2/ I add some commit/changes
 3/ Someone else uploads 1.0-2
 4/ I import the 1.0-2 source archive (with dgit pull)
 
 But I guess that in fact this scenario is fine since the branch where that
 pseudo-merge happens is the (remoted) dgit one, not the one where I'm working.

If this happens you can just say dgit pull, or dgit fetch followed
by an explicit merge, and the right thing will happen.  dgit will
prevent you from accidentally overwriting the 1.0-2 changes - but only
if the 1.0-2 changes either (a) were made with dgit (b) have shown up
in your mirror.

 And I guess that you're not allowed to push anything in the dgit/*
 branches (on dgit.debian.net) except when you do a real upload.

Those branches aren't in refs/heads/ precisely to stop git pushing
them by itself.

  Thus if multiple people have to collaborate to prepare the next
 revision, they must do so on another branch. The dgit branch cannot
 be used to accumulate changes in preparation of the next upload.
 
 Is that correct?

Doing that isn't a good idea, indeed.  You can do this with your local
dgit/suite branch but the remote suite tracking branches track
uploads only.  You can make a refs/heads/suite branch if you want to
share your work.

 In particular when moving to dgit-repos (which is the only solution if he
 wants to avoid the duplication) implies loosing the possibility to
 hand out access to non-DD (which collab-maint makes possible).

Maybe there should be a way to make the dgit-repos tree a symlink to
the collab-maint tree.  Of course doing that would make it impossible
ever to sever the two services.

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21016.57824.130779.683...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
archive):
 I took you up on this.  I'll send the autopkgtest diff to the BTS.

It turns out that I lied and the thing did go to unstable :-/.

I will replace it with something looking like a reasonable NMU.
(The version in unstable is not broken; it just has a pointless change
to the upstream source.)

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21016.58930.322877.810...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
archive):
 It turns out that I lied and the thing did go to unstable :-/.
 
 I will replace it with something looking like a reasonable NMU.
 (The version in unstable is not broken; it just has a pointless change
 to the upstream source.)

Now done.  I have also added a safety catch to prevent bugs like this
from causing damage in the future.  (The bug was in an unreleased
version, so hasn't caught anyone else.)

I hope it's not too much of a mess, and that the autopkgtest is worthb
it :-).

Regards,
Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21016.61945.962874.232...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Ian Jackson
Ian Jackson writes (Introducing dgit - git integration with the Debian 
archive):
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.

I've just uploaded 0.9.  Thanks for the offers of packages for
testing.  I think it works better with qulity packages now.

Unfortunately there is a problem that means it's difficult for others
to test.  The alioth git directory is not writeable by the right
group.  I think what's needed is this:
  sudo setfacl -m group:Debian:rwx default:group:Debian:rwx /git/dgit-repos

I've asked admin@alioth but not had any reply yet...

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21016.62343.2720.421...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Joey Hess
Ian Jackson wrote:
 Maybe there should be a way to make the dgit-repos tree a symlink to
 the collab-maint tree.  Of course doing that would make it impossible
 ever to sever the two services.

I suspect you can manually remove the collab-maint repository and
symlink it to the dgit-repos repository (after already pushing all the
collab-maint repository contents to it of course). 

Since dgit is already making repositories that are writable by
scm_collab-qa, everyone who could write to a collab-maint repository
before should still have write access.

I look forward to trying that out once the permissions are finally fixed
so I can dgit push new repos ...

-- 
see shy jo


signature.asc
Description: Digital signature


Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Guillem Jover
On Sat, 2013-08-24 at 17:40:00 +0100, Ian Jackson wrote:
 Did you know that dpkg-source doesn't notice if sensible-editor exits
 nonzero ?

Nope, thanks I've just fixed this locally now, will be included in
dpkg 1.17.2.

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130824210841.ga21...@gaara.hadrons.org



Re: Introducing dgit - git integration with the Debian archive

2013-08-24 Thread Raphael Hertzog
Hi,

On Sat, 24 Aug 2013, Ian Jackson wrote:
 Did you know that dpkg-source doesn't notice if sensible-editor exits
 nonzero ?

I saw it when looking up the exact details to write my former reply.  But
since at that point all the changes are already applied (and the editor
only allows you to change the automatic header in something more sensible),
I'm not sure what a proper failure would bring.

If we want to abort the whole commit operation, like git would do, it
will need a lot more work for very little gain IMO.

  And I guess that you're not allowed to push anything in the dgit/*
  branches (on dgit.debian.net) except when you do a real upload.
 
 Those branches aren't in refs/heads/ precisely to stop git pushing
 them by itself.

Oh, this is pretty confusing... because we do have dgit/suite branches
in refs/heads/ after a dgit clone.

What you mean is that on the server side the dgit-branches containing the
uploads are in refs/dgit/ and not in refs/heads/. But there will also
be refs/heads/dgit/suite branches on the server side since that's where
people will naturally push stuff:

$ dgit clone dgit
$ cat .git/HEAD
ref: refs/heads/dgit/sid
$ cat .git/config
…
[remote dgit]
fetch = +refs/dgit/*:refs/remotes/dgit/dgit/*
[remote origin]
url = git+ssh://git.debian.org/git/dgit-repos/dgit.git
fetch = +refs/heads/*:refs/remotes/origin/*
$ git push --dry-run
To git+ssh://git.debian.org/git/dgit-repos/dgit.git
 * [new branch]  dgit/sid - dgit/sid

So there's a refs/head/dgit/suite and it can be pushed to the server.
But it will be different to the refs/dgit/suite that gets merged by
dgit pull...

Speaking of braindamage... :-)

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130824222618.ga13...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Raphael Hertzog
Hello Ian,

thanks for your work on dgit. It looks very promising!

I'm sorry to have missed that impromptu BOF at debconf, I would have
definitely arranged myself to be there if I had known its existence.

On Thu, 22 Aug 2013, Ian Jackson wrote:
 With regret I must note that currently you can't use it if you don't
 have a full DD account on both the Debian systems and on Alioth.  Even
 if you just want a read-only mode.  This is due to deficiencies in the
 archive software and in Alioth, which I have worked around in very
 unpleasant ways.  I'm hoping that this will improve fairly soon.
 Please see the BUGS section of the manpage.

“For `3.0 (quilt)' packages, dgit has to do more work to work around some
braindamage in way dpkg-source handles changes made to this format. See also
the  BUGS  section.   We recommend against the use of `3.0 (quilt)'.”

“`3.0  (quilt)'  packages have an additional difficulty: if these
are edited in the most normal way, and then fed to dpkg-buildpackage,
dpkg-source will add extra quilt patch metadata to  the  source  tree
during  the source  package build.  This extra metadata is then of course
not included in the git history.  So dgit push needs to commit it for you,
to make sure that the git history and archive contents are identical.
That this is necessary is a bug in the `3.0 (quilt)' format.”

I don't think that 3.0 (quilt) does things without justification so
calling those features braindamage is not really nice. Clearly the
quilt format duplicates some of the VCS features (with its .pc directory)
and this is probably what you are referring to in the above paragraph.

That said I have always thought of the 3.0 (quilt) source package as
something that we should be able to generate ouf of a VCS tree, not
something to be directly stored in a VCS repository.

In any case, I'm open to suggestions to improve the format to better suit
your needs... so can you can explain me more precisely what's causing you
issues? Either here or better in a bug report against dpkg-dev.

I would really like to find a good workflow that suits 3.0 (quilt).
Badly supporting half the archive looks like a problem. 

BTW, instead of advising against 3.0 (quilt) you should rather recommend
the usage of the --single-debian-patch option when using that format...
this will make it mostly work like 1.0.

I have read the manual page but there some things that I don't really
understand on the design yet. So let me ask some questions:

- is there some server side part (running on alioth)?
  what does it take care of?

- what/who creates the initial repository? and what/who keeps it in sync with
  the (non-dgit-based) archive uploads?

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130823074507.gb8...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Dmitrijs Ledkovs
On 23 August 2013 00:38, Charles Plessy ple...@debian.org wrote:
 Le Thu, Aug 22, 2013 at 08:52:10PM +0100, Ian Jackson a écrit :
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.

 From the manpage:

dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]

dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git
view of the package can contain the usual upstream git history,
and will be augmented by commits representing uploads done by
other developers not using dgit.  This git history is stored in
a canonical location known as dgit-repos which lives outside
the Debian archive (currently, on Alioth).

 Thanks a lot for this development !

 For the packages that I maintain with Git, I commit build logs (the local one
 for the uploaded binary packages, plus the buildd ones) in separate branches.
 In some cases I found it quite useful.  Have you considered integrating logs 
 in
 dgit ?  In a somehow similar goal (finding difference between builds), have 
 you
 also considered committing the contents of the unpacked binary packages in
 other branches ?


Apart from designated release dgit branches, it's a normal git
repository into which one can push whatever one wants: pristine-tar,
various git/quilt patch management branches, build-logs, upstream
branches et. al.

Regards,

Dmitrijs.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/canbhluhxyz9qkbbbwsvejcbsre9azss5e5ymgpcnpqkxpie...@mail.gmail.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Svante Signell
On Fri, 2013-08-23 at 10:15 +0100, Dmitrijs Ledkovs wrote:

 Apart from designated release dgit branches, it's a normal git
 repository into which one can push whatever one wants: pristine-tar,
 various git/quilt patch management branches, build-logs, upstream
 branches et. al.

Hi, a somewhat related question: Anybody using guilt (qit+quilt)for
patch management?



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1377251889.4078.5.ca...@g3620.my.own.domain



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Ian Jackson
Raphael Hertzog writes (Re: Introducing dgit - git integration with the Debian 
archive):
 I'm sorry to have missed that impromptu BOF at debconf, I would have
 definitely arranged myself to be there if I had known its existence.

Yes, sorry about that, but it was arranged ad hoc at zero notice.

 I don't think that 3.0 (quilt) does things without justification so
 calling those features braindamage is not really nice. Clearly the
 quilt format duplicates some of the VCS features (with its .pc directory)
 and this is probably what you are referring to in the above paragraph.

The thing that really bothers dgit is that it is not always possible
to round trip a tree through dpkg-source.  (And the case where it
doesn't work is the common one.)

That is, if I do this:
  1. dpkg-source -x
  2. edit things 
  3. dpkg-source -b
  4. dpkg-source -x on the output from stage 3
then the output of step 4 is not always the same as the input
to step 3.  Typically the output of step 4 has additional metadata
files inside the tree.

It seems to me that the most fundamental requirement for an archive
format of any kind is that packing something into the archive, and
then unpacking it again, should give you back what you started with.

 That said I have always thought of the 3.0 (quilt) source package as
 something that we should be able to generate ouf of a VCS tree, not
 something to be directly stored in a VCS repository.

With dgit, at the moment, all that happens is that the gitish
changes get piled into the dpkg-source-generated patch.

For some more sophisticated workflow, it is necessary to round trip
the patch stack through the archive and back into git.

 In any case, I'm open to suggestions to improve the format to better suit
 your needs... so can you can explain me more precisely what's causing you
 issues? Either here or better in a bug report against dpkg-dev.

Ideally, packing something up and then unpacking it again should not
produce changes.  NB that if you change this you absolutely must not
change the meaning of existing source archives - ie you can make
changes to the packing algorithm but not to transformation implied by
the unpack algorithm.

 I would really like to find a good workflow that suits 3.0 (quilt).
 Badly supporting half the archive looks like a problem. 

The support is no worse than NMUs are now, from the maintainer's POV.
From the git POV you just see a git history of some kind and commit on
it.

 BTW, instead of advising against 3.0 (quilt) you should rather recommend
 the usage of the --single-debian-patch option when using that format...
 this will make it mostly work like 1.0.

This is a command-line option ?  How does one specify in the source
package that this is to be used.

 I have read the manual page but there some things that I don't really
 understand on the design yet. So let me ask some questions:
 
 - is there some server side part (running on alioth)?
   what does it take care of?

Yes.  It is a git repository.  That is all that it is.

 - what/who creates the initial repository? and what/who keeps it in sync with
   the (non-dgit-based) archive uploads?

The repos are created only by dgit users.  Different dgit users see
the same commits from the archive: the synthesised commits are
deterministic.

In the future there might be a robot keeping it in sync with the
archive.

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21015.18345.930607.890...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Raphael Hertzog
Hi,

On Fri, 23 Aug 2013, Ian Jackson wrote:
 The thing that really bothers dgit is that it is not always possible
 to round trip a tree through dpkg-source.  (And the case where it
 doesn't work is the common one.)
 
 That is, if I do this:
   1. dpkg-source -x
   2. edit things 
   3. dpkg-source -b
   4. dpkg-source -x on the output from stage 3
 then the output of step 4 is not always the same as the input
 to step 3.  Typically the output of step 4 has additional metadata
 files inside the tree.

Well, by default step 3 fails if you have upstream changes. Unless you
use --auto-commit or --single-debian-patch. So the logical thing to do
is to call dpkg-source --commit . patch-name when you know that you
have upstream changes compared to the last debian package.

I believe that after the --commit you have the same metadata that you
would have after step 4.

 For some more sophisticated workflow, it is necessary to round trip
 the patch stack through the archive and back into git.

I'm not following you here. Can you elaborate ? (round-trip through the
archive doesn't mean anything obvious to me)

What I would like to have with 3.0 (quilt) source packages is a git
repository with the patches applied but without the .pc/ quilt dir (that's
just noise). We would probably have a debian/source/local-options to
indicate to dpkg-source that the patches are already applied and that
it should deal with it.

Then I add commits like usual and they would be automatically
converted to new quilt patches in a later step when I tell dgit
to build the source. (Bonus point: we could add some metadata
in commit notices to merge the change in a existing quilt patch)

The really tricky part is how to update the patches when you merge
a new upstream version that changes the underlying code so that the
patches no longer apply. See http://bugs.debian.org/572204 for how Colin
Watson deals with this. I think he uses bzr and I'm not sure if we
can force git to proceed with the merge if we have local uncommitted
changes.

Basically, it would be good enough I believe if we can approximate this
with:
1/ a commit that reverts the Debian patches
2/ a commit that merges the upstream changes + reapplies Debian patches
   (git merge --no-commit + quilt push -a with the required human fixes)
   Bonus point if your replace the quilt push -a with git cherry-pick of
   all patches and automatic refresh of the patches.

  BTW, instead of advising against 3.0 (quilt) you should rather recommend
  the usage of the --single-debian-patch option when using that format...
  this will make it mostly work like 1.0.
 
 This is a command-line option ?  How does one specify in the source
 package that this is to be used.

You put single-debian-patch in debian/source/local-options (or …/options
if you want to keep it in the generated source package).

(= with local-options this is another case of something that you can
have in your tree and that you won't have in the unpack of the generated
source package)

  - what/who creates the initial repository? and what/who keeps it in sync 
  with
the (non-dgit-based) archive uploads?
 
 The repos are created only by dgit users.  Different dgit users see
 the same commits from the archive: the synthesised commits are
 deterministic.

OK, assuming that they are created on top of the same commit I guess.

Are those commits replacing all the files or are they actually a merge of
changes from upload_n-1 to upload_n in the git repository?

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130823164508.gc30...@x230-buxy.home.ouaza.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Ian Jackson
Raphael Hertzog writes (Re: Introducing dgit - git integration with the Debian 
archive):
 On Fri, 23 Aug 2013, Ian Jackson wrote:
  The thing that really bothers dgit is that it is not always possible
  That is, if I do this:
1. dpkg-source -x
2. edit things 
3. dpkg-source -b
4. dpkg-source -x on the output from stage 3
  then the output of step 4 is not always the same as the input
  to step 3.  Typically the output of step 4 has additional metadata
  files inside the tree.
 
 Well, by default step 3 fails if you have upstream changes.

I'm not sure what you mean by upstream changes.  Do you mean changes
outside the Debian directory ?  If so I wasn't aware of it.  Perhaps I
didn't read the manpage clearly enough.  I think that is also broken.
It seems to me that the most basic requirement of an archive source
format is that the four operations I describe above should work and
DTRT.

Ie:
 (i) dpkg-source -b should be able to run on any reasonable tree
 (ii) dpkg-source -b should not modify the directory it is run in
 (iii) dpkg-source -x should always produce the same tree as
   was fed to dpkg-source -b
But as I understand you, `3.0 (quilt)' violates these and this 
deliberate.

If I have understood you, dgit won't work properly if you make the
wrong kind of change, so I need to either have this fixed, or (more
likely) to work around it (and bitch some more in the manpage).  Does
dpkg-source --commit make any changes other than to quilt metadata ?
Perhaps dgit push should do that automatically.

  For some more sophisticated workflow, it is necessary to round trip
  the patch stack through the archive and back into git.
 
 I'm not following you here. Can you elaborate ? (round-trip through the
 archive doesn't mean anything obvious to me)

I would like to focus on fixing the existing situation before getting
into this discussion, so I'm afraid I'm not going to reply to this
part right now.

  This is a command-line option ?  How does one specify in the source
  package that this is to be used.
 
 You put single-debian-patch in debian/source/local-options (or …/options
 if you want to keep it in the generated source package).

 (= with local-options this is another case of something that you can
 have in your tree and that you won't have in the unpack of the generated
 source package)

It seems to me that it is the maintainer who determines the source
package format.  Therefore putting this in local-options makes no
sense.

  The repos are created only by dgit users.  Different dgit users see
  the same commits from the archive: the synthesised commits are
  deterministic.
 
 OK, assuming that they are created on top of the same commit I guess.

There is an intervening pseudo-merge.  So all dgit users will
create the same root commits with perhaps some different pseudo-merges
if they do dgit fetch at different times and so see different states
of the dgit/suite branch in dgit-repos.

 Are those commits replacing all the files or are they actually a merge of
 changes from upload_n-1 to upload_n in the git repository?

Each upload is represented as a new root commit whose tree contains
the files which are the result of dpkg-source -x.  This is then
merged into the existing head of the suite's branch in dgit-repos with
a pseudo-merge (ie, a merge which takes content only from one of the
parents).  The dgit-repos suite branches are fast-forwarding.

Thanks,
Ian.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21015.38370.906620.363...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Ian Jackson
Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
archive):
...
 If I have understood you, dgit won't work properly if you make the
 wrong kind of change, so I need to either have this fixed, or (more
 likely) to work around it (and bitch some more in the manpage).  Does
 dpkg-source --commit make any changes other than to quilt metadata ?
 Perhaps dgit push should do that automatically.

Does anyway know of a `3.0 (quilt)' format package whose maintainer
wouldn't mind a bunch of crazy (perhaps abortive) NMUs to
experimental, which I can therefore use for testing ?  I promise not
to upload to sid by mistake.

Volunteers welcome :-).

Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/21015.39201.573731.46...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Niels Thykier
On 2013-08-23 19:17, Ian Jackson wrote:
 Ian Jackson writes (Re: Introducing dgit - git integration with the Debian 
 archive):
 ...
 If I have understood you, dgit won't work properly if you make the
 wrong kind of change, so I need to either have this fixed, or (more
 likely) to work around it (and bitch some more in the manpage).  Does
 dpkg-source --commit make any changes other than to quilt metadata ?
 Perhaps dgit push should do that automatically.
 
 Does anyway know of a `3.0 (quilt)' format package whose maintainer
 wouldn't mind a bunch of crazy (perhaps abortive) NMUs to
 experimental, which I can therefore use for testing ?  I promise not
 to upload to sid by mistake.
 
 Volunteers welcome :-).
 
 Ian.
 
 

If you promise to add autopkgtest for it, you can use any of my
packages[1]. :P

~Niels

[1] Any of which I am the sole maintainer.  E.g. mscgen.



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/52179c67.7010...@thykier.net



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Bastien ROUCARIES
Le 22 août 2013 21:52, Ian Jackson ijack...@chiark.greenend.org.uk a
écrit :

 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.

What are the pro/con and différence compared to gitpkg ?

 From the manpage:

dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]

dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git
view of the package can contain the usual upstream git history,
and will be augmented by commits representing uploads done by
other developers not using dgit.  This git history is stored in
a canonical location known as dgit-repos which lives outside
the Debian archive (currently, on Alioth).

 If you don't like git, or think I have taken the wrong approach, then
 you need read no further.  You can carry on just as before.


 If on the other hand you'd like to experiment with a new tool that
 lets you work on any package in the archive using git, even with
 upstream git history in your history if you like, and to share your
 git-based work with other uploaders, please install dgit 0.7 from
 unstable and try it out.

 For more information, the full manpage is here:
   http://www.chiark.greenend.org.uk/~ijackson/2013/dgit.html
 (this is slightly more up to date than the one on manpages.debian.net).

 This is still a very early version.  I'm not aware of anyone but me
 having used it in anger.  So there are very likely to be bugs.  Please
 report them.

 With regret I must note that currently you can't use it if you don't
 have a full DD account on both the Debian systems and on Alioth.  Even
 if you just want a read-only mode.  This is due to deficiencies in the
 archive software and in Alioth, which I have worked around in very
 unpleasant ways.  I'm hoping that this will improve fairly soon.
 Please see the BUGS section of the manpage.

 The master git repository for dgit is here:
   http://anonscm.debian.org/gitweb/?p=dgit-repos/dgit.git
 Pull requests welcome (please send them to the BTS).


 Some background:

 At Debconf we had a series of discussion on the problem of integrating
 git and the archive, culminating in an excellent bar-BOF.  We started
 with a diagram Joey Hess had drawn on a piece of cardboard, of a
 design he had; many of us came to the BOF with our own preconceptions.
 Too often these discussions result in a gigantic edifice which will
 take six-months to implement and six years to persuade people is a
 good idea.  But, during this conversation, instead, pieces were hacked
 off until the result was implementable right away.  So that's what
 dgit is.

 Key points about dgit's design:

  * It doesn't require anyone else to change their existing workflow.

  * You can use it on any package.

  * If you are the maintainer and want to use it on your own package,
it will allow you to adopt a gitish workflow and will let you
easily import into your git history the changes made in NMUs,
security updates - and hopefully eventually derivative distros.

  * I think we can extend it later to support working on quilty
packages (with patch stacks) as if they were git branches.  This is
not yet done because it's quite complicated.  See PACKAGE SOURCE
FORMATS in the manpage.


 Thanks,
 Ian.


 --
 To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
 with a subject of unsubscribe. Trouble? Contact
listmas...@lists.debian.org
 Archive:
http://lists.debian.org/21014.27626.870447.341...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-23 Thread Ian Jackson
Bastien ROUCARIES writes (Re: Introducing dgit - git integration with the 
Debian archive):
 Le 22 août 2013 21:52, Ian Jackson ijack...@chiark.greenend.org.uk a
 écrit :
  I'm pleased to announce that dgit 0.7, which is a version of dgit
  suitable for alpha and beta testers, is available in unstable.
 
 What are the pro/con and différence compared to gitpkg ?

gitpkg seems to be something entirely different.  dgit tries to make
the Debian archive look like a git repository.

Ian.


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21015.42668.430786.342...@chiark.greenend.org.uk



Introducing dgit - git integration with the Debian archive

2013-08-22 Thread Ian Jackson
I'm pleased to announce that dgit 0.7, which is a version of dgit
suitable for alpha and beta testers, is available in unstable.

From the manpage:

   dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
   dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
   dgit [dgit-opts] build|sbuild [build-opts]
   dgit [dgit-opts] push [dgit-opts] [suite]

   dgit treats the Debian archive as a version control system, and
   bidirectionally gateways between the archive and git.  The git
   view of the package can contain the usual upstream git history,
   and will be augmented by commits representing uploads done by
   other developers not using dgit.  This git history is stored in
   a canonical location known as dgit-repos which lives outside
   the Debian archive (currently, on Alioth).

If you don't like git, or think I have taken the wrong approach, then
you need read no further.  You can carry on just as before.


If on the other hand you'd like to experiment with a new tool that
lets you work on any package in the archive using git, even with
upstream git history in your history if you like, and to share your
git-based work with other uploaders, please install dgit 0.7 from
unstable and try it out.

For more information, the full manpage is here:
  http://www.chiark.greenend.org.uk/~ijackson/2013/dgit.html
(this is slightly more up to date than the one on manpages.debian.net).

This is still a very early version.  I'm not aware of anyone but me
having used it in anger.  So there are very likely to be bugs.  Please
report them.

With regret I must note that currently you can't use it if you don't
have a full DD account on both the Debian systems and on Alioth.  Even
if you just want a read-only mode.  This is due to deficiencies in the
archive software and in Alioth, which I have worked around in very
unpleasant ways.  I'm hoping that this will improve fairly soon.
Please see the BUGS section of the manpage.

The master git repository for dgit is here:
  http://anonscm.debian.org/gitweb/?p=dgit-repos/dgit.git
Pull requests welcome (please send them to the BTS).


Some background:

At Debconf we had a series of discussion on the problem of integrating
git and the archive, culminating in an excellent bar-BOF.  We started
with a diagram Joey Hess had drawn on a piece of cardboard, of a
design he had; many of us came to the BOF with our own preconceptions.
Too often these discussions result in a gigantic edifice which will
take six-months to implement and six years to persuade people is a
good idea.  But, during this conversation, instead, pieces were hacked
off until the result was implementable right away.  So that's what
dgit is.

Key points about dgit's design:

 * It doesn't require anyone else to change their existing workflow.

 * You can use it on any package.

 * If you are the maintainer and want to use it on your own package,
   it will allow you to adopt a gitish workflow and will let you
   easily import into your git history the changes made in NMUs,
   security updates - and hopefully eventually derivative distros.

 * I think we can extend it later to support working on quilty
   packages (with patch stacks) as if they were git branches.  This is
   not yet done because it's quite complicated.  See PACKAGE SOURCE
   FORMATS in the manpage.


Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/21014.27626.870447.341...@chiark.greenend.org.uk



Re: Introducing dgit - git integration with the Debian archive

2013-08-22 Thread Matthias Klumpp
2013/8/22 Ian Jackson ijack...@chiark.greenend.org.uk:
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.

 From the manpage:

dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]

dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git
view of the package can contain the usual upstream git history,
and will be augmented by commits representing uploads done by
other developers not using dgit.  This git history is stored in
a canonical location known as dgit-repos which lives outside
the Debian archive (currently, on Alioth).
 [...]
This is awesome! Many thanks for the great work!
 -- Matthias


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caknhny_bj-kp+p4spxfowmkyxfz876jjctliyobgo0ojqj3...@mail.gmail.com



Re: Introducing dgit - git integration with the Debian archive

2013-08-22 Thread Charles Plessy
Le Thu, Aug 22, 2013 at 08:52:10PM +0100, Ian Jackson a écrit :
 I'm pleased to announce that dgit 0.7, which is a version of dgit
 suitable for alpha and beta testers, is available in unstable.
 
 From the manpage:
 
dgit [dgit-opts] clone [dgit-opts] package [suite] [./dir|/dir]
dgit [dgit-opts] fetch|pull [dgit-opts] [suite]
dgit [dgit-opts] build|sbuild [build-opts]
dgit [dgit-opts] push [dgit-opts] [suite]
 
dgit treats the Debian archive as a version control system, and
bidirectionally gateways between the archive and git.  The git
view of the package can contain the usual upstream git history,
and will be augmented by commits representing uploads done by
other developers not using dgit.  This git history is stored in
a canonical location known as dgit-repos which lives outside
the Debian archive (currently, on Alioth).

Thanks a lot for this development !

For the packages that I maintain with Git, I commit build logs (the local one
for the uploaded binary packages, plus the buildd ones) in separate branches.
In some cases I found it quite useful.  Have you considered integrating logs in
dgit ?  In a somehow similar goal (finding difference between builds), have you
also considered committing the contents of the unpacked binary packages in
other branches ?

Have a nice day,

-- 
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130822233849.gb1...@falafel.plessy.net