Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Ghislain Vaillant]

Hi Martin, some pieces of advice below:

On 07/12/15 11:26, Martin Uecker wrote:

The only issue is that I have to specify the tag with 'gdb'
because it looks for 'upstream/v0.2.09' which does not
currently exist. Instead of adding these tags, I would prefer
to reconfigure 'gdb' to use the existing upstream tags without
the prefix 'upstream/'. Then I could just push the regular
upstream tags. Would this be ok?


Just add a debian/gbp.conf [1] file with the following content:

[DEFAULT]
upstream-branch = upstream
debian-branch = master
upstream-tag = v%(version)s
debian-tag = debian/%(version)s
pristine-tar = True

That way you are documenting how your packaging repository is layed out 
explicitly (always a good thing), plus gbp will pick up these options 
over the default ones under $HOME/.gbp.conf. That way, you won't need to 
explicitly specify the tag via --git-upstream-tag anymore.


[1] 
http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/man.gbp.conf.html



I also created a newer upstream tag v0.2.09d specifically
for the initial packaging work because I made some upstream
changes to make packaging easier.  The bart_0.2.09.orig.tar.gz
which is now in the pristine-tar branch actually corresonds
to v0.2.09d.  I hope that is not too messed up, but with the
next upstream version (to be released soon) this hack would
gone.
Since you have yet to submit a first Debian package publicly, you are 
kind of free to do whatever you like with the packaging specific 
branches (pristine-tar, master) of your repository (force push, 
rebase...). Just make sure everything is in order once your package is 
ready for its first submission.


Ghis

Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Gert Wollny]

On Mon, 2015-12-07 at 08:58 +, Ghislain Vaillant wrote:

> You could use something like libgsl0-dev | libgsl-dev to stay 
> compatible with earlier versions?
This should probably be the other way around,  i.e. 

  libgsl-dev | libgsl0-dev 

so that the new package takes precedence. 

Best, 
Gert 

Re: Packaging gwamar for Debian

[Andreas Tille]

Hi,

On Sun, Dec 06, 2015 at 02:33:56PM +0100, Michał Woźniak wrote:
> Hi Andreas,
> I have resolved all the issues listed in your mail. Still, if possible I
> would prefer to have a week more time to clean up console messages to be
> more clear and less confusing.

You have whatever time you need. ;-)

> Please find my answers to your comments one by one:
> 
>  0. fixed, I attach version number to each zip file when building (the
> current version is 1.15.1)

While verison 1.15.1 is fine I think you have a typo in all other files
which are named g_m_amar*.zip.
 
>   1. I have attached Apache License 2 in LICENSE file.

Fine.

>   2. Indeed I've searched "print " and found some results which  resolved.
> I've tested it on Python3.

I'll check later and will report if I might find some remainings.

>   3. indeed, this was ugly, I've moved all external tools configuration to
> a separate config_tools.txt, and change accordingly to your proposal

Thanks.

>   4. I've improved the information on what happens when no parameters are
> provided to the program. The error you were getting was caused because the
> default action "-a" parameter to set to a method which assumed some
> preprocessing. Now, I require the parameter to be explicitly set in console.

Thanks for your support and I'll keep you updated about issues I might
find in 1.15.1.  I will not upload anything befor I'll get your final
confirmation that you cleaned up everything.

Kind regards

  Andreas.

> 2015-11-21 18:53 GMT+01:00 Andreas Tille :
> 
> > Hello,
> >
> > I'm writing you on behalf of the Debian Med team that has the objective
> > to package all free software that is relevant in Biology and Medicine
> > for official Debian.
> >
> > I came across GWAMAR[1] and started the packaging in Debian Git[2].
> > When doing so I stumbled upons some issues in the download archive[3]:
> >
> >   0. Just a comment:  It would be more convenient to find versioned
> >  download archives rather than versioned directories.  The
> >  rationale behind this is that you have files with the same name
> >  but different content.
> >
> >   1. I did not found any explicite license statement neither at the
> >  website nor inside the code.  Could you be so kind to clarify
> >  this?
> >
> >   2. At Bitbucket[4] you write:
> >This software is written in Python, thus Python 2 or 3 is
> >required to run GWAMAR.
> >  When trying to build with Python3 I've git some errors
> >  (basically in print statements with Python2 syntax.  Please
> >  let me know if you are interested in patches fixing this.
> >
> >   3. I noticed that the default config file in the download archive
> >  contains several private PATH settings.  I patched these to
> >  the Debian locations in the packaging git[5].  It would be
> >  great if you could change the default config file to a more
> >  neutral setting.
> >
> >   4. Finally I endet up by an error message
> >
> >File "a1_save_details_scores_all.py", line 161, in 
> >  input_fh = open(input_fn)
> >IOError: [Errno 2] No such file or directory:
> > 'datasets/mtu173/exP//res_profiles.txt'
> >
> >  I have no idea how to deal with this since this file is
> >  neither in the download tarball nor can I see it in the
> >  repository at Bitbucket.
> >
> > I hope you like the intend to package GWAMAR for Debian and can
> > help with these issues.
> >
> > Kind regards
> >
> >   Andreas.
> >
> > [1] http://bioputer.mimuw.edu.pl/gwamar/
> > [2] git://anonscm.debian.org/debian-med/gwamar.git
> > [3] http://bioputer.mimuw.edu.pl/gwamar/software/gwamar_v1.14/gwamar.zip
> > [4] https://bitbucket.org/mimowo/gwamar
> > [5]
> > https://anonscm.debian.org/cgit/debian-med/gwamar.git/tree/debian/patches/adapt_debian_locations_of_binaries.patch
> >
> > --
> > http://fam-tille.de
> >

-- 
http://fam-tille.de

Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Uecker, Martin]

Hi Andreas,

> On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote:
> > > Put simply, pristine-tar is our way to encapsulate access to the source 
> > > tarball used for packaging. Someone who checks out a d-science 
> > > repository does not need to know where the tarball comes from (github, 
> > > bitbucket, PyPI...), he or she can just check it out using pristine-tar 
> > > on the packaging repository.
> > 
> > Ok, I created a tar ball using a git archive (which matches what
> > github does) and then used pristine-tar to check it in.
> 
> I think this is a misunderstanding.  You should write a debian/watch file
> (line 22 of this template
>   
> https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511=markup
> is your friend) and use the downloaded tarball when importing pristine-tar.

Ok, done.

Please note that there is no difference between downloading 
tar balls from github which uses 'git archive' to create them
or creating them locally using 'git archive' (with the
right arguments). This already produces bit-identical results
(with the same hash)! So there is really no point in downloading
upstream tarballs from Github when one has a local copy of the git
repository. 
 

> > gbp can also create tar balls from the same tag and check
> > in ione step, but somehow the hash does not seem to match
> > exactly (the content does). I wonder why...
> 
> You stumbled upon the very problem pristine-tar is solving:  Tarballs
> simply have different checksums even if featuring identical content.
> This is for instance due to different time stamps, user ids etc after
> unpackaging on a target system.  Feel free to seek Debian lists for
> several discussions explaining the problem.

I already know ;-) I am (in)famous for starting a flamewar on
debian-devel about reproducible builds in 2007...


> You could add these in additional python-bart octave-bart binary
> packages (sorry, matlab can not be provided as official Debian package).
> You should read the according pages at wiki.debian.org where to put
> Python modules (or you just check your local system where these are
> stored) and Octave files (I never dealt with these but I guess there is
> a wiki paga as well).  Feel free to ask me if you are struck in the
> jungle of documentation and I'll provide more specific pointers.

Ok, I have to look at it. There are only very few small scripts, 
so I would rather put it in the same package.

> Another remark to the packaging:  Currently there is a libgsl migration
> ongoing and you should use libgsl-dev instead of libgsl0-dev.

Done. Although now it doesn't build locally on my Ubuntu machine
anymore (only using pbuilder in a sid change root).

Martin

Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Ghislain Vaillant]

On 07/12/15 08:18, Uecker, Martin wrote:


Hi Andreas,


On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote:

Put simply, pristine-tar is our way to encapsulate access to the source
tarball used for packaging. Someone who checks out a d-science
repository does not need to know where the tarball comes from (github,
bitbucket, PyPI...), he or she can just check it out using pristine-tar
on the packaging repository.


Ok, I created a tar ball using a git archive (which matches what
github does) and then used pristine-tar to check it in.


I think this is a misunderstanding.  You should write a debian/watch file
(line 22 of this template
   
https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511=markup
is your friend) and use the downloaded tarball when importing pristine-tar.


Ok, done.

Please note that there is no difference between downloading
tar balls from github which uses 'git archive' to create them
or creating them locally using 'git archive' (with the
right arguments). This already produces bit-identical results
(with the same hash)! So there is really no point in downloading
upstream tarballs from Github when one has a local copy of the git
repository.


Actually, you can do it with `gbp buildpackage` by passing the 
--git-no-pristine-tar and --git-pristine-tar-commit, which effectively 
will produce the pristine tarball from the tag you based your packaging off.


Andreas gave you the general guideline which works for any source. The 
--git*pristine-tar* options only work if you are using the upstream git 
repository.


FYI, make sure you have a valid watch file (although you are not using 
uscan), because that is also what is used by the Debian Package Tracking 
System to signal when new upstream releases are available.



You could add these in additional python-bart octave-bart binary
packages (sorry, matlab can not be provided as official Debian package).
You should read the according pages at wiki.debian.org where to put
Python modules (or you just check your local system where these are
stored) and Octave files (I never dealt with these but I guess there is
a wiki paga as well).  Feel free to ask me if you are struck in the
jungle of documentation and I'll provide more specific pointers.


Ok, I have to look at it. There are only very few small scripts,
so I would rather put it in the same package.


Another remark to the packaging:  Currently there is a libgsl migration
ongoing and you should use libgsl-dev instead of libgsl0-dev.


Done. Although now it doesn't build locally on my Ubuntu machine
anymore (only using pbuilder in a sid change root).


You could use something like libgsl0-dev | libgsl-dev to stay compatible 
with earlier versions?


Ghis

Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Andreas Tille]

Hi Martin,

On Mon, Dec 07, 2015 at 08:18:59AM +, Uecker, Martin wrote:
> 
> Hi Andreas,
> 
> > On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote:
> > > > Put simply, pristine-tar is our way to encapsulate access to the source 
> > > > tarball used for packaging. Someone who checks out a d-science 
> > > > repository does not need to know where the tarball comes from (github, 
> > > > bitbucket, PyPI...), he or she can just check it out using pristine-tar 
> > > > on the packaging repository.
> > > 
> > > Ok, I created a tar ball using a git archive (which matches what
> > > github does) and then used pristine-tar to check it in.
> > 
> > I think this is a misunderstanding.  You should write a debian/watch file
> > (line 22 of this template
> >   
> > https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511=markup
> > is your friend) and use the downloaded tarball when importing pristine-tar.
> 
> Ok, done.
> 
> Please note that there is no difference between downloading 
> tar balls from github which uses 'git archive' to create them
> or creating them locally using 'git archive' (with the
> right arguments). This already produces bit-identical results
> (with the same hash)! So there is really no point in downloading
> upstream tarballs from Github when one has a local copy of the git
> repository. 

I have no doubt that *Github* can create bit-identical results.
However, I have doubt that you can create from a local clone the
very same tarball.  This is the role of the pristine-tar branch
and currently I can not build bart by the following procedure:


   ssh://git.debian.org/git/debian-med/bart.git
   cd bart
   gbp buildpackage


I simply get diffs between the upstream branch and the orig.tar.gz.
The above should be possible and it becomes possible if you do


   cd bart
   uscan --verbose --force-download
   gbp import-orig --pristine-tar ../bart*.orig.tar.gz


> > You could add these in additional python-bart octave-bart binary
> > packages (sorry, matlab can not be provided as official Debian package).
> > You should read the according pages at wiki.debian.org where to put
> > Python modules (or you just check your local system where these are
> > stored) and Octave files (I never dealt with these but I guess there is
> > a wiki paga as well).  Feel free to ask me if you are struck in the
> > jungle of documentation and I'll provide more specific pointers.
> 
> Ok, I have to look at it. There are only very few small scripts, 
> so I would rather put it in the same package.

OK, you are the expert here. :-)
 
> > Another remark to the packaging:  Currently there is a libgsl migration
> > ongoing and you should use libgsl-dev instead of libgsl0-dev.
> 
> Done. Although now it doesn't build locally on my Ubuntu machine
> anymore (only using pbuilder in a sid change root).

That's an unfortunate consequence of the transition.

Kind regards

   Andreas. 

-- 
http://fam-tille.de

Re: Re: [MoM] Re: bart - tools for computational magnetic resonance imaging

[Martin Uecker]

Hi Andreas,

> On Mon, Dec 07, 2015 at 08:18:59AM +, Uecker, Martin wrote:
> > > On Sun, Dec 06, 2015 at 11:59:02PM +, Uecker, Martin wrote:
> > > > > Put simply, pristine-tar is our way to encapsulate access to the 
> > > > > source 
> > > > > tarball used for packaging. Someone who checks out a d-science 
> > > > > repository does not need to know where the tarball comes from 
> > > > > (github, 
> > > > > bitbucket, PyPI...), he or she can just check it out using 
> > > > > pristine-tar 
> > > > > on the packaging repository.
> > > > 
> > > > Ok, I created a tar ball using a git archive (which matches what
> > > > github does) and then used pristine-tar to check it in.
> > > 
> > > I think this is a misunderstanding.  You should write a debian/watch file
> > > (line 22 of this template
> > >   
> > > https://anonscm.debian.org/viewvc/debian-med/trunk/package_template/watch?revision=20511=markup
> > > is your friend) and use the downloaded tarball when importing 
> > > pristine-tar.
> > 
> > Ok, done.
> > 
> > Please note that there is no difference between downloading 
> > tar balls from github which uses 'git archive' to create them
> > or creating them locally using 'git archive' (with the
> > right arguments). This already produces bit-identical results
> > (with the same hash)! So there is really no point in downloading
> > upstream tarballs from Github when one has a local copy of the git
> > repository. 
> 
> I have no doubt that *Github* can create bit-identical results.
> However, I have doubt that you can create from a local clone the
> very same tarball.  

I agree that it is a bit surprising, but it does work:

wget https://github.com/mrirecon/bart/archive/v0.2.09d.tar.gz
git archive --prefix=bart-0.2.09d/ -o bart_0.2.09d.tar.gz v0.2.09d
sha256sum v0.2.09d.tar.gz bart_0.2.09d.tar.gz 
35879d41dcdefedc9fbca40267490d055e3a4840d7731a0f221c6976035dfff0  
v0.2.09d.tar.gz
35879d41dcdefedc9fbca40267490d055e3a4840d7731a0f221c6976035dfff0  
bart_0.2.09d.tar.gz

> This is the role of the pristine-tar branch
> and currently I can not build bart by the following procedure:
> 
> 
>ssh://git.debian.org/git/debian-med/bart.git
>cd bart
>gbp buildpackage

The following works for me:

git clone ssh://uecker-gu...@git.debian.org/git/debian-med/bart.git
cd bart
gbp buildpackage --git-upstream-tag=v0.2.09d --git-pbuilder

Also the following should also work correctly:

git clone ssh://uecker-gu...@git.debian.org/git/debian-med/bart.git
cd bart
pristine-tar checkout ../bart_0.2.09.orig.tar.gz
dpkg-buildpackage 


The only issue is that I have to specify the tag with 'gdb'
because it looks for 'upstream/v0.2.09' which does not
currently exist. Instead of adding these tags, I would prefer
to reconfigure 'gdb' to use the existing upstream tags without
the prefix 'upstream/'. Then I could just push the regular
upstream tags. Would this be ok?


I also created a newer upstream tag v0.2.09d specifically
for the initial packaging work because I made some upstream
changes to make packaging easier.  The bart_0.2.09.orig.tar.gz 
which is now in the pristine-tar branch actually corresonds
to v0.2.09d.  I hope that is not too messed up, but with the
next upstream version (to be released soon) this hack would 
gone.


Martin