Re: Build-Depends on source itself [libgap-sage]

[Bill Allombert]

On Wed, Nov 02, 2016 at 12:11:37PM +0800, Paul Wise wrote:
> On Wed, Nov 2, 2016 at 11:22 AM, Jerome BENOIT wrote:
> 
> > Let give a try. I am dealing with the libgap-sage package [1].
> 
> Thanks for the extensive details.
> 
> > To begin with, GAP is a Computer Algebra System (CAS).
> > From an upstream point of view, libgap is not part of GAP itself.
> > libgap is rather a library wrapper for gap meant to get a better
> > access to the GAP kernel and to be used within Sage, which is a kind
> > a umbrella for multiple CAS (GAP, Singular, and a myriad of
> > scientific oriented software packages).
> > Note that, for now at least, GAP itself does not furnish any library; hence 
> > libgap.
> 
> The libgap project seems like a workaround for this bug in GAP?

This is not a bug in GAP. GAP has its own language and does not
provide a C interface. It also uses its own memory manager which is
not compatible with malloc.

Designing a library implies defining an API and an ABI, something that
has not been formalized for libgap.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Re: Build-Depends on source itself [libgap-sage]

[Bill Allombert]

On Wed, Nov 02, 2016 at 03:22:41AM +, Jerome BENOIT wrote:
> Hello,
> 
> > On Tue, Nov 1, 2016 at 12:43 PM, Jerome BENOIT wrote:
> >
> >> for one of my package, libgap-sage [1], the source material used for build
> >> is in fact seded meterial from an other package, gap: grossely the sed 
> >> process
> >> is the main part of libgap package: is there a standard way to use the 
> >> Debian
> >> source package of gap inside my own package libgap-sage ?
> >> Any hint or example is welcome.

Hello Jérome,
I was requested by Felix Salfelder to build gap-dev object files with
-fPIC for this purpose.
Instead of using sed, Sage should use a linker script, this would be
much cleaner.

gap (4r6p5-3) unstable; urgency=low

  * debian/rules:
   - build gap-dev object files with -fPIC to be compatible with use in
 shared libraries.

 -- Bill Allombert   Thu, 19 Sep 2013 15:00:06 +0200

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Re: Build-Depends on source itself [libgap-sage]

[Jerome BENOIT]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Thanks for your prompt reply.

On 02/11/16 04:11, Paul Wise wrote:
> On Wed, Nov 2, 2016 at 11:22 AM, Jerome BENOIT wrote:
> 
>> Let give a try. I am dealing with the libgap-sage package [1].
> 
> Thanks for the extensive details.


You are welcome. 
> 
>> To begin with, GAP is a Computer Algebra System (CAS).
>> From an upstream point of view, libgap is not part of GAP itself.
>> libgap is rather a library wrapper for gap meant to get a better
>> access to the GAP kernel and to be used within Sage, which is a kind
>> a umbrella for multiple CAS (GAP, Singular, and a myriad of scientific 
>> oriented software packages).
>> Note that, for now at least, GAP itself does not furnish any library; hence 
>> libgap.
> 
> The libgap project seems like a workaround for this bug in GAP?

Indeed.

> 
>> Basically libgap adds a prefix to the GAP functions and variables,
>> and gather them in a library. We must have in mind the following points:
>> 1] The libgap source ball provides the modified source files with some delay
>> (current modified gap version in libgap is 4.8.3 while the current gap 
>> version is 4.8.5 ,
>> this version being the one in Stretch. In any case, the copyright headers of 
>> the source files
>> are not modified, so they cannot be packaged for copyright reasons.
> 
> That seems like a very suboptimal way to do it.

I guess that everybody is agree here.

> 
>> 2] The scripts that modify the original GAP source files is not distributed 
>> within
>> the libgap upstream source ball, but it is available via the libgap git 
>> repository [2] at Bitbucket
>> along some documentation for generating our own modified GAP source code. 
>> The current Debian source ball
>> for libgap is the git repository material (which unmodified contains but 
>> obsolete GAP material, version 4.8.3).
> 
> Ok, I'm glad this is not a DFSG violation. It could easily have been one.

I took care about this part of the story.

> 
>> 3] The libgap Debian package must be synchronized with the GAP Debian 
>> package, so modifying the (potentially)
>> patched GAP src/ is certainly not only a good idea but also a factor of 
>> stability and good maintenance.
> 
> Agreed.
> 
>> 4] Just the material in the subfolder src/ within the GAP source ball is 
>> needed, that is to say, not the all
>> source ball.
> 
> Makes sense.
> 
>> 5] We want a long term solution to ease the maintenance of Sage[Math] and 
>> its friends.
> 
> It seems clear to me that the only sane long-term solution is for GAP
> upstream to add a proper shared library.

Absolutely agree. My understanding is that the GAP upstream team is working on 
it.
But GAP is an heavy machinery, and  it might be not one of the top priorities.


 Has this been discussed with
> them at all?

I bet we can find long discussion about on the internet.

 Until GAP upstream are willing to do this,

Let focus on this part.


 I suggest one
> of the following:
> 
> Drop the libgap-sage source package entirely and add a secondary
> tarball to the Debian GAP source package containing only the libgap
> scripts and have the Debian build process for GAP use those scripts to
> create libgap-sage dev and lib binary packages. dpkg-source format v3
> can have multiple tarballs, which makes this doable sanely.
>

Keeping the `intersection' with the GAP debian package minimal is also a factor.

 
> OR
> 
> Get the GAP sources removed from the Bitbucket repository.
> 
> Have the build scripts in the libgap Bitbucket repository:
> 
> 1. require info on which source tree to copy or which version to download
> 2. copy that source tree or download that version
> 3. modify the local copy using the scripts as per normal
> 4. build the scripts as per normal
> 
> There might need to be some checking of the copied source tree to make
> sure the scripts still support it.

The current scheme of the libgap-sage package tends to this solution:
the first priority was to bring libga[-sage] to Debian.


> 
> Create a gap-libgap-sage-source (or similar) package from the GAP
> Debian package, containing the GAP src/ directory somewhere under
> /usr/src.
> 
> Have the libgap-sage package build-depend on gap-libgap-sage-source
> and point the libgap scripts at the GAP src/ directory under /usr/src.
> 
> Make sure that the libgap-sage package is binNMUed or rebuilt after
> every gap upload. I expect the script will need to change reasonably
> often due to changes in GAP though.

This is what I had in mind after reading your previous emails and the one of 
Den.

@Bill :
Which solution would you like to favour ?

Personally I prefer the second solution as it treats the GAP material as raw 
material
and that it puts a clear line between GAP and Sage as concerns Debian 
maintenance.

Can I start to write patch against the GAP debian/ material ?

Thanks,
Jerome

> 

- -- 
Jerome BENOIT | calculus+at-rezozer^dot*net
https://qa.debian.org/developer.php?login=calcu...@rezozer.net

Re: Build-Depends on source itself [libgap-sage]

[Paul Wise]

On Wed, Nov 2, 2016 at 11:22 AM, Jerome BENOIT wrote:

> 2] The scripts that modify the original GAP source files is not distributed 
> within
> the libgap upstream source ball, but it is available via the libgap git 
> repository [2] at Bitbucket
> along some documentation for generating our own modified GAP source code. The 
> current Debian source ball
> for libgap is the git repository material (which unmodified contains but 
> obsolete GAP material, version 4.8.3).

One thing I just noticed: Please get libgap upstream to use the Python
subprocess module with shell=False instead of os.system, which is
deprecated and vulnerable to shell meta-character injection.

https://sources.debian.net/src/libgap-sage/4.8.3%2Bg69a66f0%2Bdsx-1/scripts/libGAPify.py/#L465

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Build-Depends on source itself [libgap-sage]

[Paul Wise]

On Wed, Nov 2, 2016 at 11:22 AM, Jerome BENOIT wrote:

> Let give a try. I am dealing with the libgap-sage package [1].

Thanks for the extensive details.

> To begin with, GAP is a Computer Algebra System (CAS).
> From an upstream point of view, libgap is not part of GAP itself.
> libgap is rather a library wrapper for gap meant to get a better
> access to the GAP kernel and to be used within Sage, which is a kind
> a umbrella for multiple CAS (GAP, Singular, and a myriad of scientific 
> oriented software packages).
> Note that, for now at least, GAP itself does not furnish any library; hence 
> libgap.

The libgap project seems like a workaround for this bug in GAP?

> Basically libgap adds a prefix to the GAP functions and variables,
> and gather them in a library. We must have in mind the following points:
> 1] The libgap source ball provides the modified source files with some delay
> (current modified gap version in libgap is 4.8.3 while the current gap 
> version is 4.8.5 ,
> this version being the one in Stretch. In any case, the copyright headers of 
> the source files
> are not modified, so they cannot be packaged for copyright reasons.

That seems like a very suboptimal way to do it.

> 2] The scripts that modify the original GAP source files is not distributed 
> within
> the libgap upstream source ball, but it is available via the libgap git 
> repository [2] at Bitbucket
> along some documentation for generating our own modified GAP source code. The 
> current Debian source ball
> for libgap is the git repository material (which unmodified contains but 
> obsolete GAP material, version 4.8.3).

Ok, I'm glad this is not a DFSG violation. It could easily have been one.

> 3] The libgap Debian package must be synchronized with the GAP Debian 
> package, so modifying the (potentially)
> patched GAP src/ is certainly not only a good idea but also a factor of 
> stability and good maintenance.

Agreed.

> 4] Just the material in the subfolder src/ within the GAP source ball is 
> needed, that is to say, not the all
> source ball.

Makes sense.

> 5] We want a long term solution to ease the maintenance of Sage[Math] and its 
> friends.

It seems clear to me that the only sane long-term solution is for GAP
upstream to add a proper shared library. Has this been discussed with
them at all? Until GAP upstream are willing to do this, I suggest one
of the following:

Drop the libgap-sage source package entirely and add a secondary
tarball to the Debian GAP source package containing only the libgap
scripts and have the Debian build process for GAP use those scripts to
create libgap-sage dev and lib binary packages. dpkg-source format v3
can have multiple tarballs, which makes this doable sanely.

OR

Get the GAP sources removed from the Bitbucket repository.

Have the build scripts in the libgap Bitbucket repository:

1. require info on which source tree to copy or which version to download
2. copy that source tree or download that version
3. modify the local copy using the scripts as per normal
4. build the scripts as per normal

There might need to be some checking of the copied source tree to make
sure the scripts still support it.

Create a gap-libgap-sage-source (or similar) package from the GAP
Debian package, containing the GAP src/ directory somewhere under
/usr/src.

Have the libgap-sage package build-depend on gap-libgap-sage-source
and point the libgap scripts at the GAP src/ directory under /usr/src.

Make sure that the libgap-sage package is binNMUed or rebuilt after
every gap upload. I expect the script will need to change reasonably
often due to changes in GAP though.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Build-Depends on source itself [libgap-sage]

[Jerome BENOIT]

Hello,


On 01/11/16 05:19, Paul Wise wrote:
> On Tue, Nov 1, 2016 at 12:43 PM, Jerome BENOIT wrote:
>
>> for one of my package, libgap-sage [1], the source material used for build
>> is in fact seded meterial from an other package, gap: grossely the sed 
>> process
>> is the main part of libgap package: is there a standard way to use the Debian
>> source package of gap inside my own package libgap-sage ?
>> Any hint or example is welcome.
>
> At this time there is no way for source packages to build-depend on
> other source packages.
>
> The most often used workaround for this is binary packages with
> -source at the end of their names:
>
> $ aptitude search -- -source$


On 01/11/16 17:47, Paul Wise wrote:
> On Wed, Nov 2, 2016 at 12:38 AM, Jerome BENOIT wrote:
> 
>> Is there any guidance for such source packages ? Which ones of them I may 
>> consider
>> a good example.
> 
> They are all about the same, ship the source under /usr/src, done.
> 
>> I guess it is the best way to proceed.
> 
> It is probably the least good way to proceed, please provide some
> details so we can discuss a proper solution.
> 

Let give a try. I am dealing with the libgap-sage package [1].

To begin with, GAP is a Computer Algebra System (CAS).
>From an upstream point of view, libgap is not part of GAP itself.
libgap is rather a library wrapper for gap meant to get a better
access to the GAP kernel and to be used within Sage, which is a kind
a umbrella for multiple CAS (GAP, Singular, and a myriad of scientific oriented 
software packages).
Note that, for now at least, GAP itself does not furnish any library; hence 
libgap.

Basically libgap adds a prefix to the GAP functions and variables,
and gather them in a library. We must have in mind the following points:
1] The libgap source ball provides the modified source files with some delay
(current modified gap version in libgap is 4.8.3 while the current gap version 
is 4.8.5 ,
this version being the one in Stretch. In any case, the copyright headers of 
the source files
are not modified, so they cannot be packaged for copyright reasons.
2] The scripts that modify the original GAP source files is not distributed 
within
the libgap upstream source ball, but it is available via the libgap git 
repository [2] at Bitbucket
along some documentation for generating our own modified GAP source code. The 
current Debian source ball
for libgap is the git repository material (which unmodified contains but 
obsolete GAP material, version 4.8.3).
3] The libgap Debian package must be synchronized with the GAP Debian package, 
so modifying the (potentially)
patched GAP src/ is certainly not only a good idea but also a factor of 
stability and good maintenance.
4] Just the material in the subfolder src/ within the GAP source ball is 
needed, that is to say, not the all
source ball.
5] We want a long term solution to ease the maintenance of Sage[Math] and its 
friends.

For now, instead of a full source package and as suggested by Ben Finney, I 
think that a pure technical package
that would contain main the src/ subfolder material is the best option.
Of course, I am willing to write the patch against the GAP debian/ material 
that will introduce this src package.
And, before all, it must be done with the agreement of the GAP Debian package 
maintainer.

Thanks,
Jerome


[1] https://packages.qa.debian.org/libg/libgap-sage.html
[2] https://bitbucket.org/vbraun/libgap

Re: Build-Depends on source itself

[Ben Finney]

Jerome BENOIT  writes:

> for one of my package, libgap-sage [1], the source material used for build
> is in fact seded meterial from an other package, gap

One good solution would be to have the ‘gap’ package build a new binary
package, maybe ‘gap-data’, that contains the data you need in a format
likely to be useful for many purposes.

Could you work with the ‘gap’ package maintainer to achieve that?

Once ‘gap-data’ (if that is the name the ‘gap’ maintainers decide)
exists, your poposed pacakge can “Build-Depends: gap-data” and work
forward from there.

-- 
 \   “Don't worry about what anybody else is going to do. The best |
  `\ way to predict the future is to invent it.” —Alan Kay |
_o__)  |
Ben Finney

Re: Build-Depends on source itself

[Paul Wise]

On Wed, Nov 2, 2016 at 12:38 AM, Jerome BENOIT wrote:

> Is there any guidance for such source packages ? Which ones of them I may 
> consider
> a good example.

They are all about the same, ship the source under /usr/src, done.

> I guess it is the best way to proceed.

It is probably the least good way to proceed, please provide some
details so we can discuss a proper solution.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Build-Depends on source itself

[Jerome BENOIT]

Hello, thanks for the quick reply.

On 01/11/16 05:19, Paul Wise wrote:
> On Tue, Nov 1, 2016 at 12:43 PM, Jerome BENOIT wrote:
> 
>> for one of my package, libgap-sage [1], the source material used for build
>> is in fact seded meterial from an other package, gap: grossely the sed 
>> process
>> is the main part of libgap package: is there a standard way to use the Debian
>> source package of gap inside my own package libgap-sage ?
>> Any hint or example is welcome.
> 
> At this time there is no way for source packages to build-depend on
> other source packages.
> 
> The most often used workaround for this is binary packages with
> -source at the end of their names:
> 
> $ aptitude search -- -source$

Is there any guidance for such source packages ? Which ones of them I may 
consider
a good example.


> 
> If you can provide some details about your situation, we can probably
> find a better solution.
> 

I guess it is the best way to proceed.

Thanks,
Jerome

Build-Depends on source itself

[Jerome BENOIT]

Dear mentors,

for one of my package, libgap-sage [1], the source material used for build
is in fact seded meterial from an other package, gap: grossely the sed process
is the main part of libgap package: is there a standard way to use the Debian
source package of gap inside my own package libgap-sage ?
Any hint or example is welcome.

Thanks in advance,
Jerome


[1] https://packages.qa.debian.org/libg/libgap-sage.html
-- 
Jerome BENOIT | calculus+at-rezozer^dot*net
https://qa.debian.org/developer.php?login=calcu...@rezozer.net
AE28 AE15 710D FF1D 87E5  A762 3F92 19A6 7F36 C68B

Re: Build-Depends on source itself

[Paul Wise]

On Tue, Nov 1, 2016 at 12:43 PM, Jerome BENOIT wrote:

> for one of my package, libgap-sage [1], the source material used for build
> is in fact seded meterial from an other package, gap: grossely the sed process
> is the main part of libgap package: is there a standard way to use the Debian
> source package of gap inside my own package libgap-sage ?
> Any hint or example is welcome.

At this time there is no way for source packages to build-depend on
other source packages.

The most often used workaround for this is binary packages with
-source at the end of their names:

$ aptitude search -- -source$

If you can provide some details about your situation, we can probably
find a better solution.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Build-Depends on source itself

[Jerome BENOIT]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Dear mentors,

for one of my package, libgap-sage [1], the source material used for build
is in fact seded meterial from an other package, gap: grossely the sed process
is the main part of libgap package: is there a standard way to use the Debian
source package of gap inside my own package libgap-sage ?
Any hint or example is welcome.

Thanks in advance,
Jerome

 
[1] https://packages.qa.debian.org/libg/libgap-sage.html
- -- 
Jerome BENOIT | calculus+at-rezozer^dot*net
https://qa.debian.org/developer.php?login=calcu...@rezozer.net
AE28 AE15 710D FF1D 87E5  A762 3F92 19A6 7F36 C68B
-BEGIN PGP SIGNATURE-

iQQcBAEBCgAGBQJYGB1rAAoJED+SGaZ/NsaL+TUgALx4hUZVrTaZELF6szZXMunC
kFDTosjret1CmpDtNzaxeqpgvbc4csca/a9KUqd8ryigSmwEATN9dd6bZs+aE8Xn
sw8Xuk1yqf91AW76z87pVnX0FBuBKdaSKJpQGqgOa/uiNZ/iw7QBUAawhMeTNGPr
iOvRbqPp36c5pmEZI92Oc0nKpxrXfAKclbwXd/Neqpbb/LoRlxiBhau3KXJRQuwa
CCFLm783c8RqImeJnRCnFTB2VsqGO2fRLxR6+x0xyYQxKY4EZn6yD6adQQf8PUT3
t+z+P89JrFqwCAZrn8Sxg5BuQ1azF3lEoVxPr30GrWjoK6T3tNblYtSRrnmPJsNT
vR4QN7RKClfH2dqgDDmqKRlOsjtYchY7TxjdY6gl4vKxAoj1yEy4NTC3H5VLww54
aj1hBacYXVaEHvxKC0ojiLd7TLUbWv5/rmUp+mGsTaKEN5RD7pf4WrCr9dWuZM/6
QHG8Tz/xxAfYXeVmDccJfKzqg7N+/vKpUOnmRceMInatXEnhMeqz6r/XMktlFDNg
jINVvS2Gc/Z17bzK/4vZSGFR7/UhRzW3nE/uG2hN9fSlSLsv5qQ96wHnuIslBmtM
gyRaHRK7GPo7rptbQ6fuGgX6ZS13LZGNBRALgw8+sDp/M4ju5wOKWhb0sXUDjSk/
3JuzWmToUdUjMisbiqyo0G53Na0Eoy6ZLjO7Q6l4KGdfm+EiSN/To3LN/b9pLH0M
hi92CPMVuzm72cpdwgAArmRTJoIcuvG5r9e73PdTqoOd+IEgqNCd2THrGFixsOVp
ZXTJ8O49Vi9q8cpQM8MCUH1GR4TMBNCSns604Tv+DJgj/GQQ9TBGv3crR80b7bNo
4QTllXYAkZDDu/c2YDQV31bni36ivVQmfmHpiFkAkdjsNXNlMxWKPBijrIo/+Xmz
rjcPlmiDWzZDMlfPvM6r+TOOUnDCaV9VkYnVWdu72hWp4Sr0IlDnp34I5wHuWQWP
ECMSFoUh84MaD3DK8hRfhU7ORMDEPOCNPfo/lg/kwzYtfxxt99berq6NrvSy57j9
LoRyShy09n44gKkd5gKvp31Ir55da3vXuMu+PI6q+9aMY4mpGm8HPMu0fta8MvCb
fv6Eth7bni1ITcm4B8EmjqZdY7ahRxZJs51zLky8B9AssFW8BOP0eRai6a2v0OiO
dNSO+fZryRn+pqEWwho5XuVyqPeI3qq2AOrAdWqcMuWgY8iFuqRwzXimDp5+voI+
jrmdTvTSU+/1noanBHgHYbv7HUgzc3YEpLyA0RVcQv41NshIICFLPeLU7H3Jv5/v
ebTpp4oqyLVXKLjnACqcukiPWM9D4P7DEGSj7ZmXRpBJzg535CAFzAGylmNs2BQ=
=Xt8K
-END PGP SIGNATURE-