On Wed, May 16, 2012 at 12:19 AM, William Stein <wst...@gmail.com> wrote:
> On Wed, May 16, 2012 at 2:35 AM, Robert Bradshaw
> <rober...@math.washington.edu> wrote:
>> On Tue, May 15, 2012 at 5:40 PM, Keshav Kini <keshav.k...@gmail.com> wrote:
>>> John H Palmieri <jhpalmier...@gmail.com> writes:
>>>> On Monday, May 14, 2012 11:43:09 PM UTC-7, Keshav Kini wrote:
>>>>
>>>> William Stein writes:
>>>> > On Mon, May 14, 2012 at 12:31 AM, William Stein wrote:
>>>> >> On Sun, May 13, 2012 at 5:13 PM, Jeroen Demeyer wrote:
>>>> >>> I disagee when it comes to removing parts of a spkg. Several
>>>> packages
>>>> >>> include only partial sources. They contain the upstream tree but
>>>> with
>>>> >>> some files/directories (which Sage doesn't need) removed. I think
>>>> this
>>>> >>> is fine and should be allowed.
>>>> >>
>>>> >> Indeed. Many spkg's are full of stuff we absolutely don't want to
>>>> >
>>>> > s/spkg/upstream sources (not spkg's!)
>>>> >
>>>> >> ship. They have windows binaries in them, java binaries, big pdf's,
>>>> >> and other random stuff that wastes space and makes some people
>>>> >> nervous.
>>>>
>>>> OK, that's reasonable. I withdraw my objection, though it would still
>>>> be
>>>> nice if we had a better history of what is or was in the src/
>>>> directories of SPKGs, if it is no longer considered possible to
>>>> determine this simply from the package version minus the patch number.
>>>>
>>>>
>>>> Any changes like this are supposed to be documented in SPKG.txt, right? If
>>>> not,
>>>> the spkg needs work.
>>>
>>> Sure, but a blurb in SPKG.txt is not really the same as having true
>>> versioning of the directory. Of course, we do mostly have copies of all
>>> old versions of all SPKGs (going back at least as far as /home/release
>>> on sage.math contains versions of Sage). So we could reconstruct the
>>> history of these src/ directories with some effort.
>>>
>>> Note: I'm not saying we should put src/ under version control. I just
>>> think it's best if we modify it only at install time via patches, rather
>>> than at packaging time, but as Jeroen and William point out, sometimes
>>> it's an issue of file size, or just not wanting to include binary
>>> blobs...
>>
>> I don't think we should be storing the unpacked src/ directory in the
>> spkg at all, instead we should store a pointer to a tarball (which
>> could be external or right in the spkg itself. Upon install this
>> tarball would get unpacked into src/. The sage-spkg script would
>> unpack this tarball, compare it to src/ (if it exists) making sure
>> there are no differences (that are not accounted for by the existing
>> patches, or perhaps even creating a patch to describe the difference),
>> and then create the .spkg file excluding the src/ directory (but
>> possibly including the pristine, unpacked tarball).
>
> I read the above 4 times and I don't understand what you're proposing,
> except maybe (?) to make it impossible to build Sage if you're not
> connected to the internet...? Or to basically change nothing?
> Confused.
Sorry, I was really tired when I wrote that (and a lot of state was
just in my head). Keshav summarized things nicely. The only part I
would add is that if we set it up this way, we could use the (unpacked
and ignored by sage-spkg) src/ directory, if present, to automatically
verify or generate the patch files.
I don't like spkg.txt because it's just a (under-specified) convention
of what to put there when you make manual changes. Better would be to
have some specification file (json?) like
{ upstream = "http://original.project/releaseX.tar";,
upstream_sha1 = "...",
excise = [ "windows.exe", "binary.blob" ], # remove from the above
when creating our tarball (below)
sources = "project.X.tar",
sources_sha1 = "...",
patches = [ "fixOne.patch", "fix2.patch" ], # to be applied to
sources, once unpacked
}
Then an spkg could be automatically generated by changing the upstream
url (most common case), and otherwise one could tweak/meddle with src
and automatically create a new patch once you get it working.
This may be a pain to do with shell scripts (how do you parse a json
file?) but perhaps a lot could be cleaned up if we assumed *some*
python as a prerequisite.
- Robert
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
