Am 16.07.2012 11:00:07 schrieb(en) Sylvain Le Gall:
Hi,
2012/7/14 Gerd Stolpmann <i...@gerd-stolpmann.de>:
> Am 13.07.2012 09:08:24 schrieb(en) Sylvain Le Gall:
>
>> > Regarding the Windows port, how far would you go? I always found
it
>> > reasonable to provide a binary release, as many Windows users
get scary when
>> > it comes to software builds. But I never found the time to do
this, and
>> > hoped a group of interested people could maintain such a
release. Skylift
>> > has now the required features: Users can download binary
packages from
>> > remote repositories as if they were local. The format of the
binary packages
>> > was extended for ensuring binary compatibility (the checksum
thing). For
>> > publishers, there is scripting support for copying the packages
to a public
>> > directory. I could imagine we could attract a lot more beginners
and serious
>> > users with such a release.
>> >
>>
>> Lets start small. Make the GODI build process (and a few selected
>> packages) works as reliably as possible.
>>
>> Once we will have time and be confident in process, we can use the
>> builder to publish binary packages required to do the bootstrap.
If we
>> can get to the point that we can create a binary installer that
will
>> help to skip all the initial bootstrap process, it will be a big
Win
>> and a very argument for beginner.
>>
>> I see some problems however, to reach this state:
>> - GODI binary packages need to be relocatable, i.e. forcing the
>> standard directory to be c:\cygwin\opt\godi would feel strange for
a
>> Windows guy
>
>
> Agreed. However, a full solution seems to be some work. Not inside
GODI (I
> guess it is simple to make it relocatable by changing config
files), but in
> the packages.
>
> There are a few problems
>
> - hardcoded paths
> - ocamlrun executables need to be relocated (on win32
> there is a executable header, don't know this works
> in detail)
>
I think most library I know avoid hardcoded path -- as much as
possible. Only camomile has a strange reason not to do so.
Not only Camomile... It is more widespread than you think. I also use
hardcoded paths in Ocamlnet, and I know they also exist in e.g. omake
or in hevea, and even findlib has a problem here (it needs to know the
site-lib path).
I talked to
upstream and he replies that he was doing it for security reason: he
use Marshal to load data and want to be sure that he loads what he has
installed. I don't really buy his argument and the sole effect of
this, is that I am more and more considering dropping camomile in
favor of anything else more flexible with regard to relocation.
He is right. The problem are setuid programs using Camomile. If you can
change the path, you can inject code, at least code pointers. (Well,
nobody has ever shown that this attack could be successful, but we
cannot exclude it.)
A possible workaround is to compute a checksum before loading the files
- only the checksum would then be hardcoded. So, I think this is
solvable without opening security holes.
In Ocamlnet (btw, Camomile originally forked from the Netconversion
module in Ocamlnet), I at least allow to set the path from the program
- in the context of a concrete application it is normally harmless to
override the path, so let the developer decide.
But anyway, the "path problem" exists, and I don't think it is a good
idea to introduce per-library environment variables to fix it. Users
won't like it.
>> - some packages are definitively not relocateable (camomile e.g.),
we
>> will need to skip them
>> - once relocated we should be able to continue using GODI
>
>
> For adapting hardcoded paths we have a number of options:
> 1. Use relative paths, e.g. relative to the executable
> 2. Use environment variables
> 3. Use registry
> 4. Use a property system like in Java
>
> For me, 1. and 4. look promising (but 4. only if there is support
in core
> OCaml). Option 2 has the disadvantage of not being scalable (you
don't want
> users to set 20 variables, and Windows limits environments to 32K).
3. is
> Windows-only, and it is hard to sell to Unix users.
>
> The property system could look like:
> - There are simply hierarchic variables a.b.c.d which have string
values
> - These variables can be taken from different sources and override
each
> other
> - There would be a default <stdlib>/properties with the
deployment-wide
> path settings
>
> There would only be a single function Sys.property to get the
value. Such a
> system would even be attractive to Unix users because you can
generally
> avoid to bake paths into code.
>
> Of course, this is a long-term solution only - we would need
properties in
> core OCaml first (which can take months). Providing properties in a
library
> would be too unattractive for software developers.
Months -> years. UNIX users has the solution of environment variable
and they use it. Trying to convince anyone on the ground of "it can be
useful for Windows user", we probably be hard in the OCaml community,
which is somehow UNIX centric.
I'm not so skeptical. The key for being successful is that the solution
must also be attractive for Unix users who run into the path problem.
Actually, hardcoding paths in normal in Unix, and providing here a
programming pattern would be welcome. Windows users want everything in
the registry, and we should ignore this wish to make the Unix users
happy (it's politics).
I think it would be really be best to have this feature in core OCaml,
at least it is worth a try. Another option would be a findlib extension
- simply because findlib has also to do with deployment stuff, and it
is always available.
My best bet on this: create an external library that wrap nicely
Unix.getenv into something useable and providing additional benefits
to UNIX user -- AND sneak in what we need for Windows.
For the integration into the core stdlib, to be honnest, I don't
believe we can. Let start with something external.
Hard to say - I was recently successful with suggesting extensions.
>> FYI, if we are able to do it for Windows, we can also create such
>> installer for Mac and Linux with little additional effort.
>
>
> There is another problem here that does not occur for Windows: the
object
> format may include paths to libraries. Maybe this is solvable with
objcopy.
>
So called "rpath", we can easily get rid of it and it is not a big
deal. We do it everytime on Debian and it doesn't break anything.
Well, I guess you are too much used to Debian. The situation is
different on other OS. On Debian you don't need it because all libs are
installed in standard places. E.g. look at the situation on Macs.
But this is certainly low priority.
>> Does skylift bootstrap process produces the binary packages ? If
this
>> is the case, where are they located ?
>
>
> build/packages/All
>
>
>> If the binary packages are
>> there, providing them is a matter of second.
>
>
> It's easy: set EXPORT_PACKAGES to the directory where the public
packages
> should go to, and call "godi_make export-write-package" after each
build.
> This copies the package, and updates a file PKGMANIFEST in the
export
> directory.
>
> Alternatively, you can also copy the package from build/packages/All
> yourself, and create this file manually:
>
> godi_console ptool -in-binary "$package_file" -out-manifest
>
> This command creates the part of the manifest for a single package
> (concatenate for all packages).
>
> For importing such a directory into another GODI deployment set
GODI_BINPKG
> to the path or URL of the export directory.
>
> Btw., this text summarized the new features of Skylift:
> http://godi.camlcity.org/godi/project-doc/skylift-intro.txt
>
> Gerd
So my idea is the following:
- build godi-skylift
- extract binary packages built for the bootstrap
- inject it into an installer godi-skylift.exe
So godi-skylift.exe can install the base GODI system without compiling
a single line of source code ! Do you think it is feasible ?
Yes. Basically it works already... You can extract a minimal file tree
from a Skylift deployment with "godi_make export-seed", and use this to
add all further packages. Wrap this in an installer, and you are done.
Well, it is still relatively untested, but I'm very optimistic.
If we can achieve that, it will be a big win for the proof of concept
(to be honnest, I don't think it will mean +2k users in a week -- but
at least we will bring the OCaml community on step forward). To start
this project, I think we can go with "loose constraints": force user
to install cygwin by themselves, etc...
Agreed, proceed step by step.
Gerd
Gerd: can you provide me a new skylift .tar.gz with updated console to
investigate janestreet-core problem tonight ?
Cheers
Sylvain
--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany g...@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details: http://www.camlcity.org/contact.html
Company homepage: http://www.gerd-stolpmann.de
------------------------------------------------------------
_______________________________________________
Godi-list mailing list
Godi-list@ocaml-programming.de
https://godirepo.camlcity.org/mailman/listinfo/godi-list
