Dear list members, I'd like to share some sketchy ideas I have on the subject to address some of the issues raised.
At 12:14 22/03/05 +0000, Malcolm Wallace wrote: >I cannot see any of the Haskell compilers ever implementing this idea >as presented. It would introduce an enormous raft of requirements >(networking client, database mapping, caching, etc) that do not belong >in a compiler - they belong in separate (preprocessing/packaging) >tools. Furthermore, these tools already exist, albeit they are young >and have a long maturation process still ahead of them. An external program acting as an "URI streamer" might be the solution. Such a program (identified via an environment variable or a compiler's command line option, just like Hugs external editor) would take an URI as its command line argument, and respond with a streamed contents of that URI on its stdout. Like e. g. curl/wget -O -. All the compiler has to do is popen() that program if an import statement contains an URI. Using curl/wget helps get around various issues with proxies/encryption/etc. as those programs are specifically designed for that. I do not believe this would result in significant overhead comparing to regular fopen() used by the compiler for opening source files. On a non-networked system, such a program would be a not so complicated shell script pretending it downloads from an URI, but reading from local disk (flash, any other kind of storage) instead. To address the problem of module/package URI changes over time, the following may be suggested. Either purl.org is used (and then it is responsibility of a package maintainer to keep its URL pointer valid). Or, some kind of a purl server may be set up somewhere (at haskell.org for example) which also supports mirroring. This means that for each package/module registered with this server, multiple locations are known (well, probably willing people might allocate their computer resources for that, at least I would not object as long as I have my own web server). The hypothetical purl server serves redirects as usual, but shifting randomly to another mirroring location for each new request for a module/package. So, if an attempt to retrieve a module/package fails, it may be repeated, and other mirror location will be tried. Mirrors will be synchronized behind the scenes. Will such a centralized purl server a bottleneck or a single point of failure? Probably not more than a centralized Hackage database (or is it planned to be distributed?) Also, some resolver might be part of the URI streamer which maps module names to URIs. For example, the Prelude will most likely be stored locally, but some other module will not. This means that the resolver consults the local package database (cabal), and its own cache, and either streams a local file or popens curl with some URI constructed specifically based on the desired module name. Once downloaded, a module may be cached with some TTL, so further recompilations do not result in curl involvement (until the TTL expires). PS This all is written in the assumption that Haskell source files are served. Binary distributions, of course would require different techniques. -- Dimitry Golubovsky Anywhere on the Web _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe