Re[2]: compiler-independent core libraries infrastructure
Hello Neil, Thursday, September 14, 2006, 6:14:30 PM, you wrote: then, a base library may be written against virtual Haskell compiler, which provides uniform set of low-level features while 'base' decorates these features with user-friendly interfaces Nice idea. There are a few practical issues - for example does this virtual Haskell copmiler support higher rank types? Multi-parameter type classes? Bang patterns? It quickly gets a lot more complicated. i mean by virtual Haskell compiler set of _library_ functions, not the language features, pragmas and other differences. problem of availability of language features for standard libraries typically solved in conservative way - i.e. we try to use as less language features as possible and separate modules that use non-standard features in separate packages. for example, modern array libraries should go into separate package because they use MPTC and therefore not available for many compilers. shebang patters, of course, should be no used - it's a feature for applications, not for writers of standard packages - ghc-base/hugsbase/.. libs to implement _subset_ of common low-level API Sounds like a very good idea. - base lib to equalize several compilers and compiler versions, Yes, some operations might be implemented in the base library, but have more efficient versions in the ghc-base library. For example, map for base should be defined the obvious way, for ghc it should be defined with foldr. How can you accomodate this? i'm pragmatic and don't think that base package should hide _all_ the compiler differences and especially all compiler-specific optimizations. going this way, we should put to ghc-base byte strings and many other things my proposal is to extract from 'base' package all high-level, written in pure Haskell algorithms and put it in ~10 'application' packages, say ByteString, Array, DataStructures, FFI... 'map' definitely should be defined here, even if its definition will have ghc-specific version 'base' package should provide common API to compiler libraries. for example, it should provide functions integerMult, arrayCreate, type IO and so on *hc-base packages should provide implementations of these functions, specific for concrete compiler, i.e. arrayCreate = arrayCreate# integerMult (S# a) (S# b) = intMult# a รจ ... newtype IO = IO (...) 'base' package should then analyze compiler name and version and import appropriate modules or define operations itself if there is no implementation: module ArrayOps where #if GHC import GHC.Arr #elseif Hugs import Hugs.Arr #else type Array a b = [(a,b)] arrayCreate = [] ... #endif all functions/types implemented in pure Haskell, all complex algorithms, all class definitions should go away from this package! it just provides common set of low-level operations and of no interest for end users. it's just a tool which provides virtual Haskell compiler API, which allows to write all other libraries in rather portable way last line: i have some experience of writing compiler-independent code with Haskell and C++ and believe that this plan is realistic The differences between Haskell compilers may well be bigger than those between C++ compilers! I wish you the best of luck, and think this would be really nice to hvae - unfortunately I think its unobtainable - but if we could just get some of this goodness that would be fantastic! you've missed one point - we _already_ have working solution, the 'base' library itself. all that we need is just to split it carefully to modules which may be independently upgraded, plus add compatibility with previous compiler version that 'base' currently lacks. so, it's more moving code around and careful planning task than a real technological challenge :) i think you just misunderstood me - i don't plan to make ultimate solution, just to solve some current meaningless problems - say, that we can't use old MArray interface with ghc 6.6 or new implementation of HashTable with ghc 6.2. cabal provided us with all the instruments required - all that is remain is to refactor base library to make it compiler-version-independent -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re[2]: compiler-independent core libraries infrastructure
Hello Ian, Friday, September 15, 2006, 8:20:36 PM, you wrote: what is a 'base' library now? it is the library that implements common set of operations for latest versions of ghc, hugs and nhc. it contains low-level implementation for ghc, but relies on separate hugsbase package for hugs (the same for nhc, afaiu). so, first step is obvious - separate ghc-base library from the rest. hugsbase, ghc-base and nhc-base packages should provide common set of low-level operations, As it happens I was working on getting GHC to use cabal to build base et al on the plane the other day, and I had a brief look at this. Unfortunately there is a tangled web of dependencies, e.g. you need the low level Int# stuff in ghc-base, then Int in base, but then any other GHC-specific stuff can't use Int because it's in base. We could put everything into ghc-base and just re-export the common stuff in base, but then we can't share any code between ghc, hugs etc. I haven't looked in detail to see just how bad the problem is, but I agree it would be really good if we could split things up somehow so that base (or whatever base gets split into) is the same everywhere. yes, it is one of problems that i was overlooked (and i expect that discussing my plan will show other problems i skipped by ignorance) first, let's specify that i propose (my today letter in haskell list contains more detailed plan): ghc-base should export Int operations. why? because it can't export Int# operations, they are not supported by other compilers (as the whole unboxed type concept), so they are useless to export. 'core' library should provide some common API. *hc-core libs should provide _subset_ of this API with hope that 'core' will emulate missing features but problem your mentioned still remains - while ghc-base defines operations on Int, it don't contains class Num definition, so that (*) or (+) operations can't be used. so that can we do? we should use intMul, intAdd and other operations directly. we can even define (*) and (+) operations for _internal_ use inside our ghc-base package, but not export them. while this seems a little Draconic, it will allow us to share Num defining code with other compilers and even introduce libraries with alternative Num/(*) definitions while idea of using some internal (*), (+) ... definitions may seem like work duplication, my experience says that it's much better to define duplicate operations for internal use only rather than try to implement whole Num class inside each compiler-specific library - because this definition should be a high-quality code and we don't want to copy such code over and over again and i hope that *hc-base libraries will not use Num operations too much because their main purpose is to give standard interface to compiler-specific functions, not to implement any algorithms. for example, looking to GHC.* modules in my own ArrayRef lib (which implements boxed and unboxed arrays), i don't see any arithmetic in _process_ of rewriting base lib, we should not have problems with GHC, because we can use recursive imports. but in order to retain compatibility with Hugs we may need to move Hugs.* modules inside 'base' package (the same for nhc). well, i don't know the best plan for intermediate versions. one possible but slow variant is to introduce intAdd/... operations, then rewrite ghc.*/hugs.*/... using these operations, then move out non-core stuff and then rewrite it back... -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: compiler-independent core libraries infrastructure
Hi Bulat, Just a partial answer for now: On Wed, Sep 13, 2006 at 12:29:58PM +0400, Bulat Ziganshin wrote: Friday, September 8, 2006, 5:52:57 AM, you wrote: what is a 'base' library now? it is the library that implements common set of operations for latest versions of ghc, hugs and nhc. it contains low-level implementation for ghc, but relies on separate hugsbase package for hugs (the same for nhc, afaiu). so, first step is obvious - separate ghc-base library from the rest. hugsbase, ghc-base and nhc-base packages should provide common set of low-level operations, As it happens I was working on getting GHC to use cabal to build base et al on the plane the other day, and I had a brief look at this. Unfortunately there is a tangled web of dependencies, e.g. you need the low level Int# stuff in ghc-base, then Int in base, but then any other GHC-specific stuff can't use Int because it's in base. We could put everything into ghc-base and just re-export the common stuff in base, but then we can't share any code between ghc, hugs etc. I haven't looked in detail to see just how bad the problem is, but I agree it would be really good if we could split things up somehow so that base (or whatever base gets split into) is the same everywhere. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
