Re: base package (Was: GHC 7.8 release?)
Hi, Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery kf8nh: On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote: On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner m...@joachim-breitner.de wrote: I don't think having FFI far down the stack is a problem. There are lots of pure data types we'd like in the pure data layer (e.g. bytestring) that uses FFI. As long as the I/O layer itself (System.IO, the I/O manager, etc) doesn't get pulled in there's no real problem in depending on the FFI. I think it would be nice, also to other Haskell implementations that might have not have FFI, to separate the really basic stuff from pure-but-impurely-implemented stuff. At least as long as it does not caues trouble. GHC.Fingerprint does not need to be crippled when it is going to use a pure hashing; I quickly added some simple fingerpriting found via Wikipedia that was easier than MD5. https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc The choice is of course not final, maybe something with more bits is desirable. Doesn't the FFI pull in some part of the I/O layer, though? In particular threaded programs are going to end up using forkOS? Another good reason to try to have a pure ground library. Based on base-pure, the next step would be to check if FFI can be provided without IO (but I doubt that is difficult), so there would be an base-io package on top of base-pure, and then bytestring can depend on that base-io and base-pure, while users of bytestring of course don’t have to depend on base-io (as long as they are not using the IO-related functions of bytestring). Questions: * Does anyone have a tool to compare package APIs? It would be interesting to diff base’s API with the combined APIs of the package we are creating right now. * Currently, base-pure exports lots of modules that should not be part of its “pure” API (all the GHC.* packages). But I assume they need to be exported for the benefit of packages like base-io built on top. Should we provide another package that re-exports those that are for public consumption and is likely to have a more stable API? Again I feel the need for packages re-exporting modules without incurring a conflict. * What to do with Prelude. Should it be in base-io (which is potentially the lowest package containing everything needed) or rather in a package of its own? Or should it live in a package of its own? Or can we use the haskell2010 package for that somehow? * Should base-io provide just the IO monad and all, say, file-related stuff in a separate package or is this going too far? Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: base package (Was: GHC 7.8 release?)
| Doesn't the FFI pull in some part of the I/O layer, though? In | particular threaded programs are going to end up using forkOS? | | Another good reason to try to have a pure ground library. Remember that we have UNSAFE ffi calls and SAFE ones. The SAFE ones may block, cause GC etc. They involve a lot of jiggery pokery and I would not be surprised if that affected the I/O manager. But UNSAFE ones are, by design, no more than fat machine instructions that are implemented by taking an out-of-line call. They should not block. They should not cause GC. Nothing. Think of 'sin' and 'cos' for example. Fingerprinting is a classic example, I would have thought. So my guess is that it should not be hard to allow UNSAFE ffi calls in the core (non-IO-ish) bits, leaving SAFE calls for higher up the stack. Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
Hi, more progress: On top of base-pure, I created base-io involving GHC/IO and everything required to build it (which pulled in ST, some of Foreign and unfortunately some stuff related to Handles and Devices, because it is mentioned in IOException). This is the list of modules: Foreign.C.Types, Foreign.ForeignPtr, Foreign.ForeignPtr.Imp, Foreign.ForeignPtr.Safe, Foreign.ForeignPtr.Unsafe, Foreign.Ptr, Foreign.Storable, GHC.ForeignPtr, GHC.IO.BufferedIO, GHC.IO.Buffer, GHC.IO.Device, GHC.IO.Encoding.Types, GHC.IO.Exception, GHC.IO.Handle.Types, GHC.IO, GHC.IORef, GHC.MVar, GHC.Ptr, GHC.Stable, GHC.ST, GHC.Storable, GHC.STRef It is on a different branch on my github repo: https://github.com/nomeata/packages-base/tree/base-io GHC would complain that the CInt type is not valid in a ffi call (probably due to the different package name), so I replaced foreign declarations by regular ones defined using “undefined” – after all I’m just trying to discover how things can be split at all and just work towards building stuff. ST can probably be pulled below this package, after all it is quite pure. Either a package of its own, or in base-pure. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
On 15/02/13 09:36, Simon Peyton-Jones wrote: | Doesn't the FFI pull in some part of the I/O layer, though? In | particular threaded programs are going to end up using forkOS? | | Another good reason to try to have a pure ground library. Remember that we have UNSAFE ffi calls and SAFE ones. The SAFE ones may block, cause GC etc. They involve a lot of jiggery pokery and I would not be surprised if that affected the I/O manager. But UNSAFE ones are, by design, no more than fat machine instructions that are implemented by taking an out-of-line call. They should not block. They should not cause GC. Nothing. Think of 'sin' and 'cos' for example. Fingerprinting is a classic example, I would have thought. So my guess is that it should not be hard to allow UNSAFE ffi calls in the core (non-IO-ish) bits, leaving SAFE calls for higher up the stack. Actually as far as the Haskell-level API goes, there's no difference between safe and unsafe FFI calls, the difference is all in the codegen. I don't think safe calls cause any more difficulties for splitting up the base. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
On 15/02/13 08:36, Joachim Breitner wrote: Hi, Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery kf8nh: On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote: On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner m...@joachim-breitner.de wrote: I don't think having FFI far down the stack is a problem. There are lots of pure data types we'd like in the pure data layer (e.g. bytestring) that uses FFI. As long as the I/O layer itself (System.IO, the I/O manager, etc) doesn't get pulled in there's no real problem in depending on the FFI. I think it would be nice, also to other Haskell implementations that might have not have FFI, to separate the really basic stuff from pure-but-impurely-implemented stuff. At least as long as it does not caues trouble. GHC.Fingerprint does not need to be crippled when it is going to use a pure hashing; I quickly added some simple fingerpriting found via Wikipedia that was easier than MD5. https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc The choice is of course not final, maybe something with more bits is desirable. Remember that fingerprinting is not hashing. For fingerprinting we need to have a realistic expectation of no collisions. I don't think FNV is suitable. I'm sure it would be possible to replace the C md5 code with some Haskell. Performance *is* important here though - Typeable is in the inner loop of certain generic programming libraries, like SYB. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
On 15/02/13 12:22, Joachim Breitner wrote: Hi, more progress: On top of base-pure, I created base-io involving GHC/IO and everything required to build it (which pulled in ST, some of Foreign and unfortunately some stuff related to Handles and Devices, because it is mentioned in IOException). This is the list of modules: Foreign.C.Types, Foreign.ForeignPtr, Foreign.ForeignPtr.Imp, Foreign.ForeignPtr.Safe, Foreign.ForeignPtr.Unsafe, Foreign.Ptr, Foreign.Storable, GHC.ForeignPtr, GHC.IO.BufferedIO, GHC.IO.Buffer, GHC.IO.Device, GHC.IO.Encoding.Types, GHC.IO.Exception, GHC.IO.Handle.Types, GHC.IO, GHC.IORef, GHC.MVar, GHC.Ptr, GHC.Stable, GHC.ST, GHC.Storable, GHC.STRef You have a random collection of modules here :) I think you want to have the IO *monad* (GHC.IO) live in a lower layer, separate from the IO *library* (GHC.IO.Device and so on). Every Haskell implementation will need the IO monad, but they might want to replace the IO library with something else. Things like GHC.IORef, GHC.MVar can all live in a low-down layer because they're just wrappers over the primops. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Building the GHC library without building GHC
Hi, I was going to do some hacking on Haddock. Haddock depends on the GHC API from the development version of GHC, however, stage2 crashes on my system when I try to build the newest GHC from the repository. Since I don't actually need to compile with the new GHC, I'm hoping there's a workaround. Is there a way to build only the GHC library so that I can get back to Haddock? ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Building the GHC library without building GHC
Er, the GHC library *is* GHC. The answer is very probably no. -- brandon s allbery kf8nh Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, February 15, 2013 at 9:52 AM, C Rodrigues wrote: Hi, I was going to do some hacking on Haddock. Haddock depends on the GHC API from the development version of GHC, however, stage2 crashes on my system when I try to build the newest GHC from the repository. Since I don't actually need to compile with the new GHC, I'm hoping there's a workaround. Is there a way to build only the GHC library so that I can get back to Haddock? ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org (mailto:Glasgow-haskell-users@haskell.org) http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
On Thu, Feb 14, 2013 at 03:48:51PM +0100, Joachim Breitner wrote: Yesterday, I experimented a bit with base’s code, [...] Maybe the proper is to reverse the whole approach: Leave base as it is, and then build re-exporting smaller packages (e.g. a base-pure) on top of it. The advantage is: * No need to rewrite the tightly intertwined base. * Libraries still have the option to have tighter dependencies. * Base can evolve with lots of breaking changes, as long as they do not affect the API by the smaller packages. * Development of this collection can happen outside the GHC tree. * Alternative implementations for (some of) these packages can be created, if the reason why they could not be moved out of base is one of implementation, not of API How does that sound? Essentially good to me... One might consider instead (as has been proposed before, I believe), to rename the current ``base'' to something like ``ghc-base'' which is not intended to be depended on by packages not shipped with GHC (that is, by default ``hidden'' in ghc-pkg), and instead export: base with a very stable interface io with a very stable interface GHCwith a probably rapidly evolving interface. * possibly other packages giving access to internals Most packages that currently depend on ``base'' would then depend only on ``base'' and possibly ``io'', and by virtue of the stability of these two interfaces would therefore not be affected by most GHC releases. This would effectively be ``splitting the interfaces GHC and io out from base'' instead of ``deprecating base and replacing it with the three new interfaces base-pure, io, and GHC''. That choice is possibly mostly a matter of taste --- I think that the name ``base'' is good for a user-facing interface, and the name ``ghc-base'' more indicative of its implementation-dependent character. Wolfram ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
On Thu, Feb 14, 2013 at 6:48 AM, Joachim Breitner m...@joachim-breitner.dewrote: Maybe the proper is to reverse the whole approach: Leave base as it is, and then build re-exporting smaller packages (e.g. a base-pure) on top of it. The advantage is: * No need to rewrite the tightly intertwined base. * Libraries still have the option to have tighter dependencies. * Base can evolve with lots of breaking changes, as long as they do not affect the API by the smaller packages. * Development of this collection can happen outside the GHC tree. * Alternative implementations for (some of) these packages can be created, if the reason why they could not be moved out of base is one of implementation, not of API How does that sound? I'm not in favor of this approach as it precludes pushing any data types down the stack. In particular, we want text and bytestring to be below the I/O layer, so we can defined Handles that work with those in base itself. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
As a super obvious aside, we could just switch code paths based on platform/environment. That lets you keep the fast code path and have a pure fallback for the javascripters. Just propagate an FFI_AVAILABLE flag into the compilation unit. We're going to have a number of these points which force a tension between generality And speed as we go, and it'd nice to have the ability to gracefully fall back. On Feb 15, 2013, at 9:45 AM, Simon Marlow marlo...@gmail.com wrote: On 15/02/13 08:36, Joachim Breitner wrote: Hi, Am Donnerstag, den 14.02.2013, 21:41 -0500 schrieb brandon s allbery kf8nh: On Thursday, February 14, 2013 at 8:14 PM, Johan Tibell wrote: On Thu, Feb 14, 2013 at 2:53 PM, Joachim Breitner m...@joachim-breitner.de wrote: I don't think having FFI far down the stack is a problem. There are lots of pure data types we'd like in the pure data layer (e.g. bytestring) that uses FFI. As long as the I/O layer itself (System.IO, the I/O manager, etc) doesn't get pulled in there's no real problem in depending on the FFI. I think it would be nice, also to other Haskell implementations that might have not have FFI, to separate the really basic stuff from pure-but-impurely-implemented stuff. At least as long as it does not caues trouble. GHC.Fingerprint does not need to be crippled when it is going to use a pure hashing; I quickly added some simple fingerpriting found via Wikipedia that was easier than MD5. https://github.com/nomeata/packages-base/commit/b7f80066a03fd296950e0cafa2278d43a86f82fc The choice is of course not final, maybe something with more bits is desirable. Remember that fingerprinting is not hashing. For fingerprinting we need to have a realistic expectation of no collisions. I don't think FNV is suitable. I'm sure it would be possible to replace the C md5 code with some Haskell. Performance *is* important here though - Typeable is in the inner loop of certain generic programming libraries, like SYB. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package (Was: GHC 7.8 release?)
Hi, Am Freitag, den 15.02.2013, 14:50 + schrieb Simon Marlow: On 15/02/13 12:22, Joachim Breitner wrote: Hi, more progress: On top of base-pure, I created base-io involving GHC/IO and everything required to build it (which pulled in ST, some of Foreign and unfortunately some stuff related to Handles and Devices, because it is mentioned in IOException). This is the list of modules: You have a random collection of modules here :) I think you want to have the IO *monad* (GHC.IO) live in a lower layer, separate from the IO *library* (GHC.IO.Device and so on). Every Haskell implementation will need the IO monad, but they might want to replace the IO library with something else. Things like GHC.IORef, GHC.MVar can all live in a low-down layer because they're just wrappers over the primops. Right, that is my aim, and I started with GHC.IO. But unfortunately, the IO monad calls failIO, which is an IOError which has a field of type ioe_handle :: Maybe Handle (and one of type CInt) which pulls in all the rest there, and so far I did not have a good idea how to untangle that. What would break if fail would not raise an IOError, but a separate exception type, e.g. IOFailError? Probably too much, as users expect to catch the exception raised by fail with an exception handler that matches IOError. Greetings, Joachim -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
