Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
Also, I'm not really sure this is a bug, per se. In my opinion when you allow for some sort of generic operations (whether via GHC.Generics or Data), it's roughly equivalent to exporting all the constructors. I don't see how it would work otherwise. But since there's precedent for Typeable, maybe Generics should be restricted in SafeHaskell as well. On Mon, Oct 7, 2013 at 1:05 AM, John Lato jwl...@gmail.com wrote: Well, no. Presumably the example shouldn't compile at all. That message is more an indication that the demonstration is working as intended. On Mon, Oct 7, 2013 at 12:31 AM, Carter Schonwald carter.schonw...@gmail.com wrote: i assume https://github.com/JohnLato/safe-bugtest/blob/master/Main.hs#L13should say putStrLn Should print \Pos (2)\ rather than -2? On Mon, Oct 7, 2013 at 1:23 AM, Carter Schonwald carter.schonw...@gmail.com wrote: ooo, thats illuminating. thanks for cooking that up On Mon, Oct 7, 2013 at 1:13 AM, John Lato jwl...@gmail.com wrote: On Sun, Oct 6, 2013 at 10:14 PM, Ryan Newton rrnew...@gmail.comwrote: On Sun, Oct 6, 2013 at 6:28 PM, Ganesh Sittampalam gan...@earth.liwrote: - Referential transparency: e.g. no unsafePerformIO - Module boundary control: no abstraction violation like Template Haskell and GeneralizedNewtypeDeriving - Semantic consistency: importing a safe module can't change existing code, so no OverlappingInstances and the like Is this change necessary to preserve the existing properties, or are you hoping to add a new one? I'm not currently aware of ways to break these invariants *just* with GHC.Generics. Hmm, but I would like to know why it is marked trustworthy and not inferred-safe... How about this demo repo? https://github.com/JohnLato/safe-bugtest I'm really not a safe haskell expert, but I believe this is a demonstration of using GHC.Generics to violate a module's abstraction boundaries with SafeHaskell enabled. If I'm incorrect, I would appreciate if somebody could explain my error. If, however, I'm correct, then I think that Ryan's proposal of marking GHC.Generics Unsafe is the best way to remedy the problem. A possible stumbling block may involve base and package-trust, but I'm not certain of the current status. John L. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
Andres is right that it's not as evil as defining your own Typeable. The crux of the matter is that Generic essentially allows full access to the data type. Unfortunately it's easy to forget this... On Mon, Oct 7, 2013 at 1:43 AM, Andres Löh and...@well-typed.com wrote: While I understand you all feel uncomfortable with this, I do not think the problem demonstrated by John has anything to do with Generic. I've made a fork here https://github.com/kosmikus/safe-bugtest that shows an (IMHO) similar problem using Show and Read instead of Generic. (And something slightly different could certainly also produced using Enum). If you're deriving Generic, then yes, you gain the functionality of that class, which is to inspect the structure of the type, and then yes, you can in principle construct any value of that type. So deriving Generic for types that should be abstract is always going to be risky. But this is no different than deriving any other class, only that Generic gives you particularly fine-grained access to the internals of a type. Also, at least in my opinion, it is entirely valid to define your own Generic instances. It's more work, and while I haven't used it often so far, I can imagine that there are good use cases. I don't think it's anywhere near as evil as defining your own Typeable instances. Cheers, Andres -- Andres Löh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself? Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead? The new rule would be that if you hand-implemented Generic or Generic1 in your module it isn't Safe. That would make it so that derived Generic, Generic1 would be considered Safe, and you wouldn't break literally every user of the library who care about Safe Haskell. As it stands the things are damn-near impossible to get right instantiating them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them! -Edward On Sun, Oct 6, 2013 at 5:32 PM, Ryan Newton rrnew...@gmail.com wrote: Hi all, We had a discussion on haskell-cafe, which only confirmed the unreliability of Eq and Ord. Fine. But if I instead want to define a SafeEq class and instances based on GHC.Generics, then I can't have users writing Generic instances that lie about the structure of their types. But as you can see this module is currently marked Trustworthy: http://www.haskell.org/ghc/docs/7.4.1/html/libraries/ghc-prim-0.2.0.0/GHC-Generics.html I simply propose that this is changed to Unsafe! [1] *Context:* The bottom line is that I believe in the Safe-Haskell mission here, and I want to be able to deliver practical libraries that give a strong guarantee that types can't be broken. Currently, the LVishhttp://www.cs.indiana.edu/~rrnewton/papers/2013_07_LVish_quasiDet_working_draft.pdflibrary has one big hole in it because of this Eq/Ord limitation; the problem is documented herehttp://www.cs.indiana.edu/~rrnewton/haddock/lvish/Control-LVish.html#g:1 . If we can provide incontrovertible Safe-Haskel guarantees, this is a huge, qualitative difference compared to what's possible in the C++, Java's and MLs of the world. There are plenty of libraries that provide guarantees like deterministic parallelism IFF the user does everything right and breaks no rules (CnC, Pochoir, etc). But we can do better! Best, -Ryan [1] Small detail... some of these bindings, like the class name Generic itself, will still need to be accessible from a Trustworthy library. I propose GHC.Generics.Safe for that, following existing precedent. On Sun, Oct 6, 2013 at 5:18 PM, Ryan Newton rrnew...@gmail.com wrote: Thanks for the responses all. I'm afraid the point about GHC.Generics got lost here. I'll respond and then rename this as a specific library proposal. I don't want to fix the world's Eq instances, but I am ok with requiring that people derive Generic for any data they want to put in an LVar container. (From which I can give them a SafeEq instance.) It's not just LVish that is in this boat any library that tries to provide deterministic parallelism outside of the IO monad has some very fine lines to walk. Take a look at Accelerate. It is deterministic (as long as you run only with the CUDA backend and only on one specific GPU... otherwise fold topologies may look different and non-associative folds may leak). Besides, runAcc does a huge amount of implicit IO (writing to disk, calling nvcc, etc)! At the very least this could fail if the disk if full. But then again, regular pure computations fail when memory runs out... so I'm ok grouping these in the same bucket for now. Determinism modulo available resources. A possible problem with marking instance Eq as an unsafe feature is that many modules would be only Trustworthy instead of Safe. My proposal is actually more narrow than that. My proposal is to mark GHC.Generics as Unsafe. That way I can define my own SafeEq, and know that someone can't break it by making a Generic instance that lies. It is very hard for me to see why people should be able to make their own Generic instances (that might lie about the structure of the type), in Safe-Haskell. That would go against my every purely functional module is automatically safe because the compiler checks that it cannot launch the missiles understanding of Safe Haskell. Heh, that may already be violated by the fact that you can't use other extensions like OverlappingInstances, or provide your own Typeable instances. Actually, Eq instances are not unsafe per se, but only if I also use some other module that assumes certain properties about all Eq instances in scope. So in order to check safety, two independent modules (the provider and the consumer of the Eq instance) would have to cooperate. I've found, that this is a very common problem that we have when trying to make our libraries Safe-Haskell compliant -- often we want to permit and deny combinations of modules. I don't have a solution I'm afraid. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
Here‘s a thought: doesn’t Generic already have an unused phantom type that's only there “just in case we need to add something in the future”? How about using that to track whether an instance was derived or not, and enforce *that* with SafeHaskell? Your SafeEq can then use a Generic Derived constraint or whatever. Ah this sounds promising. You mean that 'x' param in from/to? So to use that, whereever I call to/from I would have to constrain that x to be Derived. But do I understand correctly that GHC would have to modified so that its derived instances of Generic force that x param to be Derived? ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett ekm...@gmail.com wrote: Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself? Exactly. Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead? The new rule would be that if you hand-implemented Generic or Generic1 in your module it isn't Safe. Ah... so you're proposing adding an extra rule to the GHC compiler itself? Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.) That would make it so that derived Generic, Generic1 would be considered Safe, and you wouldn't break literally every user of the library who care about Safe Haskell. Isn't that also accomplished just by import GHC.Generic.Safe (Generic)? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-). As it stands the things are damn-near impossible to get right instantiating them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them! Exactly! ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
I'd much rather have code that can compile as Safe rather than Trustworthy. Trustworthy code is a pain to annotate the safety of correctly. A one line change to GHC to ensure the safety of GHC.Generics and to maximize the number of operations you can use safely seems entirely reasonable. Otherwise I have to mark perfectly Safe-by-construction code Trustworthy, enlarging the trusted code base considerably. Using properly constructed Generic instances is perfectly safe. That library level fix has knock-on effects across the entire ecosystem for anyone who cares about SafeHaskell. I tend to instantiate GHC.Generics for any type in most of my packages. A quick grep finds 209 modules broken in just the projects I have open in my working directory. That is a mixture of public and private repos. The majority of which are public. So, I first have to change a couple hundred import statements... conditionally on compiler version, since I maintain wide support windows. This fix would literally break SafeHaskell for me silently across almost everything I have written, forcing me to run through tons of packages trying to figure out what I have to change to get it back. I've done this song and dance before trying to get lambdabot to build, since it uses lens these days. If I don't get every one of them right lens builds as unsafe. There isn't a good debugging process for this, as the only real way I have to know it worked is to go through each one and make sure I didn't compromise safe haskell by looking at the resulting haddocks for each case. The code that *uses* GHC.Generics uses their internals and is safe by construction. It is only the instance construction that is unsafe, so lets mark it as such. -Edward On Mon, Oct 7, 2013 at 11:11 AM, Ryan Newton rrnew...@gmail.com wrote: On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett ekm...@gmail.com wrote: Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself? Exactly. Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead? The new rule would be that if you hand-implemented Generic or Generic1 in your module it isn't Safe. Ah... so you're proposing adding an extra rule to the GHC compiler itself? Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really want you can mark your module as TrustWorthy even with manual instances. (And then, yes, we'd need a GHC.Generics.Safe to pull the Generic symbol itself from when we just want to derive it.) That would make it so that derived Generic, Generic1 would be considered Safe, and you wouldn't break literally every user of the library who care about Safe Haskell. Isn't that also accomplished just by import GHC.Generic.Safe (Generic)? And the handful of hypothetical people that are using this right now can make that one-line fix. I will *personally* make that fix for all those people if they would come forward and share the link to their code ;-). As it stands the things are damn-near impossible to get right instantiating them by hand anyways, so I expect this would affect only 1 or 2 users rather than all of them! Exactly! ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
If I understand correctly, then all of the code in question should be in compiler/typecheck/TcInstDcls.lhs There are two code paths that would need changing around line 418: ; when (safeLanguageOn dflags) $ mapM_ (\x - when (typInstCheck x) (addErrAt (getSrcSpan $ iSpec x) typInstErr)) local_infos ; when (safeInferOn dflags) $ mapM_ (\x - when (typInstCheck x) recordUnsafeInfer) local_infos The first is when the Safe pragma is present, the second is when we're merely inferring safety. You'd need to figure out how you wanted to refactor the helper functions splitTypeable, typInstCheck, typInstErr, and instMsg in the where clause immediately below, or if you just wanted to duplicate the Typeable logic. -Edward On Mon, Oct 7, 2013 at 1:33 PM, Ryan Newton rrnew...@gmail.com wrote: Thanks Edward! That provides a real example of someone who uses both SafeHaskell and GHC.Generics in their libraries! I'm glad to know such a person exists but that you don't depend on handwritten Generic instances. (I'm also glad to hear that you care about SafeHaskell in the first place!) I totally agree that #ifdef'ing on GHC version is a pain, and all that code would need to conditionally import GHC.Generics or GHC.Generics.Safe to retain -XSafe status. I'm perfectly happy with the GHC-internal fix if someone can point me to how to do it / where it should go. I guess I'll start by greping for Typeable in the compiler. Is this something you can do trivially Edward? (Or others?) Just for reference here is a small patch to Base representing my library-based way of making the change discussed in this thread: https://github.com/rrnewton/packages-base/commit/23f8f464a0b46ca17c655c4144f7643c4bc6ea61 Again, my use-case (closed/safe Eq Ord instances) is a few steps removed from this Generics business and I don't feel strongly about how it gets done. But having *SOME* way of doing it will be a massive boon to my lab's particular effort at maximizing the abilities of guaranteed-Safe deterministic parallelism in Haskell. Further, I think anyone else who tries to do something in SafeHaskell that depends on guaranteeing a closed/controlled set of instances (SafeEq, SafeOrd but also others) based on GHC.Generics will *also* find this change useful. Best, -Ryan On Mon, Oct 7, 2013 at 12:44 PM, Edward Kmett ekm...@gmail.com wrote: I'd much rather have code that can compile as Safe rather than Trustworthy. Trustworthy code is a pain to annotate the safety of correctly. A one line change to GHC to ensure the safety of GHC.Generics and to maximize the number of operations you can use safely seems entirely reasonable. Otherwise I have to mark perfectly Safe-by-construction code Trustworthy, enlarging the trusted code base considerably. Using properly constructed Generic instances is perfectly safe. That library level fix has knock-on effects across the entire ecosystem for anyone who cares about SafeHaskell. I tend to instantiate GHC.Generics for any type in most of my packages. A quick grep finds 209 modules broken in just the projects I have open in my working directory. That is a mixture of public and private repos. The majority of which are public. So, I first have to change a couple hundred import statements... conditionally on compiler version, since I maintain wide support windows. This fix would literally break SafeHaskell for me silently across almost everything I have written, forcing me to run through tons of packages trying to figure out what I have to change to get it back. I've done this song and dance before trying to get lambdabot to build, since it uses lens these days. If I don't get every one of them right lens builds as unsafe. There isn't a good debugging process for this, as the only real way I have to know it worked is to go through each one and make sure I didn't compromise safe haskell by looking at the resulting haddocks for each case. The code that *uses* GHC.Generics uses their internals and is safe by construction. It is only the instance construction that is unsafe, so lets mark it as such. -Edward On Mon, Oct 7, 2013 at 11:11 AM, Ryan Newton rrnew...@gmail.com wrote: On Mon, Oct 7, 2013 at 10:22 AM, Edward Kmett ekm...@gmail.com wrote: Am I correct in understanding your issue arises from manually rolled instances of Generic, not from Generic itself? Exactly. Wouldn't then perhaps the better fix be to resurrect the old rule for derived Typeable instances and apply it to Generic and Generic1 instead? The new rule would be that if you hand-implemented Generic or Generic1 in your module it isn't Safe. Ah... so you're proposing adding an extra rule to the GHC compiler itself? Isn't it simpler to do it just with a library-level fix? I.e. my proposal is that GHC.Generics is marked as Unsafe. And then if you really
Re: Proposal: GHC.Generics marked UNSAFE for SafeHaskell
My understanding is that the proposed changes to ghc/Generic would mean that all data used with LVish would have to expose it's implementation to everybody through the derived Generic instance. Either that or they would have use TrustWorthy to mark a module containing custom equality/ordering operations as safe according to our definition, and take the proof obligations that come with that. In fact, this is what we would plan to do to opt in ADTs like bytestring, vector, containers. We can trust as many people/libraries as we like, but we want to retain the ability to run untrusted code with the right set of flags to ensure that it can't break our library. (And that's what we can't do without locking down the Generic backdoor.) ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
ghc-compete, a repository of fingerprints, and continuous integration
Dear Devs, in the best scratch-an-itch-tradition I have created a special git repository, called “ghc-complete”, which tracks the combined state of all GHC-related repositories. It uses GHC’s fingerprint utility and simply tracks the change of the fingerprint file over time: https://github.com/nomeata/ghc-complete The main use of this is to have a repo which I can enable to be continuously checked by Travis (a free CI hoster). So instead of having several people ask on IRC about what build or test suite failures are actually not their fault, they can now simply check the travis page: https://www.travis-ci.org/nomeata/ghc-complete/builds The Travis tests are a bit stripped down, in order to finish reliably before the 50 minute timeout, see https://github.com/nomeata/ghc-complete/blob/master/validate.sh for the precise options. The log files are too large for the JavaScript enhanced view on Travis. So if you click on a specific build result, e.g. https://www.travis-ci.org/nomeata/ghc-complete/builds/12216328 then better immediately browse to the raw build log (the rounded square button in the top right corner with the four horizontal bars). Another use of this repo might be bisecting a problem. If you find this useful, I’d propose to let this run on git.haskell.org proper, and also not as a cronjob (every 15 minutes at the moment), but rather triggered by the actual pushes. On my TODO list is to make the commit messages of the repository more useful, by including the log of the new commits in it. If (not sure if this is planned to that extend) eventually all related repositories of GHC are submodules, this will have become obsolete. Which of course is fine. Enjoy, Joachim -- Joachim “nomeata” Breitner m...@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nome...@joachim-breitner.de • GPG-Key: 0x4743206C Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: -dynamic: all or nothing?
please share what you find out! I too, wonder if i can (ab)use dylibs as static libs for the end executable On Mon, Oct 7, 2013 at 4:22 PM, Andreas Voellmy andreas.voel...@gmail.comwrote: Hi all, From what I gather, using -dynamic with ghc --make compiles an executable using dynamic linking for ALL dependent Haskell libraries. Is it possible to choose to link some dependent libraries to link statically and some to link dynamically? If so, how can I do this? -Andi ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
#8276
Is there a workaround for http://ghc.haskell.org/trac/ghc/ticket/8276. I haven't been able to build HEAD for a while due to this issue. -Andi ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: #8276
Andi, Is there a workaround for http://ghc.haskell.org/trac/ghc/ticket/8276. I haven't been able to build HEAD for a while due to this issue. Put the following one line to your mk/build.mk. HADDOCK_DOCS = NO The entire contents of my build.mk is: InstallExtraPackages = YES HADDOCK_DOCS = NO --Kazu ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: #8276
Thanks! That worked. On Tue, Oct 8, 2013 at 12:35 AM, Kazu Yamamoto k...@iij.ad.jp wrote: Andi, Is there a workaround for http://ghc.haskell.org/trac/ghc/ticket/8276. I haven't been able to build HEAD for a while due to this issue. Put the following one line to your mk/build.mk. HADDOCK_DOCS = NO The entire contents of my build.mk is: InstallExtraPackages = YES HADDOCK_DOCS = NO --Kazu ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
