Interruptible exception wormholes kill modularity

[Edward Z. Yang]

In 2010, in the thread "Asynchronous exception wormholes kill modularity" [1],
Bas van Dijk observed that 'unblock :: IO a -> IO a' broke modularity,
as the sequence of calls 'block . block . unblock $ io' would result in
'io' being run with asynchronous exceptions unblocked, despite the outer
'block' "expecting" that asynchronous exceptions cannot be thrown.

I would like to make two claims:

1. The new mask/restore interface is insufficient to "solve"
this modularity problem, as *interruptible* operations can
still be used to catch asynchronous exceptions.

2. Thus, we should provide an unblock combinator which
can be used to catch asynchronous exceptions from a 'mask'
(though not an 'uninterruptibleMask')--though it is
doubtful if anyone should ever use 'mask' in the first
place.

Claim 1: Here is some code which reimplements 'unblock':

import Control.Exception
import Control.Concurrent
import Control.Concurrent.MVar

unblock :: IO a -> IO a
unblock io = do
m <- newEmptyMVar
_ <- forkIO (io >>= putMVar m)
takeMVar m

The main idea is that 'takeMVar' is an interruptible operation:
when it blocks, the thread can now receive asynchronous exceptions.
In general, a thread can unmask exceptions by blocking.  Here
is a simple test-case:

main = do
let x = 1000 -- Just do a bit of work
tid <- myThreadId
forkIO $ (threadDelay 1 >> killThread tid)
r <- mask $ \restore -> do
-- restore $ do
-- unblock $ do
-- do something non-blocking
evaluate (f x [])
-- If the exception is delivered in a timely manner,
-- shouldn't get here.
print r

f 0 r = r
f n r = f (n-1) (n:r)

With both restore and unblock commented, the ThreadKilled
exception is delayed; uncommenting either restore or unblock
causes the exception to be delivered.

This admonition does not apply to uninterruptibleMask, for
which there are no interruptible exceptions.

Claim 2:  Thus, I come to the conclusion that we were wrong
to remove 'unblock', and that it is no worse than the
ability for interruptible actions to catch asynchronous
exceptions.

You could very well argue that interruptible actions are a design flaw.
Then you should use 'uninterruptibleMask' instead, which effectively
removes the concept of interruptibility--and is thus modular.  Indeed,
Eyal Lotem proposed [2] that 'bracket' should instead use
'uninterruptibleMask', for precisely the reason that it is too easy to
reenable asynchronous exceptions in 'mask'.  But assuming that
interruptible masks are a good idea (Simon Marlow has defended them
as "a way avoid reasoning about asynchronous exceptions except
at specific points, i.e., where you might block"), there should
be an 'unblock' for this type of mask.

It should be said that the absence of 'unblock' for
'uninterruptibleMask' only implies that a passed in IO action (e.g., the
cleanup action in bracket) does not have access to the exceptions thrown
to the current thread; it doesn't actually guarantee uninterruptibility,
since the passed in IO action could always raise a normal exception.
Haskell's type system is not up to the task of enforcing such
invariants.

Cheers,
Edward

[1]
https://mail.haskell.org/pipermail/libraries/2010-March/013310.html
https://mail.haskell.org/pipermail/libraries/2010-April/013420.html

[2]
https://mail.haskell.org/pipermail/libraries/2014-September/023675.html

P.S. You were CC'ed to this mail because you participated in the original
"Asynchronous exception wormholes kill modularity" discussion.

P.P.S. I have some speculations about using uninterruptibleMask more
frequently: it seems to me that there ought to be a variant of
uninterruptibleMask that immediately raises an exception if
the "uninterruptible" action blocks.  This would probably of
great assistance of noticing and eliminating blocking in
uninterruptible code.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Template Haskell determinism

[Edward Z. Yang]

Oh drat, that's right, local names don't get given a package
key / package id, and externally visible local names aren't given a
deterministic name until we tidy (which is too late to help
Template Haskell.)  So I suppose there is not much we can do here.

Edward

Excerpts from Michael Sloan's message of 2016-06-29 13:41:13 -0400:
> No, NameU and NameL both lack package key / package id.
> 
> -Michael
> 
> On Wed, Jun 29, 2016 at 7:34 AM, Edward Z. Yang  wrote:
> > No, nameBase is not the right thing to use here; you also need the
> > unit ID (in GHC 8.0 parlance; package key in GHC 7.10; package id
> > in GHC 7.8 and before).  If you have that information, then
> > GHC establishes an invariant that if two names compare stably equal,
> > then the uniques associated with them are the same.
> >
> > Edward
> >
> > Excerpts from Michael Sloan's message of 2016-06-10 17:16:44 -0400:
> >> Hey, sorry for not getting back to this sooner!
> >>
> >> Perhaps I should have added the following to my list of goals in 
> >> contention:
> >>
> >> (3) (==) shouldn't yield True for Names that have different unique ids.
> >>
> >> We can only have stable comparisons if goal (3) isn't met, and two
> >> different unique Names would be considered to be equivalent based on the
> >> nameBase.  This is because Ord is a total order, not a partial order.  As
> >> described in my prior email, PartialOrd could be added, but it'd be
> >> inconvenient to use with existing Ord based containers.
> >>
> >> -Michael
> >>
> >> On Sun, Jun 5, 2016 at 10:15 AM, Edward Z. Yang  wrote:
> >>
> >> > I must admit, I am a bit confused by this discussion.
> >> >
> >> > It is true that every Name is associated with a Unique.  But you don't
> >> > need the Unique to equality/ordering tests; the names also contain
> >> > enough (stable) information for stable comparisons of that sort.  So
> >> > why don't we expose that instead of the Unique?
> >> >
> >> > Edward
> >> >
> >> > Excerpts from Michael Sloan's message of 2016-06-04 18:44:03 -0700:
> >> > > On Thu, Jun 2, 2016 at 4:12 AM, Simon Peyton Jones <
> >> > simo...@microsoft.com>
> >> > > wrote:
> >> > >
> >> > > > If names get different ordering keys when reified from different
> >> > modules
> >> > > > (seems like they'd have to, particularly given ghc's "-j"), then we
> >> > end up
> >> > > > with an unpleasant circumstance where these do not compare as equal
> >> > > >
> >> > > >
> >> > > >
> >> > > > The I believe that global, top level names (NameG) are not subject to
> >> > this
> >> > > > ordering stuff, so I don’t think this problem can occur.
> >> > > >
> >> > >
> >> > > True, top level names are NameG.  The reified Info for a top level Dec
> >> > may
> >> > > include NameU, though.  For example, the type variables in 'Maybe' are
> >> > > NameU:
> >> > >
> >> > > $(do TyConI (DataD _ _ [KindedTV (Name _ nf) _] _ _ _) <- reify ''Maybe
> >> > >  lift (show nf))
> >> > >
> >> > > The resulting expression is something like "NameU 822083586"
> >> > >
> >> > > > This is a breaking change and it doesn't fix the problem that
> >> > NameFlavour
> >> > > > is
> >> > > >
> >> > > > not abstract and leaks the Uniques. It would break at least:
> >> > > >
> >> > > >
> >> > > >
> >> > > > But why is NameU exposed to clients?   GHC needs to know, but clients
> >> > > > don’t.  What use are these packages making of it?
> >> > > >
> >> > >
> >> > > It's being leaked in the public inteface via Ord.  The Eq instance is
> >> > fine,
> >> > > because these are Uniques, so the results should be consistent.
> >> > >
> >> > > There are two goals in contention here:
> >> > >
> >> > > 1) Having some ordering on Names so that they can be used in Map or Set
> >> > > 2) Having law-abiding Eq / Ord instances.  We'd need a 'PartialOrd' to
> >> > > really handle these well.  In that case, the ordering would be based on
> >> > > everything but the NameU int, but 'Eq' would still follow it
> >> > >
> >> > > A few ideas for different approaches to resolving this:
> >> > >
> >> > > 1) Document it.  Less appealing than fixing it in the API, but still
> >> > would
> >> > > be good.
> >> > >
> >> > > 2) Remove the 'Ord' instance, and force the user to pick 
> >> > > 'NamePartialOrd'
> >> > > newtype (partial ord on the non-unique info), or 'UnstableNameOrd'
> >> > newtype
> >> > > (current behavior).  A trickyness of this approach is that you'd need
> >> > > containers that can handle (PartialOrd k, Eq k) keys.  In lots of cases
> >> > > people are using the 'Ord' instance with 'Name's that are not 'NameU', 
> >> > > so
> >> > > this would break a lot of code that was already deterministic.
> >> > >
> >> > > 3) Some approaches like this ordering key, but I'm not sure how it will
> >> > > help when comparing NameUs from different modules?
> >> > >
> >> > > > S
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of
> 

Re: Linker.c broken

[Edward Z. Yang]

I'm guessing it's:

commit 6377757918c1e7f63638d6f258cad8d5f02bb6a7
Author: Simon Marlow 
Date:   Wed Jun 29 21:50:18 2016 +0100

Linker: some extra debugging / logging

which added ghci_find.

Edward

Excerpts from Simon Peyton Jones via ghc-devs's message of 2016-07-01 18:51:20 
-0400:
> Aargh!  Windows is broken /again/.   Some mess-up in Linker.c.
> I have not yet tried reverting recent patches.  Might someone fix please?
> It’s really helpful to validate on Windows when making RTS changes.
> Simon
> 
> 
> 
> rts\Linker.c: In function 'ghci_find':
> 
> 
> 
> rts\Linker.c:1482:52: error:
> 
>  error: pointer type mismatch in conditional expression [-Werror]
> 
>   oc->archiveMemberName : oc->fileName);
> 
> ^
> 
> 
> 
> rts\Linker.c:1480:28: error:
> 
>  error: format '%ls' expects argument of type 'wchar_t *', but argument 3 
> has type 'void *' [-Werror=format=]
> 
>  debugBelch("%p is in %" PATH_FMT, addr,
> 
> ^
> 
> "inplace/bin/ghc-stage1.exe" -optc-fno-stack-protector -optc-Wall 
> -optc-Werror -optc-Wall -optc-Wextra -optc-Wstrict-prototypes 
> -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline 
> -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn 
> -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes 
> -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header 
> -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build 
> -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common 
> -optc-Irts/dist/build/./autogen -optc-Wno-error=inline -optc-O2 
> -optc-fomit-frame-pointer -optc-g -optc-fno-omit-frame-pointer -optc-g 
> -optc-O0 -optc-DRtsWay=\"rts_debug\" -optc-DWINVER=0x06000100 -static 
> -optc-DDEBUG -ticky -DTICKY_TICKY  -O0 -H64m -Wall 
> -fllvm-fill-undef-with-garbage-Werror -Iincludes -Iincludes/dist 
> -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header 
> -Irts -Irts/dist/build -DCOMPILING_RTS -this-unit-id rts -dcmm-lint  -i 
> -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen 
> -Irts/dist/build/./autogen   -O2 -O0
> -Wnoncanonical-monad-instances  -c rts/RaiseAsync.c -o 
> rts/dist/build/RaiseAsync.debug_o
> 
> 
> 
> rts\Linker.c:1483:28: error:
> 
>  error: format '%lx' expects argument of type 'long unsigned int', but 
> argument 3 has type 'long long unsigned int' [-Werror=format=]
> 
>  debugBelch(", section %d, offset %lx\n", i,
> 
> ^
> 
> 
> 
> In file included from rts\Linker.c:13:0: error:
> 
> rts\Linker.c: In function 'ocTryLoad':
> 
> 
> 
> rts\Linker.c:2563:55: error:
> 
>  error: pointer type mismatch in conditional expression [-Werror]
> 
>  oc->archiveMemberName : oc->fileName));
> 
>^
> 
> 
> 
> includes\Rts.h:300:53: error:
> 
>  note: in definition of macro 'IF_DEBUG'
> 
>  #define IF_DEBUG(c,s)  if (RtsFlags.DebugFlags.c) { s; }
> 
>  ^
> 
> 
> 
> rts\Linker.c:2561:33: error:
> 
>  error: format '%ls' expects argument of type 'wchar_t *', but argument 2 
> has type 'void *' [-Werror=format=]
> 
>  IF_DEBUG(linker, debugBelch("Resolving %" PATH_FMT "\n",
> 
>  ^
> 
> 
> 
> includes\Rts.h:300:53: error:
> 
>  note: in definition of macro 'IF_DEBUG'
> 
>  #define IF_DEBUG(c,s)  if (RtsFlags.DebugFlags.c) { s; }
> 
>  ^
> 
> cc1.exe: all warnings being treated as errors
> 
> `gcc.exe' failed in phase `C Compiler'. (Exit code: 1)
> 
> rts/ghc.mk:255: recipe for target 'rts/dist/build/Linker.debug_o' failed
> 
> make[1]: *** [rts/dist/build/Linker.debug_o] Error 1
> 
> make[1]: *** Waiting for unfinished jobs
> 
> Makefile:129: recipe for target 'all' failed
> 
> make: *** [all] Error 2
> 
> /cygdrive/c/code/HEAD$
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Linker.c broken

[Simon Peyton Jones via ghc-devs]

Aargh!  Windows is broken /again/.   Some mess-up in Linker.c.
I have not yet tried reverting recent patches.  Might someone fix please?
It’s really helpful to validate on Windows when making RTS changes.
Simon



rts\Linker.c: In function 'ghci_find':



rts\Linker.c:1482:52: error:

 error: pointer type mismatch in conditional expression [-Werror]

  oc->archiveMemberName : oc->fileName);

^



rts\Linker.c:1480:28: error:

 error: format '%ls' expects argument of type 'wchar_t *', but argument 3 
has type 'void *' [-Werror=format=]

 debugBelch("%p is in %" PATH_FMT, addr,

^

"inplace/bin/ghc-stage1.exe" -optc-fno-stack-protector -optc-Wall -optc-Werror 
-optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes 
-optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return 
-optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs 
-optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist 
-optc-Iincludes/dist-derivedconstants/header 
-optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build 
-optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common 
-optc-Irts/dist/build/./autogen -optc-Wno-error=inline -optc-O2 
-optc-fomit-frame-pointer -optc-g -optc-fno-omit-frame-pointer -optc-g -optc-O0 
-optc-DRtsWay=\"rts_debug\" -optc-DWINVER=0x06000100 -static -optc-DDEBUG 
-ticky -DTICKY_TICKY  -O0 -H64m -Wall -fllvm-fill-undef-with-garbage-Werror 
-Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header 
-Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS 
-this-unit-id rts -dcmm-lint  -i -irts -irts/dist/build -Irts/dist/build 
-irts/dist/build/./autogen -Irts/dist/build/./autogen   -O2 -O0
-Wnoncanonical-monad-instances  -c rts/RaiseAsync.c -o 
rts/dist/build/RaiseAsync.debug_o



rts\Linker.c:1483:28: error:

 error: format '%lx' expects argument of type 'long unsigned int', but 
argument 3 has type 'long long unsigned int' [-Werror=format=]

 debugBelch(", section %d, offset %lx\n", i,

^



In file included from rts\Linker.c:13:0: error:

rts\Linker.c: In function 'ocTryLoad':



rts\Linker.c:2563:55: error:

 error: pointer type mismatch in conditional expression [-Werror]

 oc->archiveMemberName : oc->fileName));

   ^



includes\Rts.h:300:53: error:

 note: in definition of macro 'IF_DEBUG'

 #define IF_DEBUG(c,s)  if (RtsFlags.DebugFlags.c) { s; }

 ^



rts\Linker.c:2561:33: error:

 error: format '%ls' expects argument of type 'wchar_t *', but argument 2 
has type 'void *' [-Werror=format=]

 IF_DEBUG(linker, debugBelch("Resolving %" PATH_FMT "\n",

 ^



includes\Rts.h:300:53: error:

 note: in definition of macro 'IF_DEBUG'

 #define IF_DEBUG(c,s)  if (RtsFlags.DebugFlags.c) { s; }

 ^

cc1.exe: all warnings being treated as errors

`gcc.exe' failed in phase `C Compiler'. (Exit code: 1)

rts/ghc.mk:255: recipe for target 'rts/dist/build/Linker.debug_o' failed

make[1]: *** [rts/dist/build/Linker.debug_o] Error 1

make[1]: *** Waiting for unfinished jobs

Makefile:129: recipe for target 'all' failed

make: *** [all] Error 2

/cygdrive/c/code/HEAD$
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs