When do we get loads/stores with volatile semantics (if we get them at all)?
Hello Haskell Friends, Recently I noticed some strange behavior in a program that uses peek/poke to manipulate memory mapped hardware registers, that smells a lot like missing volatile semantics to me. It hadn’t occurred to me that peek/poke might not have volatile semantics (they return an IO, and that IO has to happen, right?), but naturally once they’re lowered all such bets are off. This made me wonder: - Do we have a type like #Addr whose loads/stores have volatile semantics? - Do we have any primops with volatile semantics at all? - Are there any tricks one could use to get volatile semantics out of base’s peek/poke functions? Poking around, I’m afraid the answer to all three of these is “no.” If so, I’d be very interested in contributing volatile load/store primops and working out some way to make them easily available from Foreign.Storable. Does no such thing currently exist or am I missing something? Thanks for all your ongoing efforts, Travis ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: New Windows I/O manager in GHC 8.12
Hello Tamar, Just wanted to send a quick note expressing my gratitude for these improvements! My first professional work in Haskell targeted Windows platforms, and I'm sure there are many others out there who appreciate the tireless work you've done to keep GHC's Windows support chugging along. I've been kicking the tires on your WinIO branch for a few months and this is absolutely a game-changer for Windows support. Thanks! Travis On Fri, Jul 17, 2020 at 8:06 AM Phyx wrote: > > Hi All, > > In case you've missed it, about 150 or so commits were committed to master > yesterday. These commits add WinIO (Windows I/O) to GHC. This is a new I/O > manager that is designed for the native Windows I/O subsystem instead of > relying on the broken posix-ish compatibility layer that MIO used. > > This is one of 3 big patches I have been working on for years now.. > > So before I continue on why WinIO was made I'll add a TL;DR; > > WinIO adds an internal API break compared to previous GHC releases. That is > the internal code was modified to support a completely asynchronous I/O > system. > > What this means is that we have to keep track of the file pointer offset which > previously was done by the C runtime. This is because in async I/O you cannot > assume the offset to be at any given location. > > What does this mean for you? Very little. If you did not use internal GHC I/O > code. > In particular if you haven't used Buffer, BufferIO and RawIO. If you have you > will > to explicitly add support for GHC 8.12+. > > Because FDs are a Unix concept and don't behave as you would expect on > Windows, the > new I/O manager also uses HANDLE instead of FD. This means that any library > that has > used the internal GHC Fd type won't work with WinIO. Luckily the number of > libraries > that have seems quite low. If you can please stick to the external Handle > interface > for I/O functions. > > The boot libraries have been updated, and in particular process *requires* > the version > that is shipped with GHC. Please respect the version bounds here! I will be > writing > a migration guide for those that need to migrate code. The amount of work is > usually > trivial as Base provides shims to do most of the common things you would have > used Fd for. > > Also if I may make a plea to GHC developers.. Do not add non-trivial > implementations > in the external exposed modules (e.g. System.xxx, Data.xxx) but rather add > them to internal > modules (GHC.xxx) and re-export them from the external modules. This allows > us to avoid > import cycles inside the internal modules :) > > -- > > So why WinIO? Over the years a number of hard to fix issues popped up on > Windows, including > proper Unicode console I/O, cooked inputs, ability to cancel I/O requests. > This also allows libraries like Brick to work on Windows without re-inventing > the wheel or have to hide their I/O from the I/O manager. > > In order to attempt to do some of these with MIO layer upon layers of hacks > were added. This means that things sometimes worked.., but when it didn't > was rather unpredictable. Some of the issues were simply unfixable with MIO. > I will be making some posts about how WinIO works (and also archiving them > on the wiki don't worry :)) but for now some highlights: > > WinIO is 3 years of work, First started by Joey Hess, then picked up by > Mikhail Glushenkov before landing at my feet. While the majority has been > rewritten their work did provide a great jumping off point so thanks! Also > thanks to Ben and AndreasK for helping me get it over the line.. As you can > imagine I was exhausted by this point :). > > Some stats: ~8000 new lines and ~1100 removed ones spread over 130+ commits > (sorry this was the smallest we could get it while not losing some historical > context) and with over 153 files changed not counting the changes to boot > libraries. > > It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359, > #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797, > #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189 > (but only on native windows consoles, so no msys shells) and #806 which is 14 > years old! > > WinIO is a dynamic choice, so you can switch between I/O managers using the > RTS flag --io-manager=[native|posix]. > > On non-Windows native is the same as posix. > > The chosen Async interface for this implementation is using Completion Ports. > > The I/O manager uses a new interface added in Windows Vista called > GetQueuedCompletionStatusEx which allows us to service multiple request > interrupts in one go. > > Some highlights: > > * Drops Windows Vista support > Vista is out of extended support as of 2017. The new minimum is Windows 7. > This allows us to use much more efficient OS provided abstractions. > > * Replace Events and Monitor locks with much faster and efficient Conditional > Variables and SlimReaderWriterLocks. > *
Justifying sched_yield() in the RTS
Hello GHC devs, Through the course of reading some recent material posted by Linus Torvalds ^1 ^2, I remembered stumbling upon GHC Trac #3553 ^3 some years ago. Looking past the harsh tone Linus used in his notes, I think he makes some pretty reasonable points about the problems that can be caused by using spinlocks in userspace. Specifically: - A group of threads yielding while waiting for a spinlock are, in effect, simply trying to coax the scheduler into scheduling the thread that is holding the lock. This is much easier for the scheduler to do correctly with a futex or similar, since the scheduler has some context around which tasks are blocking/waking each other up. - Apparently the Linux scheduler (and maybe other schedulers) has some smarts for preserving cache locality while choosing which physical execution units run which tasks, and reordering threads in the run list in an ad-hoc way with sched_yield() interferes with this mechanism. - Using sched_yield() (or other random interference with run lists) can cause disproportionate havoc on NUMA systems, where jostling around the lock-holding thread by burning time slices and yielding is especially costly. All that said, there is perhaps one concrete advantage (aside from absolute performance) to the current spinlock implementation: the only functionality it requires from the host OS is a yield-like operation. A yielding spinlock occupies a comfortable local optimum, achieving decent performance across platforms with minimal exposure to cross-OS API differences. Revisiting #3553, it seems the only reason that a futex was not used in place of a spinlock with sched_yield() was that, despite all of the points against it, the spinlock code empirically performed better. However, these tests were performed years ago and the futex implementation in the Linux kernel has changed quite a bit. I think there is a healthy amount of empirical evidence from the GHC user community that there are problems afoot with the way parallel GC does locking, and indeed I wonder if the bad cases Linus described are playing a role. Advice like "use -qg" or "use -qnM with small M" is often parroted (this Twitter thread ^4 contains both pieces of advice). Furthermore, I would wager that an RTS using futexes for locking rather than spinning plays nicer with other CPU intensive tasks on the same machine. The "scheduler storm" caused by a large number of threads waiting for a spinlock could have a negative impact on neighboring processes, e.g. a large number of HECs spinning certainly don't do any favors for a busy neighboring DB process. I'm curious if others have thoughts on reviving the futex investigation. Perhaps the futexes provided by modern Linux are more competitive with the current spinlock implementation, or perhaps better scaling on machines with high core counts is worth some cost. In the case that futexes remain handily defeated by yielding spinlocks, it's a worthy endeavor to explain precisely _why_ this happens. Other programs have seen performance issues crop up when the kernel makes subtle changes to how sched_yield() works. ^5 1: https://www.realworldtech.com/forum/?threadid=189711=189723 2: https://www.realworldtech.com/forum/?threadid=189711=189752 3: https://gitlab.haskell.org/ghc/ghc/issues/3553 4: https://twitter.com/ndm_haskell/status/1076187051610042368?s=20 5: https://lwn.net/Articles/31462/ ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Tracking Progress of #7353 (New Windows IO Manager)
Hello Haskell friends, Even though I no longer work at a Windows shop, I've been watching https://gitlab.haskell.org/ghc/ghc/issues/7353 with great interest. Last I knew Tamar Christina was heroically hacking away on a new IO manager for Windows. Although my low-level Windows knowledge is likely insufficient for me to contribute significantly, I'd be very interested in testing the new IO manager and helping with debugging in any way I can. How might one go about building a GHC with this new Windows IO manager? Thanks, as always, for all your hard work, Travis Whitaker ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Type Annotations in the Presence of Injective Type Families, Bidirectional Pattern Synonyms, and Data Kinds
Sure, here it is.
https://gitlab.haskell.org/ghc/ghc/issues/16614
On Mon, Apr 29, 2019 at 11:46 PM Matthew Pickering <
matthewtpicker...@gmail.com> wrote:
> Please can you open a bug report anyway? It is easier to discuss it
> there than on the mailing list.
>
> Matt
>
> On Tue, Apr 30, 2019 at 5:25 AM Travis Whitaker
> wrote:
> >
> > Hello GHC Devs,
> >
> > I've found a case in which annotating a bidirectional pattern synonym
> with a type signature causes the typechecker to reject an otherwise
> typeable program. Before filing a bug report I want to check that I'm
> thinking about this correctly. Consider this module:
> >
> > {-# LANGUAGE TypeFamilies
> >, TypeFamilyDependencies
> >, DataKinds
> >, TypeOperators
> >, GADTs
> >, FlexibleContexts
> >, PatternSynonyms
> >#-}
> >
> > module Strange where
> >
> > data Expr a where
> > Tuple :: NTup ts -> Expr (Flatten ts)
> >
> > data NTup (ts :: [*]) where
> > Unit :: NTup '[]
> > Pair :: Expr a -> NTup ts -> NTup (a ': ts)
> >
> > type family Flatten ts = r | r -> ts where
> > Flatten '[] = ()
> > Flatten '[a, b] = (a, b)
> >
> > pattern P a b = Pair a (Pair b Unit)
> >
> > fstExpr :: Expr (a, b) -> (Expr a, Expr b)
> > fstExpr (Tuple (P x y)) = (x, y)
> >
> > The idea is that an NTup '[a, b, c ...] should be isomorphic to an (a,
> b, c, ...), but removes a lot of repetitive code for dealing with Exprs of
> tuples. This module compiles with GHC 8.6.4 and GHC infers the expected
> type for P.
> >
> > P :: Expr a -> Expr a1 -> NTup '[a, a1]
> >
> > However, annotating P with this type causes typechecking fstExpr to fail
> with:
> >
> > Strange.hs:26:17: error:
> > • Could not deduce: ts ~ '[a, b]
> > from the context: (a, b) ~ Flatten ts
> > bound by a pattern with constructor:
> >Tuple :: forall (ts :: [*]). NTup ts -> Expr (Flatten
> ts),
> > in an equation for ‘fstExpr’
> > at Strange.hs:26:10-22
> > ‘ts’ is a rigid type variable bound by
> > a pattern with constructor:
> > Tuple :: forall (ts :: [*]). NTup ts -> Expr (Flatten ts),
> > in an equation for ‘fstExpr’
> > at Strange.hs:26:10-22
> > Expected type: NTup ts
> > Actual type: NTup '[a, b]
> > • In the pattern: P x y
> > In the pattern: Tuple (P x y)
> > In an equation for ‘fstExpr’: fstExpr (Tuple (P x y)) = (x, y)
> > • Relevant bindings include
> > fstExpr :: Expr (a, b) -> (Expr a, Expr b)
> > (bound at Strange.hs:26:1)
> >|
> > 26 | fstExpr (Tuple (P x y)) = (x, y)
> >| ^
> >
> > It seems that providing a type signature causes GHC to forget the
> injectivity of Flatten (i.e (a, b) ~ Flatten ts implies ts ~ '[a, b]). Is
> this a bug, some limitation of pattern synonyms, or am I missing something?
> >
> > Thanks as always,
> >
> > Travis
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Type Annotations in the Presence of Injective Type Families, Bidirectional Pattern Synonyms, and Data Kinds
Hello GHC Devs,
I've found a case in which annotating a bidirectional pattern synonym with
a type signature causes the typechecker to reject an otherwise typeable
program. Before filing a bug report I want to check that I'm thinking about
this correctly. Consider this module:
{-# LANGUAGE TypeFamilies
, TypeFamilyDependencies
, DataKinds
, TypeOperators
, GADTs
, FlexibleContexts
, PatternSynonyms
#-}
module Strange where
data Expr a where
Tuple :: NTup ts -> Expr (Flatten ts)
data NTup (ts :: [*]) where
Unit :: NTup '[]
Pair :: Expr a -> NTup ts -> NTup (a ': ts)
type family Flatten ts = r | r -> ts where
Flatten '[] = ()
Flatten '[a, b] = (a, b)
pattern P a b = Pair a (Pair b Unit)
fstExpr :: Expr (a, b) -> (Expr a, Expr b)
fstExpr (Tuple (P x y)) = (x, y)
The idea is that an NTup '[a, b, c ...] should be isomorphic to an (a, b,
c, ...), but removes a lot of repetitive code for dealing with Exprs of
tuples. This module compiles with GHC 8.6.4 and GHC infers the expected
type for P.
P :: Expr a -> Expr a1 -> NTup '[a, a1]
However, annotating P with this type causes typechecking fstExpr to fail
with:
Strange.hs:26:17: error:
• Could not deduce: ts ~ '[a, b]
from the context: (a, b) ~ Flatten ts
bound by a pattern with constructor:
Tuple :: forall (ts :: [*]). NTup ts -> Expr (Flatten
ts),
in an equation for ‘fstExpr’
at Strange.hs:26:10-22
‘ts’ is a rigid type variable bound by
a pattern with constructor:
Tuple :: forall (ts :: [*]). NTup ts -> Expr (Flatten ts),
in an equation for ‘fstExpr’
at Strange.hs:26:10-22
Expected type: NTup ts
Actual type: NTup '[a, b]
• In the pattern: P x y
In the pattern: Tuple (P x y)
In an equation for ‘fstExpr’: fstExpr (Tuple (P x y)) = (x, y)
• Relevant bindings include
fstExpr :: Expr (a, b) -> (Expr a, Expr b)
(bound at Strange.hs:26:1)
|
26 | fstExpr (Tuple (P x y)) = (x, y)
| ^
It seems that providing a type signature causes GHC to forget the
injectivity of Flatten (i.e (a, b) ~ Flatten ts implies ts ~ '[a, b]). Is
this a bug, some limitation of pattern synonyms, or am I missing something?
Thanks as always,
Travis
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Validation Failures on aarch64
The patch I’m validating is actually to fix those memory barrier issues. It turns out these failures were all due to GHC complaining about the LLVM version on stderr; sorry for rubber ducking the list! On Mon, Apr 8, 2019 at 1:26 PM Phyx wrote: > Just a wild guess, but considering the time it's taken to run through the > testsuite you're running it on a reasonable AArch64 machine. > You may be hitting the weak memory ordering bugs > https://gitlab.haskell.org/ghc/ghc/issues/15449 , though I haven't > followed the ticket very closely... > > On Mon, Apr 8, 2019 at 8:55 PM Travis Whitaker > wrote: > >> Hello GHC devs, >> >> When attempting to validate a patch on aarch64, it seems there are a >> large number of validation failures: >> >> SUMMARY for test run started at Mon Apr 8 07:19:05 2019 UTC >> 0:15:35 spent to go through >> 6890 total tests, which gave rise to >>17169 test cases, of which >>10018 were skipped >> 41 had missing libraries >> 3151 expected passes >> 150 expected failures >> 11 caused framework failures >>0 caused framework warnings >>0 unexpected passes >> 3798 unexpected failures >>0 unexpected stat failures >> >> The failures seem consistent on recent-ish master, specifically the >> neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be >> expected? >> >> Thanks, >> >> Travis Whitaker >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Validation Failures on aarch64
Hello GHC devs, When attempting to validate a patch on aarch64, it seems there are a large number of validation failures: SUMMARY for test run started at Mon Apr 8 07:19:05 2019 UTC 0:15:35 spent to go through 6890 total tests, which gave rise to 17169 test cases, of which 10018 were skipped 41 had missing libraries 3151 expected passes 150 expected failures 11 caused framework failures 0 caused framework warnings 0 unexpected passes 3798 unexpected failures 0 unexpected stat failures The failures seem consistent on recent-ish master, specifically the neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be expected? Thanks, Travis Whitaker ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Cmm Memory Model (Understanding #15449)
Hello GHC Devs, I'm trying to get my head around ticket #15449 ( https://ghc.haskell.org/trac/ghc/ticket/15449). This gist of things is that GHC generates incorrect aarch64 code that causes memory corruption in multithreaded programs run on out-of-order machines. User trommler discovered that similar issues are present on PowerPC, and indeed ARMv7 and PowerPC support the same types of load/store reorderings. The LLVM code emitted by GHC may be incorrect with respect to LLVM's memory model, but this isn't a problem on architectures with minimal reordering like x86. I had initially thought that GHC simply wasn't emitting the appropriate LLVM fences; there's an elephant-gun-approach here ( https://github.com/TravisWhitaker/ghc/commits/ghc843-wip/T15449) that guards each atomic operation with a full barrier. I still believe that GHC is omitting necessary LLVM fences, but this change is insufficient to fix the behavior of the test case (which is simply GHC itself compiling a test package with '-jN', N > 1). It seems there's a long and foggy history of the Cmm memory model. Edward Yang discusses this a bit in his post here ( http://blog.ezyang.com/2014/01/so-you-want-to-add-a-new-concurrency-primitive-to-ghc/) and issues similar to #15449 have plagued GHC in the past, like #12469 ( https://ghc.haskell.org/trac/ghc/ticket/12469). Worryingly, GHC only has MO_WriteBarrier, whereas PowerPC and ARMv7 really need read, write, and full memory barriers. On ARM an instruction memory barrier might be required as well, but I don't know enough about STG/Cmm to say for sure, and it'd likely be LLVM's responsibility to emit that anyway. I'm hoping that someone with more tribal knowledge than I might be able to give me some pointers with regards to the following areas: - Does STG itself have anything like a memory model? My intuition says 'definitely not', but given that STG expressions may contain Cmm operations (via StgCmmPrim), there may be STG-to-STG transformations that need to care about the target machine's memory model. - With respect to Cmm, what reorderings does GHC perform? What are the relevant parts of the compiler to begin studying? - Are the LLVM atomics that GHC emits correct with respect to the LLVM memory model? As it stands now LLVM fences are only emitted for MO_WriteBarrier. Without fences accompanying the atomics, it seems the LLVM compiler could float dependent loads/stores past atomic operations. - Why is MO_WriteBarrier the only provided memory barrier? My hunch is that it's because this is the only sort of barrier required on x86, which only allows loads to be reordered with older stores, but perhaps I'm missing something? Is it plausible that Cmm simply needs additional barrier primitives to target these weaker memory models? Conversely, is there some property of Cmm that let's us get away without read barriers at all? Naturally, if I've got any of this wrong or are otherwise barking up the wrong tree, please let me know. Thanks for all your efforts! Travis Whitaker ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Validating with LLVM
Hello GHC Devs, I'm working on a very tiny patch for GHC. The patch concerns the LLVM code generator, and I'd like to run the validate script. ./validate ignores mk/ build.mk (which is probably correct) and it doesn't seem to be using the LLVM backend. LLVM tools are on the PATH; I'm working from the ghc-8.4 branch so I'm using LLVM 5. Is it possible to validate with the LLVM backend? If not, what's considered a sufficiently thorough test? Apologies if I'm missing something obvious. Thanks! Travis ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Nondeterministic Failure on aarch64 with -jn, n > 1
Thanks so much for the pointers, Ben.
I opened a ticket here https://ghc.haskell.org/trac/ghc/ticket/15449
On Fri, Jul 27, 2018 at 6:51 AM, Ben Gamari wrote:
> Travis Whitaker writes:
>
> > Hello GHC Devs,
> >
> > It seems to me that GHC is rather broken on aarch64, at least since 8.2.1
> > (and at least on the machines I have access to). I first noticed this
> issue
> > with Nixpkgs (https://github.com/NixOS/nixpkgs/issues/40301), so to
> check
> > that this isn't some Nixpkgs idiosyncrasy I went ahead and built my own
> GHC
> > 8.4.3 for aarch64 (there's no binary release at
> > https://www.haskell.org/ghc/download_ghc_8_4_3.html to try, but perhaps
> > I've missed something.
> >
> > It seems the only Nix idiosyncrasy was passing "--ghc-option=-j${cores}"
> to
> > "./Setup.hs configure". The issue is triggered by using '-jn' for any n
> > greater than one when building any non-trivial package, but I've found
> > hscolour1.24.4 reproduces it very reliably (perhaps because there are
> > opportunities for parallelism early in its module dependency graph?). GHC
> > very often (although not always) will fail with one of:
> >
> > - Segmentation fault.
> > - Bus fault
> > - : error:
> > ghc: panic! (the 'impossible' happened)
> > (GHC version 8.4.3 for aarch64-unknown-linux):
> > Binary.UserData: no put_binding_name
> >
> > - ghc: internal error: MUT_VAR_CLEAN object entered!
> > (GHC version 8.4.3 for aarch64_unknown_linux)
> > Please report this as a GHC bug: http://www.haskell.org/ghc/
> reportabug
> > Aborted (core dumped)
> >
> Ugh, that is awful.
>
> > The fix, excruciating as it may be on already slow arm machines, is to
> use
> > '-j1'. This issue seems present on each GHC release since 8.2.1
> (although I
> > haven't tried HEAD yet). I haven't noticed any issues with any other
> > concurrent Haskell programs on aarch64.
> >
> > There are some umbrella bugs for aarch64 in Trac, so I wanted to ask here
> > before filing a ticket. Has anyone else noticed this behavior on aarch64?
> > What's more, are there any tips for using GDB to hunt down
> synchronization
> > issues in GHC?
> >
> Definitely open a new ticket.
>
> The methodology for tracking down issues like this is quite
> case-specific but I do have some general recommendations: On x86-64 I
> use rr [1], which is an invaluable tool. Sadly this isn't an option on
> AArch64 AFAIK. I also have some gdb extensions to take much of the
> monotony away from inspecting GHC's heap and internal data structures
> [2]. I've not used them on AArch64 so there may be a few compatibility
> issues but I suspect they wouldn't be hard to fix.
>
> I know it may be hard in this case but I would at least try to reduce
> the size of the failing program to something that fits in less than a
> few hundred lines. Low-level debugging is hard enough when you can keep
> the program in your head; debugging all of GHC this way is possible but
> much harder. Given that this appears to be threading-specific, I would
> also pay particular attention to the GHC and base's use of barriers and
> atomics. It's possible that we are just missing a barrier somewhere.
>
> Finally, you might quickly try building 8.0 to see whether bisection is
> a possibility. It would be a slow process, given the speed of the
> hardware involved, but ultimately it can be much more time efficient
> once you have it setup since you can replace human debugging time (a
> very finite commodity) with computation.
>
> Good luck and let us know if you get stuck,
>
> - Ben
>
>
> [1] http://rr-project.org/
> [2] https://github.com/bgamari/ghc-utils/tree/master/gdb
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Nondeterministic Failure on aarch64 with -jn, n > 1
Hello GHC Devs,
It seems to me that GHC is rather broken on aarch64, at least since 8.2.1
(and at least on the machines I have access to). I first noticed this issue
with Nixpkgs (https://github.com/NixOS/nixpkgs/issues/40301), so to check
that this isn't some Nixpkgs idiosyncrasy I went ahead and built my own GHC
8.4.3 for aarch64 (there's no binary release at
https://www.haskell.org/ghc/download_ghc_8_4_3.html to try, but perhaps
I've missed something.
It seems the only Nix idiosyncrasy was passing "--ghc-option=-j${cores}" to
"./Setup.hs configure". The issue is triggered by using '-jn' for any n
greater than one when building any non-trivial package, but I've found
hscolour1.24.4 reproduces it very reliably (perhaps because there are
opportunities for parallelism early in its module dependency graph?). GHC
very often (although not always) will fail with one of:
- Segmentation fault.
- Bus fault
- : error:
ghc: panic! (the 'impossible' happened)
(GHC version 8.4.3 for aarch64-unknown-linux):
Binary.UserData: no put_binding_name
- ghc: internal error: MUT_VAR_CLEAN object entered!
(GHC version 8.4.3 for aarch64_unknown_linux)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Aborted (core dumped)
The fix, excruciating as it may be on already slow arm machines, is to use
'-j1'. This issue seems present on each GHC release since 8.2.1 (although I
haven't tried HEAD yet). I haven't noticed any issues with any other
concurrent Haskell programs on aarch64.
There are some umbrella bugs for aarch64 in Trac, so I wanted to ask here
before filing a ticket. Has anyone else noticed this behavior on aarch64?
What's more, are there any tips for using GDB to hunt down synchronization
issues in GHC?
Thanks for all your hard work!
Travis
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Use of -dead_strip_dylibs by default makes the -framework flag appear broken.
Hello Haskell Friends,
GHC always passes
<https://github.com/ghc/ghc/blob/master/compiler/main/DriverPipeline.hs#L1888>
-dead_strip_dylibs to the linker on macOS. This means that Haskell programs
that use Objective-C-style dynamic binding (via objc_getClass or similar)
won't actually be able to find the Objective-C methods they need at
runtime. Here's an example illustrating the problem: Consider this small
example program:
#include
extern void *objc_getClass(char *n);
void test_get_class(char *n)
{
void *cp = objc_getClass(n);
if(cp == NULL)
{
printf("Didn't find class %s\n", n);
}
else
{
printf("Found class %s\n", n);
}
}
int main(int argc, char *argv[])
{
test_get_class(argv[1]);
return 0;
}
Building like this:
clang -o hasclass main.c -lobjc -L/usr/lib -framework Foundation -F
/System/Library/Frameworks/
Yields an executable that works like this:
$ ./hasclass NSObject
Found class NSObject
$ ./hasclass NSString
Found class NSString
$ ./hasclass NSDate
Found class NSDate
otool shows that we're linked against Foundation properly:
hasclass:
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current
version 228.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
(compatibility version 300.0.0, current version 1452.23.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.50.4)
Now consider this equivalent Haskell example:
module Main where
import Foreign.C.String
import Foreign.Ptr
import System.Environment
foreign import ccall objc_getClass :: CString -> IO (Ptr a)
testGetClass :: String -> IO ()
testGetClass n = withCString n $ \cn -> do
cp <- objc_getClass cn
let m | cp == nullPtr = "Didn't find class " ++ n
| otherwise = "Found class " ++ n
putStrLn m
main :: IO ()
main = getArgs >>= (testGetClass . head)
Building like this:
ghc -o hasclass Main.hs -lobjc -L/usr/lib -framework foundation
-framework-path /System/Library/Frameworks/
Yields an executable that works like this:
$ ./hasclass NSObject
Found class NSObject
$ ./hasclass NSString
Didn't find class NSString
$ ./hasclass NSDate
Didn't find class NSDate
otool shows that our load commands for Foundation are missing:
hasclass:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.50.4)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current
version 228.0.0)
/nix/store/7jdxjpy1p5ynl9qrr3ymx01973a1abf6-gmp-6.1.2/lib/libgmp.10.dylib
(compatibility version 14.0.0, current version 14.2.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current
version 7.0.0)
Interestingly, the testGetClass function will work just fine in GHCi, since
it always loads all of the shared objects and frameworks it's asked to. As
far as I can tell the only way to get a hasclass executable with the
correct behavior is to do the final linking manually with Clang.
My understanding is that this behavior was introduced to work around symbol
count limitations introduced in macOS Sierra. It would be nice to wrap the
frameworks passed to the linker in some flags that spares them from
-dead_strip_dylibs. I haven't found such a feature in my limited digging
around, but perhaps someone who knows more about systems programming on
macOS will have an idea. Statically linking against the system frameworks
would be a workable stopgap solution, but I have yet to find an easy way to
do that.
I'm curious what others' thoughts are on this issue; it's very difficult to
call Objective-C methods from Haskell (without generating Objective-C)
without a fix for this.
Regards,
Travis Whitaker
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Availability of Coercible Instances at TH Runtime
A workaround that's suitable for what I'm after is defining a class like this for some fixed T. instance (Coercible a T) => C a I'm curious what it'd take to add a qReifyCoercible method; does the renamer know anything about Coercible? Thanks, Travis On Fri, May 4, 2018 at 11:01 AM, Richard Eisenberg <r...@cs.brynmawr.edu> wrote: > I don't think there's an easy way to do this. We could imagine extending > Quasi to have a method to check for coercibility, but I don't think there's > a way to do this in the current TH API. Sorry! > > Richard > > On May 4, 2018, at 3:44 AM, Travis Whitaker <pi.boy.tra...@gmail.com> > wrote: > > Given that Coercible instances are Deeply Magical, perhaps I'm being a bit > naive here, but I recently tried to write a TH function that can check if > one type is a newtype of another (or a newtype of a newtype of another, > etc). > > coercibleToFrom :: Type -> Type -> Q Bool > coercibleToFrom tx ty = (&&) <$> isInstance ''Coercible [tx, ty] > <*> isInstance ''Coercible [ty, tx] > > If this worked as I'd hoped, I'm almost certain checking reflexively is > redundant. However, I can't seem to get reifyInstances to ever return an > InstanceDec for a Coercible instance. Given that these instances are > generated on the fly by the typechecker, there's no way to make them > available at TH runtime, correct? And, given that, is there an easy way to > find out with TH whether not I'll be able to use coerce without taking all > the Decs apart to hunt for NewtypeD? > > Travis > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Fwd: Availability of Coercible Instances at TH Runtime
Given that Coercible instances are Deeply Magical, perhaps I'm being a bit naive here, but I recently tried to write a TH function that can check if one type is a newtype of another (or a newtype of a newtype of another, etc). coercibleToFrom :: Type -> Type -> Q Bool coercibleToFrom tx ty = (&&) <$> isInstance ''Coercible [tx, ty] <*> isInstance ''Coercible [ty, tx] If this worked as I'd hoped, I'm almost certain checking reflexively is redundant. However, I can't seem to get reifyInstances to ever return an InstanceDec for a Coercible instance. Given that these instances are generated on the fly by the typechecker, there's no way to make them available at TH runtime, correct? And, given that, is there an easy way to find out with TH whether not I'll be able to use coerce without taking all the Decs apart to hunt for NewtypeD? Travis ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Linux (ELF) Support for "ghc -static -shared"
Suppose I have some module Foo with foreign exports. On some platforms I can do something like: ghc -static -shared Foo.o ... The resulting shared library would have the base libraries and the RTS statically linked in. From what I understand this is possible on BSDs because generating PIC is the default there (for making PIEs I'd imagine), and possible on Windows because the dynamic loading process involves some technique that doesn't require PIC. On Linux (at least x86_64) this doesn't work by default since libHSbase, libHSrts et al. are not built with -fPIC unless one specifically asks for it when building GHC. As far as I know this is the only way to get -static -shared to work on this platform. While the use cases for such stand-alone shared libraries might be small niches, I was curious whether or not there was any discussion about potential strategies for making it easier to build them for Linux. At the very least, perhaps a single switch for the configure script or build.mk to make it easier to build GHC+libs with -fPIC on Linux. Another step up might be providing *_PIC.a objects for the base libraries, so that the non-PIC objects are still available for the majority of cases in which PIC is not required. Thanks for your time, Travis Whitaker ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
