Re: DWARF support

2021-11-19 Thread Artem Pelenitsyn
Thanks everyone! A blog post would be awesome, indeed.

Ben, I didn't have anything particular in mind, just curious about possibly
more ergonomic alternatives to profiling builds for getting this sort of
information. (Motivation similar to Richard's, I think.)

On Fri, Nov 19, 2021, 1:57 AM Sven Tennie  wrote:

> Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari <
> b...@smart-cactus.org>:
>
>> Artem Pelenitsyn  writes:
>>
>> > Another question would be where do I read about Haskell-native stack
>> > unwinder. The issue and MR Ben referenced have descriptions, but the MR
>> > didn't touch anything inside `docs` which is a bit scary. Are there any
>> > good recourses to dive into it besides the source code in the MR?
>> >
>> Indeed the user's guide documentation surrounding info table provenance
>> should be updated to note this new capability.
>>
>
> Hey Ben, Hey Artem, Hey all,
>
> Thanks for bringing this up! I'll add some documentation to the user's
> guide and will likely write a blog post covering the "material" from notes
> in a more approachable way.
>
> Two details to note regarding the IPE based backtrace mechanism:
>
> - It's based on return frames on the STG stack. So, it doesn't work for
> trivial programs. A return frame is produced when the scrutinee of a case
> expression is evaluated.
> The test [1] shows a working example.
>
> - To get backtraces containing references in libraries, those have to be
> compiled with `-finfo-table-map`, too.
>
> Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to
> Ben and Matthew that were always reachable and always very helpful during
> the whole development process of this feature!
>
> Best regards,
>
> Sven
>
> [1] -
> https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d818/testsuite/tests/rts/decodeMyStack.hs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Sven Tennie
Am Fr., 19. Nov. 2021 um 01:09 Uhr schrieb Ben Gamari :

> Artem Pelenitsyn  writes:
>
> > Another question would be where do I read about Haskell-native stack
> > unwinder. The issue and MR Ben referenced have descriptions, but the MR
> > didn't touch anything inside `docs` which is a bit scary. Are there any
> > good recourses to dive into it besides the source code in the MR?
> >
> Indeed the user's guide documentation surrounding info table provenance
> should be updated to note this new capability.
>

Hey Ben, Hey Artem, Hey all,

Thanks for bringing this up! I'll add some documentation to the user's
guide and will likely write a blog post covering the "material" from notes
in a more approachable way.

Two details to note regarding the IPE based backtrace mechanism:

- It's based on return frames on the STG stack. So, it doesn't work for
trivial programs. A return frame is produced when the scrutinee of a case
expression is evaluated.
The test [1] shows a working example.

- To get backtraces containing references in libraries, those have to be
compiled with `-finfo-table-map`, too.

Now that I've got the attention of many GHC-Devs: A big Thank-You! goes to
Ben and Matthew that were always reachable and always very helpful during
the whole development process of this feature!

Best regards,

Sven

[1] -
https://gitlab.haskell.org/ghc/ghc/-/blob/aed98ddaf72cc38fb570d8415cac5de9d818/testsuite/tests/rts/decodeMyStack.hs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Ben Gamari
Artem Pelenitsyn  writes:

> Another question would be where do I read about Haskell-native stack
> unwinder. The issue and MR Ben referenced have descriptions, but the MR
> didn't touch anything inside `docs` which is a bit scary. Are there any
> good recourses to dive into it besides the source code in the MR?
>
Indeed the user's guide documentation surrounding info table provenance
should be updated to note this new capability.

There is a long Note in compiler/GHC/Driver/GenerateCgIPEStub.hs which
describes the implementation. As far as the interface is concerned, it's
quite straightforward:

* One can call GHC.Stack.CloneStack.cloneMyStack, which returns a
  "frozen" copy of the calling context's stack
* One can then call GHC.Stack.CloneStack.decode to decode the stack into
  a list of frames.

Did you have any particular questions?

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Artem Pelenitsyn
Another question would be where do I read about Haskell-native stack
unwinder. The issue and MR Ben referenced have descriptions, but the MR
didn't touch anything inside `docs` which is a bit scary. Are there any
good recourses to dive into it besides the source code in the MR?

--
Best, Artem

On Thu, Nov 18, 2021, 11:31 AM Chris Smith  wrote:

> Just to satisfy my curiosity here, when talking about backtraces here, are
> you talking about a lexical call stack, or an execution stack?
>
> On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg 
> wrote:
>
>>
>>
>> On Nov 18, 2021, at 10:29 AM, Ben Gamari  wrote:
>>
>> At this point, for backtrace support I would rather put my money is on a
>> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only
>> is it more portable but it is also more robust (whereas with DWARF any
>> single object lacking debug information would break unwinding), and is
>> significantly less costly since we know much more about the structure of
>> our stack than a DWARF unwinder would.
>>
>>
>> Interesting -- this is helpful to know. I had heard about DWARF support
>> for some years and thought that it would deliver stack traces. Now I will
>> look for other sources. All good -- I understand how this is hard! -- and
>> nice to know about.
>>
>> Thanks for the writeup, Ben.
>>
>> Richard
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Chris Smith
Just to satisfy my curiosity here, when talking about backtraces here, are
you talking about a lexical call stack, or an execution stack?

On Thu, Nov 18, 2021 at 11:24 AM Richard Eisenberg 
wrote:

>
>
> On Nov 18, 2021, at 10:29 AM, Ben Gamari  wrote:
>
> At this point, for backtrace support I would rather put my money is on a
> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only
> is it more portable but it is also more robust (whereas with DWARF any
> single object lacking debug information would break unwinding), and is
> significantly less costly since we know much more about the structure of
> our stack than a DWARF unwinder would.
>
>
> Interesting -- this is helpful to know. I had heard about DWARF support
> for some years and thought that it would deliver stack traces. Now I will
> look for other sources. All good -- I understand how this is hard! -- and
> nice to know about.
>
> Thanks for the writeup, Ben.
>
> Richard
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Richard Eisenberg


> On Nov 18, 2021, at 10:29 AM, Ben Gamari  wrote:
> 
> At this point, for backtrace support I would rather put my money is on a
> native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only
> is it more portable but it is also more robust (whereas with DWARF any
> single object lacking debug information would break unwinding), and is
> significantly less costly since we know much more about the structure of
> our stack than a DWARF unwinder would.

Interesting -- this is helpful to know. I had heard about DWARF support for 
some years and thought that it would deliver stack traces. Now I will look for 
other sources. All good -- I understand how this is hard! -- and nice to know 
about.

Thanks for the writeup, Ben.

Richard___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Ben Gamari
Richard Eisenberg  writes:

> Thanks for this!
>
>> On Nov 17, 2021, at 7:27 AM, Moritz Angermann  
>> wrote:
>> 
>> For Linux and most BSDs, we have settled on the Executable and Linking 
>> Format (ELF) as the container format for
>> your machine code.  And you might see where the inspiration for DWARF might 
>> come from.
>
> This suggests to me that DWARF is the canonical format for debugging 
> information on Linux and most BSDs. Is that statement correct? If so, how is 
> that different to "platform-native"? Actually, the precise wording doesn't 
> matter: I think I'm just requesting for a more direct relationship between 
> "DWARF" and "compatibility with all the debugging and profiling tools you use 
> for other languages".
>
>> 
>> For macOS, we have mach object (mach-o) as the container format. Its 
>> distinctly different to ELF, and also the
>> reason why Linux/BSD and macOS are sometimes substantially different, wrt to 
>> executable packaging and linking.
>
> OK. So there is no macOS support here. That's fine -- I'm just trying to 
> understand the status quo.
>
>> 
>> For windows we have Portable Executable (PE) as the container format.
>
> This implies that the DWARF work is (unsurprisingly) completely inapplicable 
> for Windows.
>
>> Depending on how familiar you are with development on macOS, you might know 
>> the notion of dSYM folders,
>> where macOS usually separate the application binary into the binary, and 
>> then stores the (d)ebug (SYM)bols in
>> a separate folder. Those are iirc DWARF objects in the end.
>
> This suggests to me that the DWARF work is applicable to the macOS use case, 
> but much more work still needs to be done. OK.
>
> Looking for more information, I checked the manual. And I found this:
> https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html
> It's really helpful! And it suggests that I actually *can* do this on
> macOS. Perhaps that video will really happen after all.
>
I should clarify here that there are really two pieces to "debug
information support":

 * the code generation logic responsible for producing the DWARF
   metadata

 * the RTS support for using that information to unwind the stack at
   runtime

That section really describes the former. As noted in my earlier email
and in #20702, the library used by the RTS to implement unwinding only
targets ELF platforms.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-18 Thread Ben Gamari
Richard Eisenberg  writes:

> Hi devs,
>
> I was intrigued by Bodigrim's comment about HasCallStack in base
> (https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580)
> that there are other alternatives, such as DWARF. Over the years, I
> had tuned out every time I saw the word DWARF: it was (and is!) an
> unknown acronym and seems like a low-level detail. But Bodigrim's
> comment made me want to re-think this stance.
>
> I found Ben's series of blog posts on DWARF, starting with
> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very
> helpful! In particular, they taught me that DWARF = platform-native
> debugging metadata. Is that translation accurate? If so, perhaps we
> should use both names: if I see that GHC x.y.z has DWARF support, I
> quickly scroll to the next bullet. If I see that GHC x.y.z has support
> for platform-native debugging metadata and is now compatible with e.g.
> gdb, I'm interested.
>
> Going further, I have a key question for my use case: is this support
> available on Mac? The first post in the series describes support for
> "Linux and several BSDs" and the last post says that "Windows PDB
> support" is future work. (Is "PDB" platform-native debugging metadata
> for Windows? I don't know.) But I don't see any mention of Mac. What's
> the status here?
>
The problem is that DWARF is only one format for debugging information.
Windows uses PDB, which is practically undocumented and consequently
essentially unsupportable, AFAICT. I believe some other platforms still
use the STABS format, which I do not believe we will ever support.

Sadly, Darwin is, as usual, a bit of a tricky case. The platform seems
to use a variant of the typical "split-debug" packaging configuration
for debug information. Apple calls this separate debug information
`dsym` files. In typical Apple fasion, the documentation surrounding
this mechanism is very poor. It appears that the user or packager must
generate these files manually using dsymutils but it's quite unclear how
a consuming library is to find the dsym for a particular object.
To make matters worse, the library that we use for unwinding, libdw, has
absolutely no support for it.

Ultimately, my goals for working on DWARF were:

 1. To ensure that we had *some* baseline symbol and line information
support on platforms for low-level debugging. I have found even
basic line information invaluable when looking at code generation
issues and thankfully gdb can always use inline DWARF information,
even on platforms like on Windows where it isn't quite "native"

 2. Providing an interface for users to unwind the stack from Haskell on
the "easy" platforms (e.g. the ELF-based BSDs and Linux). I was
hoping someone would express interest in picking up Darwin and
Windows, at which point we could start to push this as a universal
backtrace mechanism, but sadly this hasn't yet happened.

 3. Provide a basis for statistic profiling support. This is sadly
complicated by the stack pointer issues that Andreas and I note in
our respective blog posts [1,2]. Sadly, overcoming this without
breaking potential users is harder than we thought due to
limitations in LLVM.

At this point, for backtrace support I would rather put my money is on a
native Haskell stack unwinder (such as Sven Tennie's work [3,4]). Not only
is it more portable but it is also more robust (whereas with DWARF any
single object lacking debug information would break unwinding), and is
significantly less costly since we know much more about the structure of
our stack than a DWARF unwinder would.

Cheers,

- Ben


[1] https://well-typed.com/blog/2020/04/dwarf-4/
[2] https://well-typed.com/blog/2021/07/ghc-sp-profiling/
[3] https://gitlab.haskell.org/ghc/ghc/-/issues/18163
[4] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5456


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-17 Thread Andreas Klebinger


Am 17/11/2021 um 17:08 schrieb Richard Eisenberg:

For windows we have Portable Executable (PE) as the container format.

This implies that the DWARF work is (unsurprisingly) completely inapplicable 
for Windows.

It's not quite as simple. Dwarf info can be embedded into windows
executables/libraries. Tools that understand dwarf can read that
information and use it.
However many tools on windows don't understand dwarf debugging
information since windows has it's own debugging format.

So it's not quite as helpful as on linux where everyone agreed to use
dwarf. But it still can be used for some things.

E.g. there are debuggers on windows (lldbg, gdb) that can read and use
this information.

Further we could in theory start emitting the same information in the
windows native format. In that case we could re-use much of the
work that went into GHC to allows us to collect debugging information.
Putting into the right format comes fairly late in the pipeline and
all steps up to that could be shared.

Cheers

Andreas

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-17 Thread Richard Eisenberg
Thanks for this!

> On Nov 17, 2021, at 7:27 AM, Moritz Angermann  
> wrote:
> 
> For Linux and most BSDs, we have settled on the Executable and Linking Format 
> (ELF) as the container format for
> your machine code.  And you might see where the inspiration for DWARF might 
> come from.

This suggests to me that DWARF is the canonical format for debugging 
information on Linux and most BSDs. Is that statement correct? If so, how is 
that different to "platform-native"? Actually, the precise wording doesn't 
matter: I think I'm just requesting for a more direct relationship between 
"DWARF" and "compatibility with all the debugging and profiling tools you use 
for other languages".

> 
> For macOS, we have mach object (mach-o) as the container format. Its 
> distinctly different to ELF, and also the
> reason why Linux/BSD and macOS are sometimes substantially different, wrt to 
> executable packaging and linking.

OK. So there is no macOS support here. That's fine -- I'm just trying to 
understand the status quo.

> 
> For windows we have Portable Executable (PE) as the container format.

This implies that the DWARF work is (unsurprisingly) completely inapplicable 
for Windows.

> Depending on how familiar you are with development on macOS, you might know 
> the notion of dSYM folders,
> where macOS usually separate the application binary into the binary, and then 
> stores the (d)ebug (SYM)bols in
> a separate folder. Those are iirc DWARF objects in the end.

This suggests to me that the DWARF work is applicable to the macOS use case, 
but much more work still needs to be done. OK.

Looking for more information, I checked the manual. And I found this: 
https://downloads.haskell.org/ghc/latest/docs/html/users_guide/debug-info.html  
It's really helpful! And it suggests that I actually *can* do this on macOS. 
Perhaps that video will really happen after all.

Forgive the noise -- I should have checked the manual first. :)

Richard
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-17 Thread Moritz Angermann
Thanks Carter!

Yes I completely forgot about the unwinding librarys.

Sorry. My bad!

Best,
 Moritz

On Wed 17. Nov 2021 at 21:08, Carter Schonwald 
wrote:

> My understanding is that the platform specific part of ghc dwarf support
> atm is the stack walking to generate dwarf data in stack traces. This is
> because the dwarf stack walking Libs that are relatively mature are mostly
> centered around elf?
>
> It should still be possible with some work to use perf and gdb style
> tools, though the complications are
>
> a) you have to make sure all the Libs are built with dwarf
>
> b) there’s some complications around loading / placing the dwarf files
> adjacent to the object code files on Darwin (at least last time I checked
> which was years ago following the wiki entry johan tibbel wrote up I
> think?)
>
> C) scheduler yields make gdb stepping through a program a tad more
> annoying, I think the “setting the yield timer to zero” is the work around
>
> D) the “source” you step through is essentially the c— z-encoded code? So
> you still need to do some puzzling out of stuff
>
>
> On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann <
> moritz.angerm...@gmail.com> wrote:
>
>> Hi Richard,
>>
>> I’m not sure using platform native AND  the term DWARF would help rather
>> than add to confusion.  Let me still try to
>> help a bit with context here.
>>
>> For Linux and most BSDs, we have settled on the Executable and Linking
>> Format (ELF) as the container format for
>> your machine code.  And you might see where the inspiration for DWARF
>> might come from.
>>
>> For macOS, we have mach object (mach-o) as the container format. Its
>> distinctly different to ELF, and also the
>> reason why Linux/BSD and macOS are sometimes substantially different, wrt
>> to executable packaging and linking.
>>
>> For windows we have Portable Executable (PE) as the container format.
>>
>> My recollection is that we implemented DWARF in the NCG only for ELF.
>> I've always wanted to scratch an itch
>> and try to make it work for mach-o as well, but never got around to it
>> (yet?).  The NCGs have flags that specify
>> if we want to emit debug info or not.  I believe most codegens except for
>> x86_64/elf ignore that flag.
>>
>> This is a non-trivial engineering effort to get done properly, I believe.
>> And we all spend time on many other things.
>>
>> Depending on how familiar you are with development on macOS, you might
>> know the notion of dSYM folders,
>> where macOS usually separate the application binary into the binary, and
>> then stores the (d)ebug (SYM)bols in
>> a separate folder. Those are iirc DWARF objects in the end.
>>
>> Hope this helps a bit; my recollection might be a bit rusty.
>>
>> Best,
>>  Moritz
>>
>>
>>
>> On Wed 17. Nov 2021 at 20:02, Richard Eisenberg 
>> wrote:
>>
>>> Hi devs,
>>>
>>> I was intrigued by Bodigrim's comment about HasCallStack in base (
>>> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580)
>>> that there are other alternatives, such as DWARF. Over the years, I had
>>> tuned out every time I saw the word DWARF: it was (and is!) an unknown
>>> acronym and seems like a low-level detail. But Bodigrim's comment made me
>>> want to re-think this stance.
>>>
>>> I found Ben's series of blog posts on DWARF, starting with
>>> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very
>>> helpful! In particular, they taught me that DWARF = platform-native
>>> debugging metadata. Is that translation accurate? If so, perhaps we should
>>> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll
>>> to the next bullet. If I see that GHC x.y.z has support for platform-native
>>> debugging metadata and is now compatible with e.g. gdb, I'm interested.
>>>
>>> Going further, I have a key question for my use case: is this support
>>> available on Mac? The first post in the series describes support for "Linux
>>> and several BSDs" and the last post says that "Windows PDB support" is
>>> future work. (Is "PDB" platform-native debugging metadata for Windows? I
>>> don't know.) But I don't see any mention of Mac. What's the status here?
>>>
>>> It would be very cool if this conversation ends with me making a video
>>> on how a few simple GHC flags can allow us to, say, get a stack trace on a
>>> pattern-match failure in a Haskell program.
>>>
>>> Thanks!
>>> Richard
>>>
>>> ___
>>> ghc-devs mailing list
>>> ghc-devs@haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-17 Thread Carter Schonwald
My understanding is that the platform specific part of ghc dwarf support
atm is the stack walking to generate dwarf data in stack traces. This is
because the dwarf stack walking Libs that are relatively mature are mostly
centered around elf?

It should still be possible with some work to use perf and gdb style tools,
though the complications are

a) you have to make sure all the Libs are built with dwarf

b) there’s some complications around loading / placing the dwarf files
adjacent to the object code files on Darwin (at least last time I checked
which was years ago following the wiki entry johan tibbel wrote up I
think?)

C) scheduler yields make gdb stepping through a program a tad more
annoying, I think the “setting the yield timer to zero” is the work around

D) the “source” you step through is essentially the c— z-encoded code? So
you still need to do some puzzling out of stuff


On Wed, Nov 17, 2021 at 7:28 AM Moritz Angermann 
wrote:

> Hi Richard,
>
> I’m not sure using platform native AND  the term DWARF would help rather
> than add to confusion.  Let me still try to
> help a bit with context here.
>
> For Linux and most BSDs, we have settled on the Executable and Linking
> Format (ELF) as the container format for
> your machine code.  And you might see where the inspiration for DWARF
> might come from.
>
> For macOS, we have mach object (mach-o) as the container format. Its
> distinctly different to ELF, and also the
> reason why Linux/BSD and macOS are sometimes substantially different, wrt
> to executable packaging and linking.
>
> For windows we have Portable Executable (PE) as the container format.
>
> My recollection is that we implemented DWARF in the NCG only for ELF.
> I've always wanted to scratch an itch
> and try to make it work for mach-o as well, but never got around to it
> (yet?).  The NCGs have flags that specify
> if we want to emit debug info or not.  I believe most codegens except for
> x86_64/elf ignore that flag.
>
> This is a non-trivial engineering effort to get done properly, I believe.
> And we all spend time on many other things.
>
> Depending on how familiar you are with development on macOS, you might
> know the notion of dSYM folders,
> where macOS usually separate the application binary into the binary, and
> then stores the (d)ebug (SYM)bols in
> a separate folder. Those are iirc DWARF objects in the end.
>
> Hope this helps a bit; my recollection might be a bit rusty.
>
> Best,
>  Moritz
>
>
>
> On Wed 17. Nov 2021 at 20:02, Richard Eisenberg 
> wrote:
>
>> Hi devs,
>>
>> I was intrigued by Bodigrim's comment about HasCallStack in base (
>> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580)
>> that there are other alternatives, such as DWARF. Over the years, I had
>> tuned out every time I saw the word DWARF: it was (and is!) an unknown
>> acronym and seems like a low-level detail. But Bodigrim's comment made me
>> want to re-think this stance.
>>
>> I found Ben's series of blog posts on DWARF, starting with
>> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very
>> helpful! In particular, they taught me that DWARF = platform-native
>> debugging metadata. Is that translation accurate? If so, perhaps we should
>> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll
>> to the next bullet. If I see that GHC x.y.z has support for platform-native
>> debugging metadata and is now compatible with e.g. gdb, I'm interested.
>>
>> Going further, I have a key question for my use case: is this support
>> available on Mac? The first post in the series describes support for "Linux
>> and several BSDs" and the last post says that "Windows PDB support" is
>> future work. (Is "PDB" platform-native debugging metadata for Windows? I
>> don't know.) But I don't see any mention of Mac. What's the status here?
>>
>> It would be very cool if this conversation ends with me making a video on
>> how a few simple GHC flags can allow us to, say, get a stack trace on a
>> pattern-match failure in a Haskell program.
>>
>> Thanks!
>> Richard
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


Re: DWARF support

2021-11-17 Thread Moritz Angermann
Hi Richard,

I’m not sure using platform native AND  the term DWARF would help rather
than add to confusion.  Let me still try to
help a bit with context here.

For Linux and most BSDs, we have settled on the Executable and Linking
Format (ELF) as the container format for
your machine code.  And you might see where the inspiration for DWARF
might come from.

For macOS, we have mach object (mach-o) as the container format. Its
distinctly different to ELF, and also the
reason why Linux/BSD and macOS are sometimes substantially different, wrt
to executable packaging and linking.

For windows we have Portable Executable (PE) as the container format.

My recollection is that we implemented DWARF in the NCG only for ELF.  I've
always wanted to scratch an itch
and try to make it work for mach-o as well, but never got around to it
(yet?).  The NCGs have flags that specify
if we want to emit debug info or not.  I believe most codegens except for
x86_64/elf ignore that flag.

This is a non-trivial engineering effort to get done properly, I believe.
And we all spend time on many other things.

Depending on how familiar you are with development on macOS, you might know
the notion of dSYM folders,
where macOS usually separate the application binary into the binary, and
then stores the (d)ebug (SYM)bols in
a separate folder. Those are iirc DWARF objects in the end.

Hope this helps a bit; my recollection might be a bit rusty.

Best,
 Moritz



On Wed 17. Nov 2021 at 20:02, Richard Eisenberg  wrote:

> Hi devs,
>
> I was intrigued by Bodigrim's comment about HasCallStack in base (
> https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580)
> that there are other alternatives, such as DWARF. Over the years, I had
> tuned out every time I saw the word DWARF: it was (and is!) an unknown
> acronym and seems like a low-level detail. But Bodigrim's comment made me
> want to re-think this stance.
>
> I found Ben's series of blog posts on DWARF, starting with
> https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very
> helpful! In particular, they taught me that DWARF = platform-native
> debugging metadata. Is that translation accurate? If so, perhaps we should
> use both names: if I see that GHC x.y.z has DWARF support, I quickly scroll
> to the next bullet. If I see that GHC x.y.z has support for platform-native
> debugging metadata and is now compatible with e.g. gdb, I'm interested.
>
> Going further, I have a key question for my use case: is this support
> available on Mac? The first post in the series describes support for "Linux
> and several BSDs" and the last post says that "Windows PDB support" is
> future work. (Is "PDB" platform-native debugging metadata for Windows? I
> don't know.) But I don't see any mention of Mac. What's the status here?
>
> It would be very cool if this conversation ends with me making a video on
> how a few simple GHC flags can allow us to, say, get a stack trace on a
> pattern-match failure in a Haskell program.
>
> Thanks!
> Richard
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


DWARF support

2021-11-17 Thread Richard Eisenberg
Hi devs,

I was intrigued by Bodigrim's comment about HasCallStack in base 
(https://github.com/haskell/core-libraries-committee/issues/5#issuecomment-970942580)
 that there are other alternatives, such as DWARF. Over the years, I had tuned 
out every time I saw the word DWARF: it was (and is!) an unknown acronym and 
seems like a low-level detail. But Bodigrim's comment made me want to re-think 
this stance.

I found Ben's series of blog posts on DWARF, starting with 
https://www.haskell.org/ghc/blog/20200403-dwarf-1.html. These are very helpful! 
In particular, they taught me that DWARF = platform-native debugging metadata. 
Is that translation accurate? If so, perhaps we should use both names: if I see 
that GHC x.y.z has DWARF support, I quickly scroll to the next bullet. If I see 
that GHC x.y.z has support for platform-native debugging metadata and is now 
compatible with e.g. gdb, I'm interested.

Going further, I have a key question for my use case: is this support available 
on Mac? The first post in the series describes support for "Linux and several 
BSDs" and the last post says that "Windows PDB support" is future work. (Is 
"PDB" platform-native debugging metadata for Windows? I don't know.) But I 
don't see any mention of Mac. What's the status here?

It would be very cool if this conversation ends with me making a video on how a 
few simple GHC flags can allow us to, say, get a stack trace on a pattern-match 
failure in a Haskell program.

Thanks!
Richard

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


How's the integration of DWARF support coming along?

2014-08-13 Thread Johan Tibell
Hi,

How's the integration of DWARF support coming along? It's probably one of
the most important improvements to the runtime in quite some time since
unlocks *two* important features, namely

 * trustworthy profiling (using e.g. Linux perf events and other
low-overhead, code preserving, sampling profilers), and
 * stack traces.

The former is really important to move our core libraries performance up a
notch. Right now -prof is too invasive for it to be useful when evaluating
the hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server,
where you can sometimes can get some crash that only happens once a day
under very specific conditions. Knowing where the crash happens is then
*very* useful.

-- Johan
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Johan Tibell
On Wed, Aug 13, 2014 at 5:07 PM, Tuncer Ayaz tuncer.a...@gmail.com wrote:

 On Wed, Aug 13, 2014 at 5:02 PM, Johan Tibell wrote:
  Hi,
 
  How's the integration of DWARF support coming along? It's probably
  one of the most important improvements to the runtime in quite some
  time since unlocks *two* important features, namely
 
   * trustworthy profiling (using e.g. Linux perf events and other
  low-overhead, code preserving, sampling profilers), and
   * stack traces.
 
  The former is really important to move our core libraries
  performance up a notch. Right now -prof is too invasive for it to be
  useful when evaluating the hotspots in these libraries (which are
  already often heavily tuned).
 
  The latter one is really important for real life Haskell on the
  server, where you can sometimes can get some crash that only happens
  once a day under very specific conditions. Knowing where the crash
  happens is then *very* useful.

 Doesn't it also enable using gdb and lldb, or is there another missing
 piece?


No, those should also work. It enables *a lot* of generic infrastructure
that programmers has written over the years.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Ömer Sinan Ağacan
Is this stack trace support different than what we have currently?
(e.g. the one implemented with GHC.Stack and cost centers)

---
Ömer Sinan Ağacan
http://osa1.net


2014-08-13 18:02 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
 Hi,

 How's the integration of DWARF support coming along? It's probably one of
 the most important improvements to the runtime in quite some time since
 unlocks *two* important features, namely

  * trustworthy profiling (using e.g. Linux perf events and other
 low-overhead, code preserving, sampling profilers), and
  * stack traces.

 The former is really important to move our core libraries performance up a
 notch. Right now -prof is too invasive for it to be useful when evaluating
 the hotspots in these libraries (which are already often heavily tuned).

 The latter one is really important for real life Haskell on the server,
 where you can sometimes can get some crash that only happens once a day
 under very specific conditions. Knowing where the crash happens is then
 *very* useful.

 -- Johan


 ___
 ghc-devs mailing list
 ghc-devs@haskell.org
 http://www.haskell.org/mailman/listinfo/ghc-devs

___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Ömer Sinan Ağacan
Will generated stack traces be different that

---
Ömer Sinan Ağacan
http://osa1.net


2014-08-13 19:56 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
 Yes, it doesn't use any code modification so it doesn't have runtime
 overhead (except when generating the actual trace) or interfere with
 compiler optimizations. In other words you can actually have it enabled at
 all time. It only requires that you compile with -g, just like with a C
 compiler.


 On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan omeraga...@gmail.com
 wrote:

 Is this stack trace support different than what we have currently?
 (e.g. the one implemented with GHC.Stack and cost centers)

 ---
 Ömer Sinan Ağacan
 http://osa1.net


 2014-08-13 18:02 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
  Hi,
 
  How's the integration of DWARF support coming along? It's probably one
  of
  the most important improvements to the runtime in quite some time since
  unlocks *two* important features, namely
 
   * trustworthy profiling (using e.g. Linux perf events and other
  low-overhead, code preserving, sampling profilers), and
   * stack traces.
 
  The former is really important to move our core libraries performance up
  a
  notch. Right now -prof is too invasive for it to be useful when
  evaluating
  the hotspots in these libraries (which are already often heavily tuned).
 
  The latter one is really important for real life Haskell on the server,
  where you can sometimes can get some crash that only happens once a day
  under very specific conditions. Knowing where the crash happens is then
  *very* useful.
 
  -- Johan
 
 
  ___
  ghc-devs mailing list
  ghc-devs@haskell.org
  http://www.haskell.org/mailman/listinfo/ghc-devs
 


___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Ömer Sinan Ağacan
Sorry for my previous email. (used a gmail shortcut by mistake)

We won't have stacks as we have in imperative(without TCO) and strict
languages. So we still need some kind of emulation and I think this
means some extra run-time operations. I'm wondering about two things:

1) Do we still get same traces as we get using GHC.Stack right now?
2) If yes, then how can we have that without any runtime costs?

Thanks and sorry again for my previous email.

---
Ömer Sinan Ağacan
http://osa1.net


2014-08-13 20:08 GMT+03:00 Ömer Sinan Ağacan omeraga...@gmail.com:
 Will generated stack traces be different that

 ---
 Ömer Sinan Ağacan
 http://osa1.net


 2014-08-13 19:56 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
 Yes, it doesn't use any code modification so it doesn't have runtime
 overhead (except when generating the actual trace) or interfere with
 compiler optimizations. In other words you can actually have it enabled at
 all time. It only requires that you compile with -g, just like with a C
 compiler.


 On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan omeraga...@gmail.com
 wrote:

 Is this stack trace support different than what we have currently?
 (e.g. the one implemented with GHC.Stack and cost centers)

 ---
 Ömer Sinan Ağacan
 http://osa1.net


 2014-08-13 18:02 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
  Hi,
 
  How's the integration of DWARF support coming along? It's probably one
  of
  the most important improvements to the runtime in quite some time since
  unlocks *two* important features, namely
 
   * trustworthy profiling (using e.g. Linux perf events and other
  low-overhead, code preserving, sampling profilers), and
   * stack traces.
 
  The former is really important to move our core libraries performance up
  a
  notch. Right now -prof is too invasive for it to be useful when
  evaluating
  the hotspots in these libraries (which are already often heavily tuned).
 
  The latter one is really important for real life Haskell on the server,
  where you can sometimes can get some crash that only happens once a day
  under very specific conditions. Knowing where the crash happens is then
  *very* useful.
 
  -- Johan
 
 
  ___
  ghc-devs mailing list
  ghc-devs@haskell.org
  http://www.haskell.org/mailman/listinfo/ghc-devs
 


___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Johan Tibell
Without any overhead we'll get the runtime stack trace, which isn't exactly
the same as what we can get with emulation, but has the benefit that we can
leave it on in all of our shipped code if we like. This latter is a really
crucial property for stack traces to be widely useful.


On Wed, Aug 13, 2014 at 7:13 PM, Ömer Sinan Ağacan omeraga...@gmail.com
wrote:

 Sorry for my previous email. (used a gmail shortcut by mistake)

 We won't have stacks as we have in imperative(without TCO) and strict
 languages. So we still need some kind of emulation and I think this
 means some extra run-time operations. I'm wondering about two things:

 1) Do we still get same traces as we get using GHC.Stack right now?
 2) If yes, then how can we have that without any runtime costs?

 Thanks and sorry again for my previous email.

 ---
 Ömer Sinan Ağacan
 http://osa1.net


 2014-08-13 20:08 GMT+03:00 Ömer Sinan Ağacan omeraga...@gmail.com:
  Will generated stack traces be different that
 
  ---
  Ömer Sinan Ağacan
  http://osa1.net
 
 
  2014-08-13 19:56 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
  Yes, it doesn't use any code modification so it doesn't have runtime
  overhead (except when generating the actual trace) or interfere with
  compiler optimizations. In other words you can actually have it enabled
 at
  all time. It only requires that you compile with -g, just like with a C
  compiler.
 
 
  On Wed, Aug 13, 2014 at 6:45 PM, Ömer Sinan Ağacan 
 omeraga...@gmail.com
  wrote:
 
  Is this stack trace support different than what we have currently?
  (e.g. the one implemented with GHC.Stack and cost centers)
 
  ---
  Ömer Sinan Ağacan
  http://osa1.net
 
 
  2014-08-13 18:02 GMT+03:00 Johan Tibell johan.tib...@gmail.com:
   Hi,
  
   How's the integration of DWARF support coming along? It's probably
 one
   of
   the most important improvements to the runtime in quite some time
 since
   unlocks *two* important features, namely
  
* trustworthy profiling (using e.g. Linux perf events and other
   low-overhead, code preserving, sampling profilers), and
* stack traces.
  
   The former is really important to move our core libraries
 performance up
   a
   notch. Right now -prof is too invasive for it to be useful when
   evaluating
   the hotspots in these libraries (which are already often heavily
 tuned).
  
   The latter one is really important for real life Haskell on the
 server,
   where you can sometimes can get some crash that only happens once a
 day
   under very specific conditions. Knowing where the crash happens is
 then
   *very* useful.
  
   -- Johan
  
  
   ___
   ghc-devs mailing list
   ghc-devs@haskell.org
   http://www.haskell.org/mailman/listinfo/ghc-devs
  
 
 

___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Arash Rouhani

Hi Johan!

I haven't done much (just been lazy) lately, I've tried to benchmark my 
results but I don't get any sensible results at all yet.


Last time Peter said he's working on a more portable way to read dwarf 
information that doesn't require Linux. But I'm sure he'll give a more 
acurate update than me soon in this mail thread.


As for stack traces, I don't think there's any big tasks left, but I 
summarize what I have in mind:


 * The haskell interface is done and I've iterated on it a bit, so it's
   in a decent shape at least. Some parts still need testing.
 * I wish I could implement the `forceCaseContinuation` that I've
   described in my thesis. If someone is good with code generation (I
   just suck at it, it's probably simple) and is willing to assist me a
   bit, please say so. :)
 * I tried benchmarking, I gave up after not getting any useful results.
 * I'm unfortunately totally incapable to help out with dwarf debug
   data generation, only Peter knows that part, particularly I never
   grasped his theoretical framework of causality in Haskell.
 * Peter and I have finally agreed on a simple and sensible way to
   implement `catchWithStack` that have all most good properties you
   would like. I just need to implement it and test it. I can
   definitely man up and implement this. :)

Here's my master thesis btw [1], it should answer Ömer's question of how 
we retrieve a stack from a language you think won't have a stack. :)


Cheers,
Arash

[1]: http://arashrouhani.com/papers/master-thesis.pdf




On 2014-08-13 17:02, Johan Tibell wrote:

Hi,

How's the integration of DWARF support coming along? It's probably one 
of the most important improvements to the runtime in quite some time 
since unlocks *two* important features, namely


 * trustworthy profiling (using e.g. Linux perf events and other 
low-overhead, code preserving, sampling profilers), and

 * stack traces.

The former is really important to move our core libraries performance 
up a notch. Right now -prof is too invasive for it to be useful when 
evaluating the hotspots in these libraries (which are already often 
heavily tuned).


The latter one is really important for real life Haskell on the 
server, where you can sometimes can get some crash that only happens 
once a day under very specific conditions. Knowing where the crash 
happens is then *very* useful.


-- Johan



___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Johan Tibell
What's the minimal amount of work we need to do to just get the dwarf data
in the codegen by 7.10 (RC late december) so we can start using e.g. linux
perf events to profile Haskell programs?


On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani rar...@student.chalmers.se
wrote:

  Hi Johan!

 I haven't done much (just been lazy) lately, I've tried to benchmark my
 results but I don't get any sensible results at all yet.

 Last time Peter said he's working on a more portable way to read dwarf
 information that doesn't require Linux. But I'm sure he'll give a more
 acurate update than me soon in this mail thread.

 As for stack traces, I don't think there's any big tasks left, but I
 summarize what I have in mind:

- The haskell interface is done and I've iterated on it a bit, so it's
in a decent shape at least. Some parts still need testing.
- I wish I could implement the `forceCaseContinuation` that I've
described in my thesis. If someone is good with code generation (I just
suck at it, it's probably simple) and is willing to assist me a bit, please
say so. :)
- I tried benchmarking, I gave up after not getting any useful results.
 - I'm unfortunately totally incapable to help out with dwarf debug
data generation, only Peter knows that part, particularly I never grasped
his theoretical framework of causality in Haskell.
- Peter and I have finally agreed on a simple and sensible way to
implement `catchWithStack` that have all most good properties you would
like. I just need to implement it and test it. I can definitely man up and
implement this. :)

 Here's my master thesis btw [1], it should answer Ömer's question of how
 we retrieve a stack from a language you think won't have a stack. :)

 Cheers,
 Arash

 [1]: http://arashrouhani.com/papers/master-thesis.pdf





 On 2014-08-13 17:02, Johan Tibell wrote:

 Hi,

  How's the integration of DWARF support coming along? It's probably one
 of the most important improvements to the runtime in quite some time since
 unlocks *two* important features, namely

   * trustworthy profiling (using e.g. Linux perf events and other
 low-overhead, code preserving, sampling profilers), and
  * stack traces.

  The former is really important to move our core libraries performance up
 a notch. Right now -prof is too invasive for it to be useful when
 evaluating the hotspots in these libraries (which are already often heavily
 tuned).

  The latter one is really important for real life Haskell on the server,
 where you can sometimes can get some crash that only happens once a day
 under very specific conditions. Knowing where the crash happens is then
 *very* useful.

  -- Johan



 ___
 ghc-devs mailing 
 listghc-devs@haskell.orghttp://www.haskell.org/mailman/listinfo/ghc-devs



 ___
 ghc-devs mailing list
 ghc-devs@haskell.org
 http://www.haskell.org/mailman/listinfo/ghc-devs


___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Arash Rouhani
Peter will have to answer that. But it seemed to me that it has been 
working fine all the time. I suppose it's just to resolve merge 
conflicts. There were some refactorings he wanted to do. In addition to 
this it will also be some packaging issues I suppose. I'm hoping Peter 
will answer in this mail thread soon, since he knows this much better.


/Arash

On 2014-08-13 20:01, Johan Tibell wrote:
What's the minimal amount of work we need to do to just get the dwarf 
data in the codegen by 7.10 (RC late december) so we can start using 
e.g. linux perf events to profile Haskell programs?



On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani 
rar...@student.chalmers.se mailto:rar...@student.chalmers.se wrote:


Hi Johan!

I haven't done much (just been lazy) lately, I've tried to
benchmark my results but I don't get any sensible results at all yet.

Last time Peter said he's working on a more portable way to read
dwarf information that doesn't require Linux. But I'm sure he'll
give a more acurate update than me soon in this mail thread.

As for stack traces, I don't think there's any big tasks left, but
I summarize what I have in mind:

  * The haskell interface is done and I've iterated on it a bit,
so it's in a decent shape at least. Some parts still need testing.
  * I wish I could implement the `forceCaseContinuation` that I've
described in my thesis. If someone is good with code
generation (I just suck at it, it's probably simple) and is
willing to assist me a bit, please say so. :)
  * I tried benchmarking, I gave up after not getting any useful
results.
  * I'm unfortunately totally incapable to help out with dwarf
debug data generation, only Peter knows that part,
particularly I never grasped his theoretical framework of
causality in Haskell.
  * Peter and I have finally agreed on a simple and sensible way
to implement `catchWithStack` that have all most good
properties you would like. I just need to implement it and
test it. I can definitely man up and implement this. :)

Here's my master thesis btw [1], it should answer Ömer's question
of how we retrieve a stack from a language you think won't have a
stack. :)

Cheers,
Arash

[1]: http://arashrouhani.com/papers/master-thesis.pdf





On 2014-08-13 17:02, Johan Tibell wrote:

Hi,

How's the integration of DWARF support coming along? It's
probably one of the most important improvements to the runtime in
quite some time since unlocks *two* important features, namely

 * trustworthy profiling (using e.g. Linux perf events and other
low-overhead, code preserving, sampling profilers), and
 * stack traces.

The former is really important to move our core libraries
performance up a notch. Right now -prof is too invasive for it to
be useful when evaluating the hotspots in these libraries (which
are already often heavily tuned).

The latter one is really important for real life Haskell on the
server, where you can sometimes can get some crash that only
happens once a day under very specific conditions. Knowing where
the crash happens is then *very* useful.

-- Johan



___
ghc-devs mailing list
ghc-devs@haskell.org  mailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs



___
ghc-devs mailing list
ghc-devs@haskell.org mailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs




___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Peter Wortmann


At this point I have a bit more time on my hands again (modulo post-thesis 
vacations), but we are basically still in “review hell”.

I think “just” for perf_events support we’d need the following patches[1]:
1. Source notes (Core support)
2. Source notes (CorePrep  Stg support)
3. Source notes (Cmm support)
4. Tick scopes
5. Debug data extraction (NCG support)
6. Generate .loc/.file directives

We have a basic “okay” from the Simons up to number 2 (conditional on better 
documentation). Number 4 sticks out because Simon Marlow wanted to have a 
closer look at it - this is basically about how to maintain source ticks in a 
robust fashion on the Cmm level (see also section 5.5 of my thesis[2]).

Meanwhile I have ported NCG DWARF generation over to Mac Os, and am working on 
reviving LLVM support. My plan was to check that I didn’t accidentally break 
Linux support, then push for review again in a week or so (Phab?).

Greetings,
  Peter

[1] https://github.com/scpmw/ghc/commits/profiling-import
[2] http://www.personal.leeds.ac.uk/~scpmw/static/thesis.pdf

On 13 Aug 2014, at 20:01, Johan Tibell 
johan.tib...@gmail.commailto:johan.tib...@gmail.com wrote:

What's the minimal amount of work we need to do to just get the dwarf data in 
the codegen by 7.10 (RC late december) so we can start using e.g. linux perf 
events to profile Haskell programs?


On Wed, Aug 13, 2014 at 7:31 PM, Arash Rouhani 
rar...@student.chalmers.semailto:rar...@student.chalmers.se wrote:
Hi Johan!

I haven't done much (just been lazy) lately, I've tried to benchmark my results 
but I don't get any sensible results at all yet.

Last time Peter said he's working on a more portable way to read dwarf 
information that doesn't require Linux. But I'm sure he'll give a more acurate 
update than me soon in this mail thread.

As for stack traces, I don't think there's any big tasks left, but I summarize 
what I have in mind:

 *   The haskell interface is done and I've iterated on it a bit, so it's in a 
decent shape at least. Some parts still need testing.
 *   I wish I could implement the `forceCaseContinuation` that I've described 
in my thesis. If someone is good with code generation (I just suck at it, it's 
probably simple) and is willing to assist me a bit, please say so. :)
 *   I tried benchmarking, I gave up after not getting any useful results.
 *   I'm unfortunately totally incapable to help out with dwarf debug data 
generation, only Peter knows that part, particularly I never grasped his 
theoretical framework of causality in Haskell.
 *   Peter and I have finally agreed on a simple and sensible way to implement 
`catchWithStack` that have all most good properties you would like. I just need 
to implement it and test it. I can definitely man up and implement this. :)

Here's my master thesis btw [1], it should answer Ömer's question of how we 
retrieve a stack from a language you think won't have a stack. :)

Cheers,
Arash

[1]: http://arashrouhani.com/papers/master-thesis.pdf





On 2014-08-13 17:02, Johan Tibell wrote:
Hi,

How's the integration of DWARF support coming along? It's probably one of the 
most important improvements to the runtime in quite some time since unlocks 
*two* important features, namely

 * trustworthy profiling (using e.g. Linux perf events and other low-overhead, 
code preserving, sampling profilers), and
 * stack traces.

The former is really important to move our core libraries performance up a 
notch. Right now -prof is too invasive for it to be useful when evaluating the 
hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server, where 
you can sometimes can get some crash that only happens once a day under very 
specific conditions. Knowing where the crash happens is then *very* useful.

-- Johan




___
ghc-devs mailing list
ghc-devs@haskell.orgmailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs



___
ghc-devs mailing list
ghc-devs@haskell.orgmailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs



___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs


Re: How's the integration of DWARF support coming along?

2014-08-13 Thread Peter Wortmann
 all most good properties you would like. I just need 
to implement it and test it. I can definitely man up and implement this. :)

Here's my master thesis btw [1], it should answer Ömer's question of how we 
retrieve a stack from a language you think won't have a stack. :)

Cheers,
Arash

[1]: http://arashrouhani.com/papers/master-thesis.pdf





On 2014-08-13 17:02, Johan Tibell wrote:
Hi,

How's the integration of DWARF support coming along? It's probably one of the 
most important improvements to the runtime in quite some time since unlocks 
*two* important features, namely

 * trustworthy profiling (using e.g. Linux perf events and other low-overhead, 
code preserving, sampling profilers), and
 * stack traces.

The former is really important to move our core libraries performance up a 
notch. Right now -prof is too invasive for it to be useful when evaluating the 
hotspots in these libraries (which are already often heavily tuned).

The latter one is really important for real life Haskell on the server, where 
you can sometimes can get some crash that only happens once a day under very 
specific conditions. Knowing where the crash happens is then *very* useful.

-- Johan




___
ghc-devs mailing list
ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs



___
ghc-devs mailing list
ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.orgmailto:ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs





___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs