RE: Plan for GHC 9.2

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Yes I agree, unlifted data types would be terrific.
>
> From: ghc-devs  On Behalf Of Sebastian Graf
> Sent: 11 February 2021 10:25
> To: Ben Gamari 
> Cc: ghc-devs 
> Subject: Re: Plan for GHC 9.2
>
> Hi,
>
> Since my hopes of finally merging Nested CPR have recently been crushed 
> again, I hope that we can include the implementation of the UnliftedDatatypes 
> extension 
> (proposal,
>  
> implementation).
> It was on ice since it depends on the BoxedRep proposal, but if
> BoxedRep is going to make it, surely UnliftedDatatypes can make it,
> too.
>
> I expect quite a few bugs, simply because I don't have much code to
> test it on yet. But I'm very confident that existing code isn't
> impacted by that, as most of the functionality (CodeGen for unlifted
> types, most importantly) was already there and I only had to refine a
> conditional here and there.
>
It would indeed be very exciting to finally have UnliftedDataTypes. If
things turn out to be non-trivial then I think we should open to letting
it slide for 9.4, but otherwise I am open to merging for 9.2 if a patch
appeared.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Plan for GHC 9.2

[Simon Peyton Jones via ghc-devs]

Yes I agree, unlifted data types would be terrific.

From: ghc-devs  On Behalf Of Sebastian Graf
Sent: 11 February 2021 10:25
To: Ben Gamari 
Cc: ghc-devs 
Subject: Re: Plan for GHC 9.2

Hi,

Since my hopes of finally merging Nested CPR have recently been crushed again, 
I hope that we can include the implementation of the UnliftedDatatypes 
extension 
(proposal,
 
implementation).
It was on ice since it depends on the BoxedRep proposal, but if BoxedRep is 
going to make it, surely UnliftedDatatypes can make it, too.
I expect quite a few bugs, simply because I don't have much code to test it on 
yet. But I'm very confident that existing code isn't impacted by that, as most 
of the functionality (CodeGen for unlifted types, most importantly) was already 
there and I only had to refine a conditional here and there.

Cheers,
Sebastian

Am Mi., 10. Feb. 2021 um 18:42 Uhr schrieb Ben Gamari 
mailto:b...@well-typed.com>>:
Roland Senn mailto:r...@bluewin.ch>> writes:

> I hope ticket #19157 will make it in the GHC 9.2 release. In the GHCi
> debugger it adds the possibility to set ignore counts to breakpoints.
> The next  times the break point is reached the program's
> execution does not stop. This feature is available in nearly every
> debugger, but until now not yet in the GHCi debugger.
> Merge request !4839 is ready for review  (and it's NOT rocket
> science...)
>
Indeed, this seems quite reasonable. I don't see any reason why we
shouldn't be able to get it in to 9.2.1.

Cheers,

- Ben

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Plan for GHC 9.2

[Sebastian Graf]

Hi,

Since my hopes of finally merging Nested CPR have recently been crushed
again, I hope that we can include the implementation of the
UnliftedDatatypes extension (proposal
,
implementation ).
It was on ice since it depends on the BoxedRep proposal, but if BoxedRep is
going to make it, surely UnliftedDatatypes can make it, too.
I expect quite a few bugs, simply because I don't have much code to test it
on yet. But I'm very confident that existing code isn't impacted by that,
as most of the functionality (CodeGen for unlifted types, most importantly)
was already there and I only had to refine a conditional here and there.

Cheers,
Sebastian

Am Mi., 10. Feb. 2021 um 18:42 Uhr schrieb Ben Gamari :

> Roland Senn  writes:
>
> > I hope ticket #19157 will make it in the GHC 9.2 release. In the GHCi
> > debugger it adds the possibility to set ignore counts to breakpoints.
> > The next  times the break point is reached the program's
> > execution does not stop. This feature is available in nearly every
> > debugger, but until now not yet in the GHCi debugger.
> > Merge request !4839 is ready for review  (and it's NOT rocket
> > science...)
> >
> Indeed, this seems quite reasonable. I don't see any reason why we
> shouldn't be able to get it in to 9.2.1.
>
> Cheers,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Stop holding hadrian back with backwards compatibility

[Moritz Angermann]

Tamar,

thanks so much for the backstory and the tickets. I’ll go dig down this
path a bit more.

Cheers,
 Moritz

On Thu, 11 Feb 2021 at 5:31 PM, Phyx  wrote:

> Hi, Just leaving my two cents feel free to ignore..
>
> > I almost suggested that this had to be the reason for the back-compat
> design
>
> You're right, but not for backwards compat of Hadrian vs Make, but for
> compat with RTS versions.
> I could be wrong, but my understanding is the current design in Make is
> just an artifact of getting something that works on all OSes without much
> pain, but has proven to be suboptimal in a very important use case (slight
> detour time):
>
> You have to make a choice of which RTS to use at compile time.  Which is
> quite bad.  Because it means that you can't swap between two RTS flavors
> with the same ABI. It also means building presents a problem, you want your
> compiler at the end of stage1 to use your new rts, not the one of the
> stage0 compiler.
>
> You can't have multiple versions of the RTS in one library, but if you
> have the full name as a dependency the dynamic loader happily loads you
> multiple copies.
>
> To solve this issue the design was made to not declare the RTS as a
> dependency on any haskell library. i.e. there's not DT_NEEDED entry for it
> on ELF operating systems.  Which means before you load a Haskell produced
> dynamic library on Linux you need to LD_PRELOAD an rts. It's clunky, but it
> works, it allows you to switch between debug and non-debug rts at
> initialization time.
>
> On Windows, this problem was punted, because everything is statically
> linked.  But the problem exists that you can have multiple DLLs with
> different RTS and ABIs.  This is fine as long as the DLLs have no
> dependencies on each other. Once they do... you have a big problem.  This
> is one of the primary blockers of shared library support on Windows.
>
> I.. don't know whatever wacky solution MacOS uses so can't comment there.
>
> Now back to the original question about version 1.0, this has nothing to
> do with Make at all. Make based system only implemented the scheme that was
> wanted. It's not like any Make system design issues forced this scheme. Now
> over the years, assumptions that the RTS is always version 1.0 could have
> krept into the build system.  But I don't believe this to have been design,
> just convenience. Right now, the design only requires you to know the GHC
> version, which is available in all makefiles.  Knowing the RTS version
> would be difficult, but the point is that in a proper design you don't need
> to know the version.
>
> Almost half a decade ago a plan was made to replace this scheme with one
> that would work on all OSes and would allow us to solve these issues. The
> design was made and debated here
> https://gitlab.haskell.org/ghc/ghc/-/issues/10352
>
> The actual solution isn't as simple as just adding the rts version to the
> library name or add it only to the build system, in fact this would be the
> wrong approach as it makes it impossible to observe backwards compatibility
> between GHC releases.
> i.e. without it, you'd need to have GHC 9.0.1 installed to run GHC 9.0.1
> programs, you can't run using GHC 9.2.x rts if the version changed.
>
> Typically ELF based platforms solve this by a combination of SONAME and
> symbol versioning.  Windows solves this by a combination of SxS Assembly
> versioning or mingw style SONAME.
>
> All of which require you to have the same filename for the libraries, but
> use a different path to disambiguate:
>
> lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so
>
> lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so
>
> lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so
>
> lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so
>
> lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so
>
> for each RTS with the same ABI. profiling libs for instance have a
> different ABI and can't use this scheme.
>
> So what has taken so long to implement this? Well.. time. As it turns out,
> getting this scheme to work required a lot of foundational work in GHC
> (Particularly on Windows where dynamic linking design wasn't optimal, but
> both GHC and the dynamic linker are happy now).
>
> On Linux it took a while to get SONAME support in cabal
> https://github.com/haskell/cabal/issues/4052 so we don't have to hack
> around in the build system.
>
> But anyway this is why the current scheme exists, and why just adding an
> rts version isn't really sufficient, especially if the name propagates to
> the shared lib.
>
> TL;DR;
>
> If we are going to change the build system, we should do it properly.
>
> The current scheme exists because GHC does not observe any mechanism to
> support multiple runtimes with the same ABI and does not really have a
> backwards compatibility story.
>
> Kind Regards,
>
> Tamar
>
> On Wed, Feb 10, 2021 at 11:00 PM Richard Eisenberg 
> wrote:
>
>>
>>
>> On Feb 10, 2021, at 8:50 AM, Simon Peyton Jones 
>> wrote:
>>
>> build with hadrian, and 

Re: Stop holding hadrian back with backwards compatibility

[Phyx]

Hi, Just leaving my two cents feel free to ignore..

> I almost suggested that this had to be the reason for the back-compat
design

You're right, but not for backwards compat of Hadrian vs Make, but for
compat with RTS versions.
I could be wrong, but my understanding is the current design in Make is
just an artifact of getting something that works on all OSes without much
pain, but has proven to be suboptimal in a very important use case (slight
detour time):

You have to make a choice of which RTS to use at compile time.  Which is
quite bad.  Because it means that you can't swap between two RTS flavors
with the same ABI. It also means building presents a problem, you want your
compiler at the end of stage1 to use your new rts, not the one of the
stage0 compiler.

You can't have multiple versions of the RTS in one library, but if you have
the full name as a dependency the dynamic loader happily loads you multiple
copies.

To solve this issue the design was made to not declare the RTS as a
dependency on any haskell library. i.e. there's not DT_NEEDED entry for it
on ELF operating systems.  Which means before you load a Haskell produced
dynamic library on Linux you need to LD_PRELOAD an rts. It's clunky, but it
works, it allows you to switch between debug and non-debug rts at
initialization time.

On Windows, this problem was punted, because everything is statically
linked.  But the problem exists that you can have multiple DLLs with
different RTS and ABIs.  This is fine as long as the DLLs have no
dependencies on each other. Once they do... you have a big problem.  This
is one of the primary blockers of shared library support on Windows.

I.. don't know whatever wacky solution MacOS uses so can't comment there.

Now back to the original question about version 1.0, this has nothing to do
with Make at all. Make based system only implemented the scheme that was
wanted. It's not like any Make system design issues forced this scheme. Now
over the years, assumptions that the RTS is always version 1.0 could have
krept into the build system.  But I don't believe this to have been design,
just convenience. Right now, the design only requires you to know the GHC
version, which is available in all makefiles.  Knowing the RTS version
would be difficult, but the point is that in a proper design you don't need
to know the version.

Almost half a decade ago a plan was made to replace this scheme with one
that would work on all OSes and would allow us to solve these issues. The
design was made and debated here
https://gitlab.haskell.org/ghc/ghc/-/issues/10352

The actual solution isn't as simple as just adding the rts version to the
library name or add it only to the build system, in fact this would be the
wrong approach as it makes it impossible to observe backwards compatibility
between GHC releases.
i.e. without it, you'd need to have GHC 9.0.1 installed to run GHC 9.0.1
programs, you can't run using GHC 9.2.x rts if the version changed.

Typically ELF based platforms solve this by a combination of SONAME and
symbol versioning.  Windows solves this by a combination of SxS Assembly
versioning or mingw style SONAME.

All of which require you to have the same filename for the libraries, but
use a different path to disambiguate:

lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so

for each RTS with the same ABI. profiling libs for instance have a
different ABI and can't use this scheme.

So what has taken so long to implement this? Well.. time. As it turns out,
getting this scheme to work required a lot of foundational work in GHC
(Particularly on Windows where dynamic linking design wasn't optimal, but
both GHC and the dynamic linker are happy now).

On Linux it took a while to get SONAME support in cabal
https://github.com/haskell/cabal/issues/4052 so we don't have to hack
around in the build system.

But anyway this is why the current scheme exists, and why just adding an
rts version isn't really sufficient, especially if the name propagates to
the shared lib.

TL;DR;

If we are going to change the build system, we should do it properly.

The current scheme exists because GHC does not observe any mechanism to
support multiple runtimes with the same ABI and does not really have a
backwards compatibility story.

Kind Regards,

Tamar

On Wed, Feb 10, 2021 at 11:00 PM Richard Eisenberg  wrote:

>
>
> On Feb 10, 2021, at 8:50 AM, Simon Peyton Jones 
> wrote:
>
> build with hadrian, and then continue using make with the artifacts
> (partially) built by Hadrian
>
>
> I almost suggested that this had to be the reason for the back-compat
> design, but I assumed I had to be wrong. I also agree this is a non-goal;
> I'm quite happy to be forced to pick one or the other and stick with that
> choice until blasting away all build products.