Re: Dynamically choosing the main function

[Shea Levy]

What would you be able to achieve with this that you couldn't achieve
with branching in a fixed custom main function?

Thanks,
Shea

Harendra Kumar  writes:

> Hi,
>
> GHC allows choosing a main function at link time using the "-main-is"
> option. I was wondering if there is a possibility to choose the main
> function at runtime. Or even better, if something equivalent to "ghc -e"
> is somehow possible in a linked binary executable. If not, are there any
> plans to achieve something like that in future? Are there any theoretical,
> practical obstacles to that?
>
> Thanks,
> Harendra
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Including remote-iserv upstream?

[Shea Levy]

OK, will do, thanks!

Simon Marlow <marlo...@gmail.com> writes:

> Absolutely, let's get this code upstream.  Just put it up on Phabricator
> and I'll be happy to review.
>
> I recall that we wanted to split up the ghci lib into modules that are
> compiled with stage0 (the client) and modules compiled with stage1 (the
> server).  Is that a part of your plans?  I think it would be a good cleanup.
>
> Cheers
> Simon
>
> On 14 January 2017 at 15:34, Shea Levy <s...@shealevy.com> wrote:
>
>> Hi Simon, devs,
>>
>> As part of my work to get TH working when cross-compiling to iOS, I've
>> developed remote-iserv [1] (not yet on hackage), a set of libraries for
>> letting GHC communicate with an external interpreter that may be on
>> another machine. So far, there are only three additions of note on top
>> of what the ghci library offers:
>>
>> 1. The remote-iserv protocol has facilities for the host sending
>>archives and object files the target doesn't have (dynlibs not yet
>>implemented but there's no reason they can't be). This works by
>>having the server send back a Bool after a loadObj or loadArchive
>>indicating whether it needs the object sent, and then just reading it
>>off the Pipe.
>> 2. The remote-iserv lib abstracts over how the Pipe it communicates over
>>is obtained. One could imagine e.g. an ssh-based implementation that
>>just uses stdin and stdout* for the communication, the implementation
>>I've actually tested on is a TCP server advertised over bonjour.
>> 3. There is a protocol version included to address forwards
>>compatibility concerns.
>>
>> As the library currently stands, it works for my use case. However,
>> there would be a number of benefits if it were included with ghc (and
>> used for local iserv as well):
>>
>> 1. Reduced code duplication (the server side copies iserv/src/Main.hs
>>pretty heavily)
>> 2. Reduced overhead keeping up to date with iserv protocol changes
>> 3. No need for an extra client-side process, GHC can just open the Pipe
>>itself
>> 4. Proper library distribution in the cross-compiling case. The client
>>needs to be linked with the ghci lib built by the stage0 compiler, as
>>it runs on the build machine, while the server needs to be linked
>>with the ghci lib built by the stage1 compiler. With a distribution
>>created by 'make install', we only get the ghci lib for the
>>target. Currently, I'm working around this by just using the ghci lib
>>of the bootstrap compiler, which in my case is built from the same
>>source checkout, but of course this isn't correct. If these libs were
>>upstream, we'd only need one version of the client lib exposed and
>>one version of the server lib exposed and could have them be for the
>>build machine and the target, respectively
>> 5. Better haskell hackers than I invested in the code ;)
>>
>> Thoughts on this? Would this be welcome upstream in some form?
>>
>> Thanks,
>> Shea
>>
>> * Note that, in the general case, having the server process's stdio be
>>   the same as the compiler's (as we have in the local-iserv case) is not
>>   possible. Future work includes adding something to the protocol to
>>   allow forwarding stdio over the protocol pipe, to make GHCi usable
>>   without watching the *server*'s console.
>>
>> [1]: https://github.com/obsidiansystems/remote-iserv
>>


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Including remote-iserv upstream?

[Shea Levy]

No, as it currently exists it has to create a ProcessHandle, and I have
to layer some stuff on top of the iserv protocol anyway, so it wouldn't
really help much. I just use -pgmi to point to the client executable.

~Shea

Alan & Kim Zimmerman  writes:

> As a matter of interest, are you making use of `createIservProcessHook`
> which allows you to set the FDs to be used when communicating with the
> iserve process?
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Including remote-iserv upstream?

[Shea Levy]

Hi Simon, devs,

As part of my work to get TH working when cross-compiling to iOS, I've
developed remote-iserv [1] (not yet on hackage), a set of libraries for
letting GHC communicate with an external interpreter that may be on
another machine. So far, there are only three additions of note on top
of what the ghci library offers:

1. The remote-iserv protocol has facilities for the host sending
   archives and object files the target doesn't have (dynlibs not yet
   implemented but there's no reason they can't be). This works by
   having the server send back a Bool after a loadObj or loadArchive
   indicating whether it needs the object sent, and then just reading it
   off the Pipe.
2. The remote-iserv lib abstracts over how the Pipe it communicates over
   is obtained. One could imagine e.g. an ssh-based implementation that
   just uses stdin and stdout* for the communication, the implementation
   I've actually tested on is a TCP server advertised over bonjour.
3. There is a protocol version included to address forwards
   compatibility concerns.

As the library currently stands, it works for my use case. However,
there would be a number of benefits if it were included with ghc (and
used for local iserv as well):

1. Reduced code duplication (the server side copies iserv/src/Main.hs
   pretty heavily)
2. Reduced overhead keeping up to date with iserv protocol changes
3. No need for an extra client-side process, GHC can just open the Pipe
   itself
4. Proper library distribution in the cross-compiling case. The client
   needs to be linked with the ghci lib built by the stage0 compiler, as
   it runs on the build machine, while the server needs to be linked
   with the ghci lib built by the stage1 compiler. With a distribution
   created by 'make install', we only get the ghci lib for the
   target. Currently, I'm working around this by just using the ghci lib
   of the bootstrap compiler, which in my case is built from the same
   source checkout, but of course this isn't correct. If these libs were
   upstream, we'd only need one version of the client lib exposed and
   one version of the server lib exposed and could have them be for the
   build machine and the target, respectively
5. Better haskell hackers than I invested in the code ;)

Thoughts on this? Would this be welcome upstream in some form?

Thanks,
Shea

* Note that, in the general case, having the server process's stdio be
  the same as the compiler's (as we have in the local-iserv case) is not
  possible. Future work includes adding something to the protocol to
  allow forwarding stdio over the protocol pipe, to make GHCi usable
  without watching the *server*'s console.

[1]: https://github.com/obsidiansystems/remote-iserv


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Reason for fixing minimum bootstrap version at 2 major releases ago?

[Shea Levy]

Hi all,

I'm wondering, why do we require ghc to be bootstrappable with the past
2 major releases instead of just the past 1? Is it a common case that
someone is compiling GHC but can't easily get the latest release?

Thanks,
Shea


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Making (useful subsets of) bytecode portable between targets

[Shea Levy]

Simon Marlow  writes:

> The right thing is to have a clean separation between runtime
> imports and compile-time imports.  Perhaps we just annotate some imports to
> say they aren't needed at compile-time for running the TH code.  but then
> we also need compile-time vs. runtime build-depends in our .cabal files,
> and so on.

Yes, I was just looking into this yesterday. We already have something
similar for plugins, though of course the TH story is much more
involved (in part because GHC has to compile haskell code, not just load
and run a pre-existing object file).

> Its a big project, but ultimately we do have to tackle it, because it's the
> right thing to do.  Anyone interested in working on this?  Maybe start a
> new proposal to flesh out the details.

Yeah, I'm going to at least try. Don't know where formal proposals go,
but what I've got so far (definitely subject to change!):

1. Add a separate package database like we have for plugins for TH
   imports, defaulting to the normal package database when empty
2. Use the TH package db for code to be *run*, but resolve all
   reifications etc. against the normal target scope.
3. Add {-# TH #-}-annotated imports to specify modules to import
   compile-time code from (if a module *also* has definitions to be
   reified, it needs to be imported twice)
   3a. If a module has any {-# TH #-} imports, enforce that *all*
   compile-time executed code is pulled in via {-# TH #-} imports.
   3b. Optionally add a warning for TH-using code that doesn't use
   {-# TH #-}, as the first start of a migration path to enforcing
   compiletime/runtime separation across the ecosystem.
   3c. Not sure what the specific difficulties are that require the
   staging restrictions, but possibly annotating same-module
   definitions with {-# TH -} might make this easier?
4. Teach cabal how to add packages to the TH database
   4a. Not sure how all this works, but probably this is the place to
   e.g. request the non-profiled version of a package if ghc is
   non-profiled and the target code is profiled
5. Modify the build process of the cross-compiler to build a stage-2
   compiler that builds code for the target *and* host (at least, enough
   of the host to build BCOs), and runs that host-targeted code in the
   interpreter.
   5a. This will require some way to distinguish
   stage-2-as-target-native-compiler VS stage-2-as-hybrid-compiler.


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Making (useful subsets of) bytecode portable between targets

[Shea Levy]

Hi all,

I'm interested in implementing a general solution for TH during
cross-compilation, which if my naive lack-of-understanding is correct
will broadly involve the following three tasks:

1. Make the generation of byte code, or at least the subset needed for
   useful TH use, target-independent
2. Teach the external-interpreter to launch a build-native iserv when
   cross-compiling
3. Teach cabal to compile dependencies and modules for the build and
   target when cross-compiling and TH is used

Of course, due to the generality of TH there will be code you can write
that would be different in this approach from what you would get with a
fully native compilation (e.g. due to GHC conditional compilation in the
TH functions or FFI that does runtime host introspection), but since
requiring a target device at build time is in many cases impractical (do
you want to hook up an iPhone to your build farm?) and in some cases
probably impossible (for targets without the resources to run a full GHC
even if they can run GHC-compiled code) I think this is a reasonable
restriction to require.

My questions to the list are:

* Is 1 above a pipe dream, keeping in mind that we are assuming the
  availability of build-native versions of all dependencies and
  already-compiled modules?
* Any pointers for getting started with 1?
* Anything I'm missing?

Thanks,
Shea


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs