RE: How to get started with a new backend?
I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. Someone should start a wiki page about this! It comes up regularly. (That doesn't mean it's a bad idea; just that we should share wisdom on it.) Maybe there is one, in which case perhaps it could be updated? Some thoughts (which, if someone would like to transfer these to the wiki page, perhaps in more coherent form): *Check out Frege. Maybe others. I think several folk are working on .NET or JVM projects. *Do you want to inter-operate with .NET, or compile to .NET bytecode. These are very different? I'll assume the latter. *Do you want to compile to verifiable .NET bytecode? This is hard. GHC's type system is more expressive than .NET's in many ways (higher kinded type variables, kind polymorphism...). And less expressive in others (sub-typing). Compiling to verifiable bytecode would require lots of run-time type-tests (which we know will succeed) to reassure .NET that it's all kosher. Figuring out how to minimise these tests would be a challenge in itself. *And of course this affects whether you can go via C#, since that requires the program to be well typed in the .NET sense. *Do you want simply to run Haskell programs on .NET or do you want Haskell programs to call .NET libraries? I assume the latter. But that means that .NET types must be manifested somehow as Haskell types. A big issue is how much of .NET's type system you want to suck into Haskell. A stressful example: can you define in Haskell a type that sub-classes an imported .NET type. Daan and Mark and Sigbjorn and I wrote papers about this stuff. Look at the papers under Foreign language integration on http://research.microsoft.com/en-us/um/people/simonpj/papers/papers.html. Oh, and this: http://research.microsoft.com/en-us/um/people/simonpj/papers/oo-haskell/index.htm *I'd be inclined to start from Core rather than STG, perhaps the output of CorePrep. STG is a bit encrusted (though we're about to simplify it a bit). But honestly they are pretty similar. Cmm would be a bad choice. It's far too low level. *Another major issues is that of primops (prelude/primops.txt.pp). Things like add Int# or Float# are fine, but what about forking a thread, or throwing an exception? o One possibility is to try to emulate them all. I have no idea how hard that would be, but GHC's I/O libraries rely heavily on lightweight concurrency. o Another is to trim the list of PrimOps, and only support some of them. That's easier, but you'd have to dump a lot of the 'base' package (particularly I/O) and re-implement it using the .NET libraries instead. I hope this is of at least some use Simon From: glasgow-haskell-users-boun...@haskell.org [mailto:glasgow-haskell-users-boun...@haskell.org] On Behalf Of Jason Dagit Sent: 28 January 2013 01:16 To: glasgow-haskell-users@haskell.org Subject: How to get started with a new backend? I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard
Re: How to get started with a new backend?
On 01/28/13 02:15 AM, Jason Dagit wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. Hello Jason, I do have a checkout of LambdaVM (GHC for JDK backend) here on a drive. If you like it, I'll upload somewhere for you. Thanks, Karel ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On 28/01/13 11:21, Simon Peyton-Jones wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. Someone should start a wiki page about this! It comes up regularly. (That doesn’t mean it’s a bad idea; just that we should share wisdom on it.) Maybe there is one, in which case perhaps it could be updated? There's the FAQ entry about this, which I believe you wrote: http://www.haskell.org/haskellwiki/GHC:FAQ#Why_isn.27t_GHC_available_for_.NET_or_on_the_JVM.3F It's on the Haskell wiki, so people could update it with more recent info. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On 28/01/13 01:15, Jason Dagit wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard the LLVM codegen is relatively simple. * Any other advice? Just to put things in perspective a bit, the LLVM backend shares the RTS with the native backend, and uses exactly the same ABI. That limits its scope significantly: it only has to replace the stages between Cmm and assembly code, everything else works as-is. You don't have this luxury with .NET (or JVM), because you can't link .NET or JVM code to native code directly, and these systems already have their own runtimes. Basically you're replacing not only the code generator, but also the runtime, and probably large chunks of the libraries. That's why it's a bigger job. You can't go from Cmm, because as Simon says it's already too low-level. You'll want .NET/JVM to manage the stack for you, and you'll want to have your own compilation scheme for functions and thunks, and so on. The right place to start is after CorePrep, where thunks are explicit (this is where the bytecode generator starts, incidentally: you might want to look at ghci/ByteCodeGen.hs). Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On 28/01/13 06:35, Christopher Done wrote: The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Wait, it can? Where's that? It used to be able to. Nowadays we cross-compile. Cheers, Simon On 28 January 2013 02:15, Jason Dagit dag...@gmail.com wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard the LLVM codegen is relatively simple. * Any other advice? Thank you in advance! Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On Jan 27, 2013, at 10:35 PM, Christopher Done chrisd...@gmail.com wrote: The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Wait, it can? Where's that? Look at unregistered builds. For example: http://hackage.haskell.org/trac/ghc/wiki/Commentary/Compiler/Backends/PprC I don't know how we'll it works, but the generated code is meant for people porting GHC to a new platform. Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On Jan 27, 2013, at 11:21 PM, David Terei davidte...@gmail.com wrote: I believe that some Microsoft Research folks at Cambridge got a fairly far along implementation of Haskell on Java or .NET many years back but concluded it wasn't a good fit. My rusty memory of a conversation with Simon Marlow about this was that all the Java libraries basically ended up in IO and matching the API and type design up with Haskell wasn't very nice. Not sure what the motivation is, but you may want to look at VMKit: http://vmkit.llvm.org/. It could be a good fit and may be easy to use through the LLVM backend. I had the impression vmkit is for building new vms. I would like to generate code that runs on the CLR (aka .NET). Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On Jan 28, 2013, at 3:21 AM, Simon Peyton-Jones simo...@microsoft.com wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. Someone should start a wiki page about this! It comes up regularly. (That doesn’t mean it’s a bad idea; just that we should share wisdom on it.) Maybe there is one, in which case perhaps it could be updated? Some thoughts (which, if someone would like to transfer these to the wiki page, perhaps in more coherent form): ·Check out Frege. Maybe others. I think several folk are working on .NET or JVM projects. ·Do you want to inter-operate with .NET, or compile to .NET bytecode. These are very different? I’ll assume the latter. Yes, the latter. ·Do you want to compile to verifiable .NET bytecode? This is hard. GHC’s type system is more expressive than .NET’s in many ways (higher kinded type variables, kind polymorphism...). And less expressive in others (sub-typing). Compiling to verifiable bytecode would require lots of run-time type-tests (which we know will succeed) to reassure .NET that it’s all kosher. Figuring out how to minimise these tests would be a challenge in itself. I'm okay with slow code. That's something I can improve later. :) ·And of course this affects whether you can go via C#, since that requires the program to be well typed in the .NET sense. I'm not sure yet about going via C# or directly to CIL. ·Do you want simply to run Haskell programs on .NET or do you want Haskell programs to call .NET libraries? I assume the latter. But that means that .NET types must be manifested somehow as Haskell types. A big issue is how much of .NET’s type system you want to suck into Haskell. A stressful example: can you define in Haskell a type that sub-classes an imported .NET type. Daan and Mark and Sigbjorn and I wrote papers about this stuff. Look at the papers under Foreign language integration on http://research.microsoft.com/en-us/um/people/simonpj/papers/papers.html. Oh, and this: http://research.microsoft.com/en-us/um/people/simonpj/papers/oo-haskell/index.htm Thanks. I'll make sure to read over those. ·I’d be inclined to start from Core rather than STG, perhaps the output of CorePrep. STG is a bit encrusted (though we’re about to simplify it a bit). But honestly they are pretty similar. Cmm would be a bad choice. It’s far too low level. Good to know. ·Another major issues is that of primops (prelude/primops.txt.pp). Things like add Int# or Float# are fine, but what about forking a thread, or throwing an exception? o One possibility is to try to emulate them all. I have no idea how hard that would be, but GHC’s I/O libraries rely heavily on lightweight concurrency. o Another is to trim the list of PrimOps, and only support some of them. That’s easier, but you’d have to dump a lot of the ‘base’ package (particularly I/O) and re-implement it using the .NET libraries instead. I think I will do whatever gets Fibonacci working first, and then reevaluate. Thanks for the pointers. Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
On Jan 28, 2013, at 8:35 AM, Simon Marlow marlo...@gmail.com wrote: On 28/01/13 01:15, Jason Dagit wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard the LLVM codegen is relatively simple. * Any other advice? Just to put things in perspective a bit, the LLVM backend shares the RTS with the native backend, and uses exactly the same ABI. That limits its scope significantly: it only has to replace the stages between Cmm and assembly code, everything else works as-is. You don't have this luxury with .NET (or JVM), because you can't link .NET or JVM code to native code directly, and these systems already have their own runtimes. Basically you're replacing not only the code generator, but also the runtime, and probably large chunks of the libraries. That's why it's a bigger job. You can't go from Cmm, because as Simon says it's already too low-level. You'll want .NET/JVM to manage the stack for you, and you'll want to have your own compilation scheme for functions and thunks, and so on. The right place to start is after CorePrep, where thunks are explicit (this is where the bytecode generator starts, incidentally: you might want to look at ghci/ByteCodeGen.hs). Will do. Thanks! ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
Someone wrote a Haskell library that could interoperate with .NET - you might get ideas for how to express .NET's type system in Haskell from it: http://www.haskell.org/haskellwiki/Salsa On Mon, Jan 28, 2013 at 8:53 AM, Jason Dagit dag...@gmail.com wrote: On Jan 28, 2013, at 3:21 AM, Simon Peyton-Jones simo...@microsoft.com wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. Someone should start a wiki page about this! It comes up regularly. (That doesn’t mean it’s a bad idea; just that we should share wisdom on it.) Maybe there is one, in which case perhaps it could be updated? Some thoughts (which, if someone would like to transfer these to the wiki page, perhaps in more coherent form): ·Check out Frege. Maybe others. I think several folk are working on .NET or JVM projects. ·Do you want to inter-operate with .NET, or compile to .NET bytecode. These are very different? I’ll assume the latter. Yes, the latter. ·Do you want to compile to verifiable .NET bytecode? This is hard. GHC’s type system is more expressive than .NET’s in many ways (higher kinded type variables, kind polymorphism...). And less expressive in others (sub-typing). Compiling to verifiable bytecode would require lots of run-time type-tests (which we know will succeed) to reassure .NET that it’s all kosher. Figuring out how to minimise these tests would be a challenge in itself. I'm okay with slow code. That's something I can improve later. :) ·And of course this affects whether you can go via C#, since that requires the program to be well typed in the .NET sense. I'm not sure yet about going via C# or directly to CIL. ·Do you want simply to run Haskell programs on .NET or do you want Haskell programs to call .NET libraries? I assume the latter. But that means that .NET types must be manifested somehow as Haskell types. A big issue is how much of .NET’s type system you want to suck into Haskell. A stressful example: can you define in Haskell a type that sub-classes an imported .NET type. Daan and Mark and Sigbjorn and I wrote papers about this stuff. Look at the papers under Foreign language integration on http://research.microsoft.com/en-us/um/people/simonpj/papers/papers.html. Oh, and this: http://research.microsoft.com/en-us/um/people/simonpj/papers/oo-haskell/index.htm Thanks. I'll make sure to read over those. ·I’d be inclined to start from Core rather than STG, perhaps the output of CorePrep. STG is a bit encrusted (though we’re about to simplify it a bit). But honestly they are pretty similar. Cmm would be a bad choice. It’s far too low level. Good to know. ·Another major issues is that of primops (prelude/primops.txt.pp). Things like add Int# or Float# are fine, but what about forking a thread, or throwing an exception? o One possibility is to try to emulate them all. I have no idea how hard that would be, but GHC’s I/O libraries rely heavily on lightweight concurrency. o Another is to trim the list of PrimOps, and only support some of them. That’s easier, but you’d have to dump a lot of the ‘base’ package (particularly I/O) and re-implement it using the .NET libraries instead. I think I will do whatever gets Fibonacci working first, and then reevaluate. Thanks for the pointers. Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Wait, it can? Where's that? On 28 January 2013 02:15, Jason Dagit dag...@gmail.com wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard the LLVM codegen is relatively simple. * Any other advice? Thank you in advance! Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: How to get started with a new backend?
I believe that some Microsoft Research folks at Cambridge got a fairly far along implementation of Haskell on Java or .NET many years back but concluded it wasn't a good fit. My rusty memory of a conversation with Simon Marlow about this was that all the Java libraries basically ended up in IO and matching the API and type design up with Haskell wasn't very nice. Not sure what the motivation is, but you may want to look at VMKit: http://vmkit.llvm.org/. It could be a good fit and may be easy to use through the LLVM backend. On 27 January 2013 17:15, Jason Dagit dag...@gmail.com wrote: I would like to explore making a backend for .NET. I've done a lot of background reading about previous .NET and JVM attempts for Haskell. It seems like several folks have made significant progress in the past and, with the exception of UHC, I can't find any code around the internet from the previous efforts. I realize that in total it's a huge undertaking and codegen is only one of several significant hurdles to success. I would like to get a very, very, very simple translation working inside GHC. If all I can compile and run is fibonacci, then I would be quite happy. For my first attempt, proof of concept is sufficient. I found a lot of good documentation on the ghc trac for how the compilation phases work and what happens in the different parts of the backend. The documentation is excellent, especially compared to other compilers I've looked at. When I started looking at how to write the code, I started to wonder about the least effort path to getting something (anything?) working. Here are some questions: * Haskell.NET seems to be dead. Does anyone know where their code went? * Did lambdavm also disappear? (JVM I know, but close enough to be useful) * Would it make sense to copymodify the -fvia-C backend to generate C#? The trac claims that ghc can compile itself to C so that only standard gnu C tools are needed to build an unregistered compiler. Could I use this trick to translate programs to C#? * What stage in the pipeline should I translate from? Core? STG? Cmm? * Which directories/source files should I look at to get familiar with the code gen? I've heard the LLVM codegen is relatively simple. * Any other advice? Thank you in advance! Jason ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
