Re: [Haskell-cafe] Hot-Swap with Haskell
What would be the semantics of hot-swapping? For, example, somewhere in memory you have a thunk of expression e. Now the user wants to upgrade e to e'. Would you require all thunks to be modified? A similar problem occurs with stack frames. You'd also have to make sure that e and e' have the same (or compatible types). e' most likely has different free variables than e, how can you translate thunk one to the other? Now assuming you don't try to do this, how would you handle the case when something goes wrong? On 16 July 2010 04:05, Andy Stewart lazycat.mana...@gmail.com wrote: Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family Anatidae on our hands. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
Thomas Schilling nomin...@googlemail.com writes: What would be the semantics of hot-swapping? For, example, somewhere in memory you have a thunk of expression e. Now the user wants to upgrade e to e'. Would you require all thunks to be modified? A similar problem occurs with stack frames. You'd also have to make sure that e and e' have the same (or compatible types). e' most likely has different free variables than e, how can you translate thunk one to the other? Now assuming you don't try to do this, how would you handle the case when something goes wrong? Good question. Infact, currently, i just use same technology with Yi/XMonad. Re-compile modules, and save some initial state before re-launch, and do re-execute some command to re-store those state after re-launch. Re-compile will drop old code, and restore state in new code. But re-compile solution not neat way since some state will lost. About your upgrade expression question, i haven no answer, that's why i post this topic to looking for a better solution. I know Emacs use some indirection symbol table for hot-swapping, but it's not safe since it not do type-check. Maybe we can wrap 'Dynamic' check type for safe re-load in runtime, if type match then upgrade, otherwise stop upgrade and keep old state. I just test re-compile technology, not try dynamic linking/load technology that Dons introduce in Plugging Haskell In.pdf since 'plugins' package is broken at the moment. Anyway, i believe this have a neat way to make static typing language can extension dynamic. Now, i'm reading Dons' new paper Dynamic Extension of Typed Functional Languages (http://www.cse.unsw.edu.au/~dons/papers/dons-phd-thesis.pdf) I hope i can find answer after read this paper... Any discuss are welcome! -- Andy On 16 July 2010 04:05, Andy Stewart lazycat.mana...@gmail.com wrote: Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
Hello Andy, 2010/7/16 Andy Stewart lazycat.mana...@gmail.com: There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. For my 2008 GSOC application I wrote a demo app that used hot code reloading and it maintained gtk state just fine. The code (and video) is available at http://paczesiowa.dw.pl/ , it worked in ghc 6.8 and needed hs-plugins, so it won't work now, but the approach should work. regards, Bartek Ćwikłowski ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
Bartek Ćwikłowski paczesi...@gmail.com writes: Hello Andy, 2010/7/16 Andy Stewart lazycat.mana...@gmail.com: There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. For my 2008 GSOC application I wrote a demo app that used hot code reloading and it maintained gtk state just fine. The code (and video) is available at http://paczesiowa.dw.pl/ , it worked in ghc 6.8 and needed hs-plugins, so it won't work now, but the approach should work. Wow, thanks for your hint, i will wait dons fix hs-plugins then try this cool feature.. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
Sorry Andy! CC'ing to the rest of -cafe in case anybody notices (I need to stop haskelling so early in the morning...) On Fri, Jul 16, 2010 at 8:59 AM, austin seipp a...@0xff.ath.cx wrote: You also may like one project I wrote, an IRC bot that used hs-plugins to do hot code reloading (only works on GHC 6.8.) Inspired by lambdabot, it could swap the entire running program as well as its own plugins, on the fly and maintain state pretty successfully. I haven't touched the source code in a long time - it was one of my first bigger Haskell projects after reading Dons paper about yi. I tried a bunch of things with various versions of it. If I wrote the code again today (providing plugins worked) it would likely be 1/10th the size and more manageable as I have become much better at Haskell, but oh well. Looking over it now though, most of the code is pretty simple. :) http://code.haskell.org/infinity/src/ On Fri, Jul 16, 2010 at 8:04 AM, Andy Stewart lazycat.mana...@gmail.com wrote: Bartek Ćwikłowski paczesi...@gmail.com writes: Hello Andy, 2010/7/16 Andy Stewart lazycat.mana...@gmail.com: There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. For my 2008 GSOC application I wrote a demo app that used hot code reloading and it maintained gtk state just fine. The code (and video) is available at http://paczesiowa.dw.pl/ , it worked in ghc 6.8 and needed hs-plugins, so it won't work now, but the approach should work. Wow, thanks for your hint, i will wait dons fix hs-plugins then try this cool feature.. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- - Austin -- - Austin ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
Generally, in Erlang or Haskell, the semantics we use is to keep all the old code in memory, for the case of closures and thunks that point back into that code. You can imagine a fine-grained semantics where as each top level function is no longer referenced, the *code* for that is swapped. I believe there have been versions of Erlang bytecode hotswapping that supported that. In GHC Haskell, we have component-level hotswapping -- only whole-modules may be swapped out at the moment. As we can't unload code based on the GC dropping references, we just keep it all in memory, migrating to new code as *new* (dynamic) bindings are introduced. The type migration issue is yet another problem. There are several approaches to migrating state types, including serialization (as xmonad et al do), or using open data types. -- Don nominolo: What would be the semantics of hot-swapping? For, example, somewhere in memory you have a thunk of expression e. Now the user wants to upgrade e to e'. Would you require all thunks to be modified? A similar problem occurs with stack frames. You'd also have to make sure that e and e' have the same (or compatible types). e' most likely has different free variables than e, how can you translate thunk one to the other? Now assuming you don't try to do this, how would you handle the case when something goes wrong? On 16 July 2010 04:05, Andy Stewart lazycat.mana...@gmail.com wrote: Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family Anatidae on our hands. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
On Fri, Jul 16, 2010 at 11:21 AM, Don Stewart d...@galois.com wrote: Generally, in Erlang or Haskell, the semantics we use is to keep all the old code in memory, for the case of closures and thunks that point back into that code. You can imagine a fine-grained semantics where as each top level function is no longer referenced, the *code* for that is swapped. I believe there have been versions of Erlang bytecode hotswapping that supported that. In GHC Haskell, we have component-level hotswapping -- only whole-modules may be swapped out at the moment. As we can't unload code based on the GC dropping references, we just keep it all in memory, migrating to new code as *new* (dynamic) bindings are introduced. The type migration issue is yet another problem. There are several approaches to migrating state types, including serialization (as xmonad et al do), or using open data types. Is hs-plugins working? I thought it was bit-rotted and needed reworking to function with modern ghc? I'm using the xmonad style static config, but I don't think it's very practical for a reason Andy didn't mention: linking is slow when a program gets big, so every little change takes a long time. xmonad gets away with it because it's tiny. I use hint to interpret code dynamically, but that makes the program even bigger and linking even longer, so it seems like to have my cake and eat it too I would need to be able to recompile a single module, and then dynamically link it into the running application. In my case, I don't think I need to worry about old closures or code migration because I would just be loading new functions to splice into an event loop that can only return data types defined in the main app. So it would be acceptable to swap a module by removing all the functions loaded from it, unloading and reloading the module, and splicing in the new list of functions it exports. I'm not sure how I could prove to a dynamic loading system that I'm holding no references to its code anymore and it can just unload it, but I suppose it would be the same as a C plugin system: if you return pointers to plugin code and stash them in the host, you will be in trouble when the module gets unloaded. I suppose that just means you have to rnf all the data coming out of the plugin before you unload it, right? Is this the sort of thing hs-plugins can do? The other thing I thought of was to use xmonad-style static config, but speed up the link by linking the larger parts dynamically. Isn't ghc supposed to support dynamic linking now? Or would it be just as slow because the time subtracted from the link phase would simply be added to the load phase? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
lazycat, You may find the following link useful, as it talks about much the same approach. http://nathanwiegand.com/wp/2010/02/hot-swapping-binaries/ Sadly, any hot-swap mechanism is going to suffer from the potential loss of state where that state is not controlled by your code. When that loss of state isn't acceptable you have a few mechanisms available. By far the easiest, I think, in the gtk2hs scenario you mention is that you could refactor all of the code that interoperated with gtk2hs into another process and spawn it, communicating with it over a pipe or other communication mechanism. That way hot-swapping the current process would leave your GUI (and the pipe/IPC mechanism) intact. You may then need to pass along whatever changes affect the gui over the pipe in a fairly manual fashion. -Edward Kmett On Thu, Jul 15, 2010 at 11:05 PM, Andy Stewart lazycat.mana...@gmail.comwrote: Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Hot-Swap with Haskell
Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Thanks, -- Andy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Hot-Swap with Haskell
lazycat.manatee: Hi all, I'm research to build a hot-swap Haskell program to developing itself in Runtime, like Emacs. Essentially, Yi/Xmonad/dyre solution is replace currently executing technology: re-compile new code with new binary entry when re-compile success $ do save state before re-launch new entry replace current entry with new binary entry (executeFile) store state after re-launch new entry There are some problems with re-compile solution: 1) You can't save *all* state with some FFI code, such as gtk2hs, you can't save state of GTK+ widget. You will lost some state after re-launch new entry. 2) Sometimes re-execute is un-acceptable, example, you running some command in temrinal before you re-compile, you need re-execute command to restore state after re-launch, in this situation re-execute command is un-acceptable. I wonder have a better way that hot-swapping new code without re-compile/reboot. Well, the other approach to reloadable modules, using either object code plugins, or bytecode plugins, giving you module-level granularity. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
