Re: Recompiling D code
On 16/08/2014 11:29 a.m., Vladimir Panteleev wrote: On Friday, 15 August 2014 at 06:54:28 UTC, Rikki Cattermole wrote: On 15/08/2014 2:58 a.m., Vladimir Panteleev wrote: How many D modules / object files is that? I haven't gone into that, I don't really want to go around modifying dub if I can help it. Well, to be blunt, I doubt you'll get satisfactory results unless you're ready to get down to the nitty-gritty. Unfortunately I agree with you. But from what I've heard this isn't bad results considering native language and all. I may work towards using something like parrotVM or lua. Could be interesting for dev vs production. Don't see why I can't get that working rather well.
Re: Recompiling D code
On Friday, 15 August 2014 at 06:54:28 UTC, Rikki Cattermole wrote: On 15/08/2014 2:58 a.m., Vladimir Panteleev wrote: How many D modules / object files is that? I haven't gone into that, I don't really want to go around modifying dub if I can help it. Well, to be blunt, I doubt you'll get satisfactory results unless you're ready to get down to the nitty-gritty.
Re: Recompiling D code
On 15/08/2014 6:06 a.m., Jacob Carlborg wrote: On 2014-08-14 13:54, Rikki Cattermole wrote: Hello everyone. As you all may know I've been working on recompiling D for web services last few weeks. Its both good news and bad news. Good: Reloading definitely possible. With dependency handling using dub. Bad: Its slow. And not in my code sort of way. Has anyone thought of trying to JIT using LDC. Supposedly the llvm is slower for generation then dmd's backend. Atleast what was said during the SDC DCONF talk. But I'm all for this, could also use it to cache code that is generated ext. in memory. Less usage on the hdd can only be a good thing. I'll leave this to somebody who does know this area of things.
Re: Recompiling D code
On 15/08/2014 2:58 a.m., Vladimir Panteleev wrote: On Thursday, 14 August 2014 at 12:56:10 UTC, Rikki Cattermole wrote: Dub automatically handles caching of dependencies such as vibe-d. So they are not rebuilt. The only things that get recompiled for example is a single code unit. This is defined as being the dependency between a route file and template files (simplified). How many D modules / object files is that? I haven't gone into that, I don't really want to go around modifying dub if I can help it. The necessary dependencies such as vibe-d then should be added as part of the build process. Incremental builds of things such as routes and templates are not a good idea. Same goes for data models. It could change the code unit output a little too much. Possibly even cause a corrupt binary from missing symbols ext. That approach sounds much more high-level than what I meant, and I don't see why you'd need to abstract things to that level. I'm not familiar with how dub builds things, but inter-module dependencies are a solved problem, thanks to DMD's -v and -deps switches. Incremental compilation was implemented in older (D1-era) build tools, such as xfbuild, however has never picked up due to DMD bugs with incrementally compiling more than one object file at a time (which might very well be fixed by now).
Re: Recompiling D code
On 15/08/2014 3:09 a.m., Justin Whear wrote: On Thu, 14 Aug 2014 14:58:20 +, John Colvin wrote: My experience with these sort of things suggests that it'll be the linker taking the time. Dynamic libraries are the solution. Dub needs proper support for dynamic library dependencies. I'll second that suggestion. Can you run the timings using -c? Obviously you won't get binaries out the end, but it'd help diagnose the problem. Yes its definitely a lot faster. Around 600ms - 800ms. Even with other applications running.
Re: Recompiling D code
On 2014-08-14 13:54, Rikki Cattermole wrote: Hello everyone. As you all may know I've been working on recompiling D for web services last few weeks. Its both good news and bad news. Good: Reloading definitely possible. With dependency handling using dub. Bad: Its slow. And not in my code sort of way. Has anyone thought of trying to JIT using LDC. -- /Jacob Carlborg
Re: Recompiling D code
On Thu, 14 Aug 2014 14:58:20 +, John Colvin wrote: > My experience with these sort of things suggests that it'll be the > linker taking the time. Dynamic libraries are the solution. > > Dub needs proper support for dynamic library dependencies. I'll second that suggestion. Can you run the timings using -c? Obviously you won't get binaries out the end, but it'd help diagnose the problem.
Re: Recompiling D code
On Thursday, 14 August 2014 at 12:56:10 UTC, Rikki Cattermole wrote: Dub automatically handles caching of dependencies such as vibe-d. So they are not rebuilt. The only things that get recompiled for example is a single code unit. This is defined as being the dependency between a route file and template files (simplified). How many D modules / object files is that? The necessary dependencies such as vibe-d then should be added as part of the build process. Incremental builds of things such as routes and templates are not a good idea. Same goes for data models. It could change the code unit output a little too much. Possibly even cause a corrupt binary from missing symbols ext. That approach sounds much more high-level than what I meant, and I don't see why you'd need to abstract things to that level. I'm not familiar with how dub builds things, but inter-module dependencies are a solved problem, thanks to DMD's -v and -deps switches. Incremental compilation was implemented in older (D1-era) build tools, such as xfbuild, however has never picked up due to DMD bugs with incrementally compiling more than one object file at a time (which might very well be fixed by now).
Re: Recompiling D code
On Thursday, 14 August 2014 at 12:56:10 UTC, Rikki Cattermole wrote: On 15/08/2014 12:47 a.m., Vladimir Panteleev wrote: On Thursday, 14 August 2014 at 11:54:57 UTC, Rikki Cattermole wrote: Atleast from my experience with this, its dmd thats actually taking the time. I can't glean this from looking at the code, but are you recompiling the entire program? Web development is a perfect fit for incremental compilation, so when changing one web page's template or view code, you should need to compile only one .d file. ld and gold seem to support incremental linking, that could help speed things up as well. Dub automatically handles caching of dependencies such as vibe-d. So they are not rebuilt. The only things that get recompiled for example is a single code unit. This is defined as being the dependency between a route file and template files (simplified). The necessary dependencies such as vibe-d then should be added as part of the build process. Incremental builds of things such as routes and templates are not a good idea. Same goes for data models. It could change the code unit output a little too much. Possibly even cause a corrupt binary from missing symbols ext. But in this case none of this matters much. Pretty much I'm only testing against a single route module. No dependencies outside phobos and vibe-d. My experience with these sort of things suggests that it'll be the linker taking the time. Dynamic libraries are the solution. Dub needs proper support for dynamic library dependencies.
Re: Recompiling D code
On 15/08/2014 12:47 a.m., Vladimir Panteleev wrote: On Thursday, 14 August 2014 at 11:54:57 UTC, Rikki Cattermole wrote: Atleast from my experience with this, its dmd thats actually taking the time. I can't glean this from looking at the code, but are you recompiling the entire program? Web development is a perfect fit for incremental compilation, so when changing one web page's template or view code, you should need to compile only one .d file. ld and gold seem to support incremental linking, that could help speed things up as well. Dub automatically handles caching of dependencies such as vibe-d. So they are not rebuilt. The only things that get recompiled for example is a single code unit. This is defined as being the dependency between a route file and template files (simplified). The necessary dependencies such as vibe-d then should be added as part of the build process. Incremental builds of things such as routes and templates are not a good idea. Same goes for data models. It could change the code unit output a little too much. Possibly even cause a corrupt binary from missing symbols ext. But in this case none of this matters much. Pretty much I'm only testing against a single route module. No dependencies outside phobos and vibe-d.
Re: Recompiling D code
On Thursday, 14 August 2014 at 11:54:57 UTC, Rikki Cattermole wrote: Atleast from my experience with this, its dmd thats actually taking the time. I can't glean this from looking at the code, but are you recompiling the entire program? Web development is a perfect fit for incremental compilation, so when changing one web page's template or view code, you should need to compile only one .d file. ld and gold seem to support incremental linking, that could help speed things up as well.
Recompiling D code
Hello everyone. As you all may know I've been working on recompiling D for web services last few weeks. Its both good news and bad news. Good: Reloading definitely possible. With dependency handling using dub. Bad: Its slow. And not in my code sort of way. You're welcome to atest my speeds that I have gotten so far (older well used hdd, Win7 x64), its around 2-3 seconds for a basic go download webpage and not much more. I've done a bit of working with it and with 2.066-rc2 got it to the lower end of 2s. Atleast from my experience with this, its dmd thats actually taking the time. I know my code isn't the best and optimized, so I have compared mine against dmd itself. 100-200ms difference normally. For those who don't know why I'm making a fuss over 2 seconds, its because you actually do notice this when doing web dev stuff. Without including start up time, this could become a lot larger. [0] https://github.com/rikkimax/livereload
