Re: Five Projects Selected for SAOC 2019
On Wednesday, 4 September 2019 at 23:56:07 UTC, Laeeth Isharc wrote: On Tuesday, 27 August 2019 at 17:32:30 UTC, Max Haughton wrote: On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: [...] I was aware of the site when i wrote the proposal, but the idea is to create the infrastructure to add more measurements too, e.g. profiling the compiler or testing it under limited memory (I found out how much memory my CTFE thing was using the other day!). Assuming I can get it to work I'd also like to throw the Linux perf system in there too, Take a look at BPF. Might be some work to wrap and if I recall right some of the C headers are a bit gnarly. But it's pretty powerful. https://github.com/brendangregg/bpf-docs I'll look into it, thank you. Seems like a big rabbit hole to go down but the project is far from set in stone.
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 17:32:30 UTC, Max Haughton wrote: On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Sorry, I haven't been following. Don't we already have an implementation of the "Create a CI or other infrastructure for measuring D’s progress and performance" project? I just haven't been maintaining it because there hasn't been a lot of interest in it while it was being maintained. Here's the original blog post: https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ I'll give it a kick and get it back online if there is interest. Seems wasteful to reimplement it from scratch, though. I was aware of the site when i wrote the proposal, but the idea is to create the infrastructure to add more measurements too, e.g. profiling the compiler or testing it under limited memory (I found out how much memory my CTFE thing was using the other day!). Assuming I can get it to work I'd also like to throw the Linux perf system in there too, Take a look at BPF. Might be some work to wrap and if I recall right some of the C headers are a bit gnarly. But it's pretty powerful. https://github.com/brendangregg/bpf-docs
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 03:55:43 UTC, Andrej Mitrovic wrote: On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: ... Solve Dependency Hell: This is considered as a crucial first step in making Phobos available via the DUB registry I'm guessing this means we might even be able to use multiple versions of Phobos one day. However before we do that, we will really need to fix the use of globals in Phobos. Honestly, I'm a bit leery about this use-multiple-versions approach. It feels like it carries some serious risks of code bloat, which may be accentuated by the fact that DUB has some serious dependency resolution bugs even with single version dependencies... :-\
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 17:11:33 UTC, Mike Franklin wrote: On Tuesday, 27 August 2019 at 12:58:20 UTC, Vladimir Panteleev wrote: It will eventually zero in to commit-level accuracy after it's been running for a while. I cleared the database as the last time it was running, it was on another CPU, so the timings are going to be different. (Still need to decide on a way to measure execution time in some deterministic way.) If you look at the vibe.d compile-time graph, you'll see there's a 2.5s increase around Mid-2014. When I zoom in and visit the commit, it's just a DDoc comment change (https://github.com/dlang/phobos/pull/3542). I don't see how that could account for the large increase in compile time. The JS part of the project was adopted from somewhere else (Mozilla's project for tracking Firefox performance), so it was a bit hammered-on to the data schema we use here. Mozilla never builds individual successive commits; you have to do it yourself to zero in on a specific change. Because of this, the UI was confusing and it wasn't obvious whether a commit is actually the cause of a change in the metrics. I've now changed it to only draw a line between two points if they're adjacent in the git history. Still not perfect but better. You can see this now by zooming into the big change in e.g. "Empty program - compilation - peak heap size". (The vibe.d one isn't resolved yet, give it a few days.) The problem with representing this data is that we have the whole commit list, which is nice and linear, but not all commits are tested, and we can't graph untested commits because they effectively don't have an Y value. There's also way too many commits to draw at once, so it groups commits that occurred near the same time together in one node, which again causes complications when it's a mix of tested and untested commits. (Currently it uses the median of the test results for tested commits.) More changes: https://github.com/CyberShadow/TrenD/commits/master
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 11:03:51 UTC, Jacob Carlborg wrote: On 2019-08-26 05:55, Andrej Mitrovic wrote: I'm guessing this means we might even be able to use multiple versions of Phobos one day. However before we do that, we will really need to fix the use of globals in Phobos. I don't think that's necessary. All symbols will have the version be part of the mangled name. Hm... this approach has a few scenarios where I imagine that it fails in weird ways: - resources that can be initialized only once that are encapsulated in a library: e.g. two versions of a logging library that write to the same file and recreate the file during initialization. On Windows, opening the same file for writing again might fail. On Linux, it would replace the existing inode entry for the file while both library instances continue to happily write to their instance of the file. - violating implicit expectations about relationships between dependencies; for example, consider two libraries libFoo and libBar working on data structures defined in a libCore. An application uses all three of them and passes pointers to libCore data structures to functions in libFoo and libBar. Recently, libFoo and libCore have been updated and the libCore data structures changed. The libBar update is not done yet. So if libBar is built with its own private copy of the old libCore, expectations about structure memory layouts are out of whack. If something like that compiles and links, it's likely going to be nearly impossible to debug. The thing is that the resulting failure cases are likely surprising, not obvious and therefore hard to debug. If you want to do this, it should be optional and require explicit opt-in from all dependencies in a chain. Libraries have to be designed in very specific ways to make this work and these restrictions would be new to the D ecosystem.
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 17:11:33 UTC, Mike Franklin wrote: If you look at the vibe.d compile-time graph, you'll see there's a 2.5s increase around Mid-2014. Sorry, that should be Mid-2015.
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Sorry, I haven't been following. Don't we already have an implementation of the "Create a CI or other infrastructure for measuring D’s progress and performance" project? I just haven't been maintaining it because there hasn't been a lot of interest in it while it was being maintained. Here's the original blog post: https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ I'll give it a kick and get it back online if there is interest. Seems wasteful to reimplement it from scratch, though. I was aware of the site when i wrote the proposal, but the idea is to create the infrastructure to add more measurements too, e.g. profiling the compiler or testing it under limited memory (I found out how much memory my CTFE thing was using the other day!). Assuming I can get it to work I'd also like to throw the Linux perf system in there too, but As mentioned in the summary, this will extend to runtime performance too but that obviously raises the question of how to ensure consistent performance of the testing machine (or give a relative figure).
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 14:42:28 UTC, Vladimir Panteleev wrote: On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Sorry, I haven't been following. Don't we already have an implementation of the "Create a CI or other infrastructure for measuring D’s progress and performance" project? I just haven't been maintaining it because there hasn't been a lot of interest in it while it was being maintained. Max, if you're still excited about this project, please ping me on IRC or Slack. Would love to brainstorm with you and hear your ideas. Vladimir, I would love to (and I need to acquire a mentor, so if you're not busy... ;) ) but I am struggling to even load the forum (I'm on holiday) but I'll be back to civilisation within a day or two. To anyone else reading: If the gods allow it, I'm *trying* to answer questions about my proposal but my internet connection is genuinely shit ATM
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 12:58:20 UTC, Vladimir Panteleev wrote: It will eventually zero in to commit-level accuracy after it's been running for a while. I cleared the database as the last time it was running, it was on another CPU, so the timings are going to be different. (Still need to decide on a way to measure execution time in some deterministic way.) If you look at the vibe.d compile-time graph, you'll see there's a 2.5s increase around Mid-2014. When I zoom in and visit the commit, it's just a DDoc comment change (https://github.com/dlang/phobos/pull/3542). I don't see how that could account for the large increase in compile time. Mike
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Sorry, I haven't been following. Don't we already have an implementation of the "Create a CI or other infrastructure for measuring D’s progress and performance" project? I just haven't been maintaining it because there hasn't been a lot of interest in it while it was being maintained. Max, if you're still excited about this project, please ping me on IRC or Slack. Would love to brainstorm with you and hear your ideas.
Re: Five Projects Selected for SAOC 2019
On Tuesday, 27 August 2019 at 09:08:58 UTC, Mike Franklin wrote: It's great to see this back up and running. The compile-time data is quite interesting. Is there any way to identify a particular offending commit. The commits identified in the data points on the chart don't seem to be precise. It will eventually zero in to commit-level accuracy after it's been running for a while. I cleared the database as the last time it was running, it was on another CPU, so the timings are going to be different. (Still need to decide on a way to measure execution time in some deterministic way.)
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 18:51:54 UTC, Vladimir Panteleev wrote: Here's the original blog post: https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ I'll give it a kick and get it back online if there is interest. Seems wasteful to reimplement it from scratch, though. It's great to see this back up and running. The compile-time data is quite interesting. Is there any way to identify a particular offending commit. The commits identified in the data points on the chart don't seem to be precise. Mike
Re: Five Projects Selected for SAOC 2019
On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Sorry, I haven't been following. Don't we already have an implementation of the "Create a CI or other infrastructure for measuring D’s progress and performance" project? I just haven't been maintaining it because there hasn't been a lot of interest in it while it was being maintained. Here's the original blog post: https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ I'll give it a kick and get it back online if there is interest. Seems wasteful to reimplement it from scratch, though.
Re: Five Projects Selected for SAOC 2019
On Monday, 26 August 2019 at 11:03:51 UTC, Jacob Carlborg wrote: On 2019-08-26 05:55, Andrej Mitrovic wrote: I'm guessing this means we might even be able to use multiple versions of Phobos one day. However before we do that, we will really need to fix the use of globals in Phobos. I don't think that's necessary. All symbols will have the version be part of the mangled name. so, the target of project is to make mangled name for using by app 2+ version of one lib/module? cuz in that case u cannot fix situation when libV1.obj will land to libV2 somehow (through globals, AA.. ) and binary implementation is different. probably will be panic, data loss and other bad things.
Re: Five Projects Selected for SAOC 2019
On 2019-08-26 05:55, Andrej Mitrovic wrote: I'm guessing this means we might even be able to use multiple versions of Phobos one day. However before we do that, we will really need to fix the use of globals in Phobos. I don't think that's necessary. All symbols will have the version be part of the mangled name. -- /Jacob Carlborg
Re: Five Projects Selected for SAOC 2019
On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Reddit: https://www.reddit.com/r/d_language/comments/cv8jtd/saoc_2019_projects_and_participants/ Congratulations to the selected projects, and Good luck!
Re: Five Projects Selected for SAOC 2019
On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: ... Solve Dependency Hell: This is considered as a crucial first step in making Phobos available via the DUB registry I'm guessing this means we might even be able to use multiple versions of Phobos one day. However before we do that, we will really need to fix the use of globals in Phobos.
Re: Five Projects Selected for SAOC 2019
On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Reddit: https://www.reddit.com/r/d_language/comments/cv8jtd/saoc_2019_projects_and_participants/ Kudos to all 8 for trying! The project seem interesting. I'd like to focus on the MLIR project since I have been looking MLIR for the last month. I belive this a project with big potential if MLIR succeeds. Roberto congratulations! I don't know how much experience you have with compilers or LDC. I'm sure Nicholas will be there to help you but here are some suggestions. There are a bunch of talks that I feel like focus more on the goals of MLIR and not the practical side (the goals are important, don't get me wrong). Aside from the general README etc., here's one tutorial I like [1]. I think it's a good introduction. This talk [2] is mostly good too. For LDC, there's of course the dlang forum LDC group, but also don't forget the Gitter: http://gitter.im/ldc-developers/main. This is the main discussion thread for LDC. Don't hesitate to ask questions there! [1] https://github.com/tensorflow/mlir/blob/master/g3doc/Tutorials/Toy/Ch-1.md [2] https://www.youtube.com/watch?v=cyICUIZ56wQ
Re: Five Projects Selected for SAOC 2019
On Sunday, 25 August 2019 at 13:38:24 UTC, Mike Parker wrote: The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Reddit: https://www.reddit.com/r/d_language/comments/cv8jtd/saoc_2019_projects_and_participants/ I need to apologize to Laeeth and Manu. I mistakenly attributed the stl-containers repository to Suleyman: https://github.com/dlang-cpp-interop/stl-containers/ That was a major oversight on my part.
Five Projects Selected for SAOC 2019
The Symmetry Autumn of Code 2019 application selection process has come to an end. This year, we've got five projects instead of three. Congratulations to everyone who was selected! You can read about them and their projects over at the D Blog: https://dlang.org/blog/2019/08/25/saoc-2019-projects-and-participants/ Reddit: https://www.reddit.com/r/d_language/comments/cv8jtd/saoc_2019_projects_and_participants/
