Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 01:03:39 UTC, Ilya Yaroshenko wrote: ldmd2/ldc2 flag -mcpu=native will optimise code for your CPU. -- Ilya Hmm, ldc seems to segfault when passing that flag. I will make a bug report for this.
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 21:39:54 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 18:08:52 UTC, Andrei Alexandrescu wrote: On 12/29/2015 11:28 AM, Robert burner Schadek wrote: On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library. Hopefully this is something that you or someone else could help by creating pull requests. Any volunteers? -- Andrei Does it means that the PR can be merged? --Ilya If there's a time constraint, perhaps we could merge it for 2.070 but keep adding documentation updates to both master and release branches?
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 22:46:28 UTC, John Colvin wrote: On Wednesday, 30 December 2015 at 21:39:54 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 18:08:52 UTC, Andrei Alexandrescu wrote: On 12/29/2015 11:28 AM, Robert burner Schadek wrote: [...] Hopefully this is something that you or someone else could help by creating pull requests. Any volunteers? -- Andrei Does it means that the PR can be merged? --Ilya If there's a time constraint, perhaps we could merge it for 2.070 but keep adding documentation updates to both master and release branches? 2.070 would be merged into stable soon.
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 18:08:52 UTC, Andrei Alexandrescu wrote: On 12/29/2015 11:28 AM, Robert burner Schadek wrote: On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library. Hopefully this is something that you or someone else could help by creating pull requests. Any volunteers? -- Andrei Does it means that the PR can be merged? --Ilya
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 14:42:43 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. Whooohooo! Thanks! Ilya I'm writing a blog post giving an overview of std.ndslice and comparing and contrasting to Numpy, to be released when this is merged, and I was wondering if I can have your permission to include your great image processing example in the post?
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 16:28:52 UTC, Robert burner Schadek wrote: On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library. OK, looks like I understand you. My style of exploration is set of examples. For some people it is much better. But I agree that it may be bad for others. About `iteration` module: 1. from documentation: -- Operators only change strides and lengths of a slice. The range of a slice remains unmodified. All operators return slice of the same type as the type of the argument. -- 2. They are very simple functions, and their combination is simple too. A user will understand how to use them. Furthermore `Binary representation` section in `Slice` docs has examples with `transposed` and `reversed`. So user can understand what exactly going on with iteration functions. About selection module: 1. from documentation: -- Selectors create new views and iteration patterns over the same data, without copying. Subspace selectors - - - Subspace selectors serve to generalize and combine other selectors easily. For a slice of Slice!(N, Range) type slice.pack!K creates a slice of slices of Slice!(N-K, Slice!(K+1, Range)) type by packing the last K dimensions of the top dimension pack, and the type of element of slice.byElement is Slice!(K, Range). Another way to use pack is transposition of dimension packs using evertPack. Examples of use of subspace selectors are available for selectors, Slice.shape , and Slice.elementsCount . -- 2. Selectors look simple, but their combinations can be _very_ crazy. Combination of this functions is art of imagination. I am not so talented writer to create good documentation. The problem here is that I have never discuss this module with significant amount of people. And I don't think that I will have such opportunity. So what I have done with documentation for selectors? I created cross examples: `unpack` has examples with - `pack` `evertPack` has examples with - `pack` - `unpack` `diagonal` has examples with - `dropOne` - `dropToHypercube` - `reversed` - `pack` - `evertPack` - Note about additional examples with `blocks` and `windows` `blocks` has examples with - `byElement` - `diagonal` - `pack` - `evertPack` - `unpack` `windows` has examples with - `byElement` - `diagonal` - `pack` - `evertPack` - `unpack` `byElement` has examples with - `pack` - `drop` - `std.range.retro` - `allReversed` `byElementInStandardSimplex` has examples with - `allReversed ` - `transposed` `indexSlice` has examples with - `byElement` In addition, example `Image Processing` in `package` description is a real world combination of `sliced`, `pack`, `windows`, `unpack`, `byElement`, `std.algorithm.iteration.map`, `std.array.array`, `Slice.shape`. This is a way how I can write documentation. I don't think that I can write using style you want or find someone who can do it instead of me. I am already spend a lot of time with English translator to improve docs. And every new sentence is a real pain, because I need to ask my translator to check it. Ilya
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 18:08:52 UTC, Andrei Alexandrescu wrote: On 12/29/2015 11:28 AM, Robert burner Schadek wrote: On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library. Hopefully this is something that you or someone else could help by creating pull requests. Any volunteers? -- Andrei Collection of posts like Jack Stouffer is writing will bring an understanding of how to improve docs. Currently we can collect links of such posts for a while and store them on package description page, thought. YouTube has 7.5K videos for "numpy" query. Ilya
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: On Monday, 28 December 2015 at 14:42:43 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. Whooohooo! Thanks! Ilya I'm writing a blog post giving an overview of std.ndslice and comparing and contrasting to Numpy, to be released when this is merged, and I was wondering if I can have your permission to include your great image processing example in the post? Thank you for doing this! Yes, you can get any examples! `movingWindowByChannel` is a good function, however small function `median` can be improved, but it is not easy to do it for small filter sizes. You may want to use convolution matrices such as the Sobel operator or Gaussian blur instead of `median`. They will work faster comparing with `median`. Another option is a note that `median` can be significantly optimised. ^^ Many thanks! Ilya
Re: Voting For std.experimental.ndslice
On 12/29/2015 11:28 AM, Robert burner Schadek wrote: On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library. Hopefully this is something that you or someone else could help by creating pull requests. Any volunteers? -- Andrei
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 15:42:32 UTC, Robert burner Schadek wrote: On Monday, 28 December 2015 at 22:39:45 UTC, Ilya Yaroshenko wrote: 1. First paragraph was replaced by your variant. 2. Binary representation was moved to Slice type documentation. 3. Small "Quick Start" was added, so new user will start from `sliced` and `Slice`. That is just patchwork ... The style of writing I presented should be used throughout the documentation so people know what everything is and how it works together. OK, lets discuss every function. Fist is `transposed`: --- N-dimensional transpose operator. Brings selected dimensions to the first position. Parameters: Slice!(N, Range) slice input slice Dimensions indexes of dimensions to be brought to the first position Returns: n-dimensional slice of the same type See Also: swapped, everted What is wrong with this documentation?
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 16:11:00 UTC, Ilya Yaroshenko wrote: OK, lets discuss every function. That is acceptably the problem. It is not about the documentation of the functions it is about the documentation binding the functions together and the documentation giving the idea of the library.
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 14:51:37 UTC, Márcio Martins wrote: On Tuesday, 29 December 2015 at 13:54:56 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 13:30:47 UTC, Márcio Martins wrote: If I am reading the code right, the number of dimensions must always be known at compile time, right? Not necessarily. The dimensions for this slice are computed at runtime. ``` auto slice = 1000.iota.sliced(5, 6, 7); ``` ``` With this declaration: ``` struct Slice(size_t _N, _Range) ``` _N, the number of dimensions, must necessarily be known at compile-time, right? Yes. -- Ilya
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 13:30:47 UTC, Márcio Martins wrote: On Tuesday, 29 December 2015 at 07:45:14 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 22:39:45 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 21:43:35 UTC, Robert burner Schadek wrote: [...] Agreed 1. First paragraph was replaced by your variant. 2. Binary representation was moved to Slice type documentation. 3. Small "Quick Start" was added, so new user will start from `sliced` and `Slice`. http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-081dd6e9e8b3810a143d0a5fcba8d60b/web/phobos-prerelease/std_experimental_ndslice.html Thanks! Ilya Fix English: http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-bfed2500425eb407bf2c97fc72e8a0c5/web/phobos-prerelease/std_experimental_ndslice.html If I am reading the code right, the number of dimensions must always be known at compile time, right? Is so, I suppose it was a performance-oriented decision and not an oversight, correct? This package is performance oriented. In the same time it is more flexible comparing with numpy. Simple optimisation check: Source: https://gist.github.com/9il/bc7966823d96557c566c LDC disassembled: https://gist.github.com/9il/47aea1621a9fba609869 (all functions are inlined)
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 21:19:19 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: ... First draft: http://jackstouffer.com/hidden/nd_slice.html Please critique. Looks solid. Good work!
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 21:19:19 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: On Monday, 28 December 2015 at 14:42:43 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. Whooohooo! Thanks! Ilya I'm writing a blog post giving an overview of std.ndslice and comparing and contrasting to Numpy, to be released when this is merged, and I was wondering if I can have your permission to include your great image processing example in the post? Thank you for doing this! First draft: http://jackstouffer.com/hidden/nd_slice.html Please critique. Awesome! Please find my notes below. The Basics == About `iota.front -> slice.font -> user accessing the data`: 1. From the `sliced` documentation for `range` parameter: "only index operator auto `opIndex(size_t index)` is required for ranges". See also recently added the last two examples for sliced http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-387174b023f8cb9612cfcddc76788896/web/phobos-prerelease/std_experimental_ndslice_slice.html#.sliced The first one is for Input Range primitives, the second one is for Random Access Range primitives. 2. Consequently `Slice` never invokes `front`, `popFront`, `empty`, and others range primitives. `Slice` invokes only `opIndex(size_t)`, and if `save` is defined, `slice.save` invokes `range.save`. In addition, `sliced` can be used like Finite Random Access Range Constructor: user may define only `opIndex(size_t)` and `save` to make a Finite Random Access Range by calling `sliced`. 3. If type of slice is `Slice!(3, Range)`, then `slice.front` do not invokes `range[indexToFront]`, it returns a slice type of `Slice!(2, Range)`. In addition, `indexToFront` may not be zero, because primitives like `popFront` or operators like `reversed` may be called before. For slices over arrays `indexToFront` is not defined because the pointer always refers to the first position: the pointer is moved when the slice changes. For more details see also examples for Internal Binary Representation http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-387174b023f8cb9612cfcddc76788896/web/phobos-prerelease/std_experimental_ndslice_slice.html#.Slice Getting Hands On == - Please use "~>0.8.7" instead of "~>0.8.0". "~>0.8.7" has LDC 0.17.0 alpha1 support. Examples == - A Basic Example With A Benchmark is not honest (difference 76 is very large). To make it honest: 1. Larger 100x1000 matrix can be used instead of 10x10, because numpy have a significant initialization overhead. 2. Both `100_000.iota.sliced(100, 1000)` and `100_000.iota.array.sliced(100, 1000)` can be tested. The last one is with `std.array.array`. Memory access is expensive. 3. Only iteration should be tested. Allocation and initilization should be separated from iteration both in D and Python. I expect smaller differnce in perfomance than 76 times. - You may want to test both DMD and LDC 0.17.0 alpha1 for bechmarks https://github.com/ldc-developers/ldc/releases/tag/v0.17.0-alpha1 LDC 0.17.0 alpha1 works well with dip80-ndslice v0.8.7. - Python users love small code and they can be afraid to see large (template constraints) `mean` funciton. An example with lambda function can be added: --- auto means = 100_000.iota.sliced(100, 1).rotated(3).map!(r => sum(r) / r.length); --- One line! - Nitpick: `transposed` (without params for 2D slice) is more clear than `rotated(3)`, IMO - `dub --build=release`can be added where you have noted dmd optimisation flags. Numpy's Main Problem, and How D Avoids It = - D version with allocation using `std.array.array` can be added (or just `new int[1000]`). The slice over `repeat` is not mutable (users from numpy world may expect that it is an array). Ilya
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 00:24:38 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 21:19:19 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: [...] Thank you for doing this! First draft: http://jackstouffer.com/hidden/nd_slice.html Please critique. Awesome! Please find my notes below. [...] - You may want to test both DMD and LDC 0.17.0 alpha1 for bechmarks https://github.com/ldc-developers/ldc/releases/tag/v0.17.0-alpha1 LDC 0.17.0 alpha1 works well with dip80-ndslice v0.8.7. [...] Ilya ldmd2/ldc2 flag -mcpu=native will optimise code for your CPU. -- Ilya
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: On Monday, 28 December 2015 at 14:42:43 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. Whooohooo! Thanks! Ilya I'm writing a blog post giving an overview of std.ndslice and comparing and contrasting to Numpy, to be released when this is merged, and I was wondering if I can have your permission to include your great image processing example in the post? Thank you for doing this! First draft: http://jackstouffer.com/hidden/nd_slice.html Please critique.
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 01:03:39 UTC, Ilya Yaroshenko wrote: On Wednesday, 30 December 2015 at 00:24:38 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 21:19:19 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 17:38:06 UTC, Ilya Yaroshenko wrote: On Tuesday, 29 December 2015 at 17:17:05 UTC, Jack Stouffer wrote: [...] Thank you for doing this! First draft: http://jackstouffer.com/hidden/nd_slice.html Please critique. Awesome! Please find my notes below. [...] - You may want to test both DMD and LDC 0.17.0 alpha1 for bechmarks https://github.com/ldc-developers/ldc/releases/tag/v0.17.0-alpha1 LDC 0.17.0 alpha1 works well with dip80-ndslice v0.8.7. [...] Ilya ldmd2/ldc2 flag -mcpu=native will optimise code for your CPU. -- Ilya The paragraph "This function does not calculate border cases in which a window overlaps the image partially. However, the function can still be used to carry out such calculations. That can be done by creating an amplified image, with the edges reflected from the original image, and then applying the given function to the new file. Note:" ends with "Note:". --Ilya
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 00:24:38 UTC, Ilya Yaroshenko
wrote:
Awesome!
Please find my notes below.
Thanks for the feedback. Good thing I posted this here before
releasing it.
Funny thing, when I made the D example use the mean lambda
function, it got way faster. Even with the larger array size, the
D code went from 138 ns with the small array and the function to
58 ns with the large array and the lambda. And, the Python code
expectantly got slower, even when I made sure to only test the
np.mean function, the time taking the mean of the large array was
145 µs up from 10.5 µs with allocation and taking the mean with
the smaller array.
So now the D version is 2474x faster.
Also, I was unable to get LDC numbers, as when I compiled my test
program with all of the optimization flags, LDC returns 0 hnsecs
the code:
===
import std.range;
import std.algorithm.iteration;
import std.experimental.ndslice;
import std.stdio;
import std.datetime;
import std.conv : to;
void f0() {
auto means = 100_000.iota.sliced(100, 1000).transposed.map!(r
=> sum(r) / r.length);
}
void main() {
auto r = benchmark!(f0)(10_000);
auto f0Result = to!Duration(r[0]);
f0Result.writeln;
}
I'm assuming that LLVM realizes that the variable means is never
used, so it removes it from the final version.
Re: Voting For std.experimental.ndslice
On Wednesday, 30 December 2015 at 06:16:23 UTC, Jack Stouffer wrote: On Wednesday, 30 December 2015 at 00:24:38 UTC, Ilya Yaroshenko wrote: [...] Thanks for the feedback. Good thing I posted this here before releasing it. [...] means in your code is lazy variable :) You may want to made means a global array. And copy mapped result to it. Please made PR to DlangSciencse/examples with your Python and D benchmarks. --Ilya
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 13:30:47 UTC, Márcio Martins wrote: If I am reading the code right, the number of dimensions must always be known at compile time, right? Not necessarily. The dimensions for this slice are computed at runtime. ``` auto slice = 1000.iota.sliced(5, 6, 7); ```
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 13:54:56 UTC, Jack Stouffer wrote: On Tuesday, 29 December 2015 at 13:30:47 UTC, Márcio Martins wrote: If I am reading the code right, the number of dimensions must always be known at compile time, right? Not necessarily. The dimensions for this slice are computed at runtime. ``` auto slice = 1000.iota.sliced(5, 6, 7); ``` ``` With this declaration: ``` struct Slice(size_t _N, _Range) ``` _N, the number of dimensions, must necessarily be known at compile-time, right?
Re: Voting For std.experimental.ndslice
On Tuesday, 29 December 2015 at 07:45:14 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 22:39:45 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 21:43:35 UTC, Robert burner Schadek wrote: [...] Agreed 1. First paragraph was replaced by your variant. 2. Binary representation was moved to Slice type documentation. 3. Small "Quick Start" was added, so new user will start from `sliced` and `Slice`. http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-081dd6e9e8b3810a143d0a5fcba8d60b/web/phobos-prerelease/std_experimental_ndslice.html Thanks! Ilya Fix English: http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-bfed2500425eb407bf2c97fc72e8a0c5/web/phobos-prerelease/std_experimental_ndslice.html If I am reading the code right, the number of dimensions must always be known at compile time, right? Is so, I suppose it was a performance-oriented decision and not an oversight, correct?
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 18:39:47 UTC, Ilya Yaroshenko wrote: Have you read the latest changes? http://forum.dlang.org/post/djgkkrdufycyjhpma...@forum.dlang.org I have. The problem with the doc is that is describes what it can be used for, but it does not describe what it is. It is like saying: "It brings you from point A to point B." Instead you should say it is a car. And a car is self moving object The first few sentences: """ The package is designed for applications such as linear algebra, physics and statistics. It would be well suited to creating machine learning and image processing algorithms, but should also be general enough for use anywhere with homogeneously-typed multidimensional data. """ does not say what it is. Should be something, as far as I understand the package, like: """ This package provides a multidimensional array implementation, suited for scientific computing. Additionally, many functions for iteration, accessing and manipulation are given. """
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: Voting has ended! Thanks to everyone who voted and reviewed. The final tally: Yes: 12 No: 0 Anything left to review/do on the PR? We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready.
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: Voting has ended! Thanks to everyone who voted and reviewed. The final tally: Yes: 12 No: 0 Anything left to review/do on the PR? Very minor DDOC style hack can be fixed. I will fix it during first alpha version. The following console code can not be placed with CCODE macros because looks like `Usage:` word breaks it. So it is D CODE with `default` and `is` keywords highlighted for now. $ median-filter --help Usage: median-filter [] [] options: --nr number of rows in window, default value is 3 --nc number of columns in window default value equals to nr -h --help This help information. We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. Whooohooo! Thanks! Ilya
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: Voting has ended! Thanks to everyone who voted and reviewed. The final tally: Yes: 12 No: 0 Anything left to review/do on the PR? We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. I believe, with Ilya's recent doc fix, it's ready to go.
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 15:52:09 UTC, Robert burner Schadek wrote: Please do not merge this with the current state of documentation. Could you elaborate what issues you have with the docs? All of the functions have Params and Returns sections and the examples show what each function does.
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: The final tally: Yes: 12 No: 0 This is not true, my yes was conditional and the documentation is still weak. So it is Yes: 11 Yes: Conditional No: 0 Please do not merge this with the current state of documentation.
Re: Voting For std.experimental.ndslice
On 12/28/2015 09:57 AM, Jack Stouffer wrote: On Monday, 28 December 2015 at 11:13:57 UTC, Martin Nowak wrote: On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: Voting has ended! Thanks to everyone who voted and reviewed. The final tally: Yes: 12 No: 0 Anything left to review/do on the PR? We're branching for 2.070 soon, would be nice if this can make it, but only if it's really ready. I believe, with Ilya's recent doc fix, it's ready to go. Let's do it. Many thanks to Ilya and all reviewers! -- Andrei
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 15:52:09 UTC, Robert burner Schadek wrote: fix Yes: 11 Yes: Conditional: 1 <<-- fix here No: 0 Please do not merge this with the current state of documentation.
Re: Voting For std.experimental.ndslice
On 12/28/2015 04:43 PM, Robert burner Schadek wrote: On Monday, 28 December 2015 at 18:39:47 UTC, Ilya Yaroshenko wrote: Have you read the latest changes? http://forum.dlang.org/post/djgkkrdufycyjhpma...@forum.dlang.org I have. The problem with the doc is that is describes what it can be used for, but it does not describe what it is. It is like saying: "It brings you from point A to point B." Instead you should say it is a car. And a car is self moving object The first few sentences: """ The package is designed for applications such as linear algebra, physics and statistics. It would be well suited to creating machine learning and image processing algorithms, but should also be general enough for use anywhere with homogeneously-typed multidimensional data. """ does not say what it is. Should be something, as far as I understand the package, like: """ This package provides a multidimensional array implementation, suited for scientific computing. Additionally, many functions for iteration, accessing and manipulation are given. """ FWIW I noticed that too. It took me a while to figure what the thing really is. -- Andrei
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 21:43:35 UTC, Robert burner Schadek wrote: On Monday, 28 December 2015 at 18:39:47 UTC, Ilya Yaroshenko wrote: Have you read the latest changes? http://forum.dlang.org/post/djgkkrdufycyjhpma...@forum.dlang.org I have. The problem with the doc is that is describes what it can be used for, but it does not describe what it is. It is like saying: "It brings you from point A to point B." Instead you should say it is a car. And a car is self moving object The first few sentences: """ The package is designed for applications such as linear algebra, physics and statistics. It would be well suited to creating machine learning and image processing algorithms, but should also be general enough for use anywhere with homogeneously-typed multidimensional data. """ does not say what it is. Should be something, as far as I understand the package, like: """ This package provides a multidimensional array implementation, suited for scientific computing. Additionally, many functions for iteration, accessing and manipulation are given. """ Agreed 1. First paragraph was replaced by your variant. 2. Binary representation was moved to Slice type documentation. 3. Small "Quick Start" was added, so new user will start from `sliced` and `Slice`. http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-081dd6e9e8b3810a143d0a5fcba8d60b/web/phobos-prerelease/std_experimental_ndslice.html Thanks! Ilya
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 15:52:09 UTC, Robert burner Schadek wrote: On Monday, 28 December 2015 at 07:11:00 UTC, Jack Stouffer wrote: The final tally: Yes: 12 No: 0 This is not true, my yes was conditional and the documentation is still weak. So it is Yes: 11 Yes: Conditional No: 0 Please do not merge this with the current state of documentation. Have you read the latest changes? http://forum.dlang.org/post/djgkkrdufycyjhpma...@forum.dlang.org I have write documentation as I understand problem. I don't know what would be not weak for you. This module has three levels. User should study them one by one if he is new be (slice, iteration, selection). I am not a book author, I can describe something to my colleagues, but I have not any understanding of what would be _not weak_. If you _really_ have read all documentation one by one with attention and you have some question, please write them. Then I can improve documentsion. Ilya
Re: Voting For std.experimental.ndslice
On Monday, 28 December 2015 at 22:39:45 UTC, Ilya Yaroshenko wrote: On Monday, 28 December 2015 at 21:43:35 UTC, Robert burner Schadek wrote: [...] Agreed 1. First paragraph was replaced by your variant. 2. Binary representation was moved to Slice type documentation. 3. Small "Quick Start" was added, so new user will start from `sliced` and `Slice`. http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-081dd6e9e8b3810a143d0a5fcba8d60b/web/phobos-prerelease/std_experimental_ndslice.html Thanks! Ilya Fix English: http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-bfed2500425eb407bf2c97fc72e8a0c5/web/phobos-prerelease/std_experimental_ndslice.html
Re: Voting For std.experimental.ndslice
On Thursday, 24 December 2015 at 17:10:25 UTC, Jack Stouffer wrote: On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. Voting ends in three days. Have a merry Christmas everyone! Voting has ended! Thanks to everyone who voted and reviewed. The final tally: Yes: 12 No: 0
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. Voting ends in three days. Have a merry Christmas everyone!
Re: Voting For std.experimental.ndslice
On Monday, 14 December 2015 at 08:29:13 UTC, Andrea Fontana wrote: On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. You have three options: * Yes * Yes with a single condition * No Yes! Thanks! Image example is a good idea but I think it should not use external libraries. If someone starts with d/phobos probably won't download/configure another library to do some tests. I suggest you to fix the example using ppm format. https://en.wikipedia.org/wiki/Netpbm_format#PPM_example It's just a couple of "map!()" away. :) Or bmp, it's not that complex to read in its common form. imageformats consists of a single file, so user can compile it without dub. It would be faster for user to find and use imageformats than to find a converter from jpeg/png to PPM. Furthermore complete dub example is available at DlangScience/examples. If you have an idea about additional example I'll be happy to discuss it. Best, Ilya
Re: Voting For std.experimental.ndslice
On Thursday, 17 December 2015 at 13:33:28 UTC, Robert burner Schadek wrote: On Wednesday, 16 December 2015 at 17:49:03 UTC, Ilya Yaroshenko wrote: In the same time I expect few articles from another engineers about ndslice like this http://dlang.org/intro-to-datetime.html . It is much better to have explanation from different engineers. Please no, put all the doc at one place. Actually, that article was written by the author of the library. People are lazy, if the doc is not in the phobos docs they will not search for it. Make it very hard for people to complain. 0. Package header was added 1. Annotations for `Category` column was added. 2. Internal Binary Representation section was added to std_experimental_ndslice.html 3. `Slice` type contains classification of slicing and indexing, so user can study it along with examples of Slice's overloaded operators. $(CCODE code) does not work if code contains `ALineLikeThat:`. So I use $(D ), but words `is` and `default` are highlighted =\ Update link: http://dtest.thecybershadow.net/artifact/website-76234ca0eab431527327d5ce1ec0ad74c6421533-904569dd4c4451a4514dc4b456c7b395/web/phobos-prerelease/std_experimental_ndslice.html
Re: Voting For std.experimental.ndslice
Yes
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 17:49:03 UTC, Ilya Yaroshenko
wrote:
* Miscellaneous
** string mixins. I think some of the string mixins can be
removed for something more readable/debuggable
**
I have not found examples where string mixins can be removed.
Please refer to particular example. The code for `sliced` and
`assumeSorted` looks ugly. But I don't know how is can be done
another way. Remove this functionality is an option.
template sliced(Names...)
you can simple compute the strings and than pass them as template
parameter or as alias to another function.
If that becomes unpractical at least use q{ } strings
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 17:49:03 UTC, Ilya Yaroshenko wrote: In the same time I expect few articles from another engineers about ndslice like this http://dlang.org/intro-to-datetime.html . It is much better to have explanation from different engineers. Please no, put all the doc at one place. Actually, that article was written by the author of the library. People are lazy, if the doc is not in the phobos docs they will not search for it. Make it very hard for people to complain.
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 17:49:03 UTC, Ilya Yaroshenko wrote: On Wednesday, 16 December 2015 at 11:01:25 UTC, Robert burner Schadek wrote: I have not found examples where string mixins can be removed. Please refer to particular example. The code for `sliced` and `assumeSorted` looks ugly. But I don't know how is can be done another way. Remove this functionality is an option. Thank you and regards, Ilya EDIT: assumeSameStructure
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 11:01:25 UTC, Robert burner
Schadek wrote:
Yes with many conditions:
* Documentation
** The documentation needs a complete rewrite. If I hadn't had
any prior knowledge, I would have needed to read the numpy
documentation to figure what this package does. That is not
acceptable. It is also not clear how the functionally in the
package is supposed to work together, and how it interacts with
the rest of phobos.
I will add description of what this module dose and how it works
under the hood. In the same time I expect few articles from
another engineers about ndslice like this
http://dlang.org/intro-to-datetime.html . It is much better to
have explanation from different engineers.
** Params, Returns ...
Agreed.
* Style
** the source code does not look like phobos
** s/assert (/assert(/g
** s/unittest {/unittest\n{/g
** unittest properties should be on the same line as the
unittest keyword
** spaces between operators
** dfmt and some manuel work is your friend
Agreed all except spaces between operators. I would like to do
not use them in someY examples exactly for readability reasons.
* Testing
** most tests only use itoa, what about arrays what about
arrays with user defined types. That's properly trivial but
should be tested.
Didn't agree. Corresponding features for arrays and
std.container.array are tested. 98% of ndslice do not do anything
with data. Its change strides and lengths _only_.
** interaction with rest of phobos. Can I call map on a Slice?
If I can, it should be tested so that it still works after the
next release.
OK, I will add test for `map`. For other Phobos Slice is just an
Random Access Range and it is already tested for iota, arrays,
std.container.array, dummyranges .
Please note this DMD bug (see reduced example by John):
https://issues.dlang.org/show_bug.cgi?id=15441
* Miscellaneous
** string mixins. I think some of the string mixins can be
removed for something more readable/debuggable
**
I have not found examples where string mixins can be removed.
Please refer to particular example. The code for `sliced` and
`assumeSorted` looks ugly. But I don't know how is can be done
another way. Remove this functionality is an option.
Thank you and regards,
Ilya
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 11:21:13 UTC, John Colvin wrote: On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Docs are hitting a server error. On Sunday, 13 December 2015 at 05:21:49 UTC, Ilya Yaroshenko wrote: Docs fixed link: http://dtest.thecybershadow.net/artifact/website-13cbdcf17d84fc31328c3f517a56bea783c418d6-dd2292a424959b594956eeeba64d391f/web/phobos-prerelease/std_experimental_ndslice.html The latest docs always can be found at PullRequest's CyberShadow/DAutoTest
Re: Voting For std.experimental.ndslice
Yes with many conditions:
* Documentation
** The documentation needs a complete rewrite. If I hadn't had
any prior knowledge, I would have needed to read the numpy
documentation to figure what this package does. That is not
acceptable. It is also not clear how the functionally in the
package is supposed to work together, and how it interacts with
the rest of phobos.
** Params, Returns ...
* Style
** the source code does not look like phobos
** s/assert (/assert(/g
** s/unittest {/unittest\n{/g
** unittest properties should be on the same line as the unittest
keyword
** spaces between operators
** dfmt and some manuel work is your friend
* Testing
** most tests only use itoa, what about arrays what about arrays
with user defined types. That's properly trivial but should be
tested.
** interaction with rest of phobos. Can I call map on a Slice? If
I can, it should be tested so that it still works after the next
release.
* Miscellaneous
** string mixins. I think some of the string mixins can be
removed for something more readable/debuggable
**
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Docs are hitting a server error.
Re: Voting For std.experimental.ndslice
On Wednesday, 16 December 2015 at 11:31:00 UTC, Dennis Ritchie wrote: On Wednesday, 16 December 2015 at 11:21:13 UTC, John Colvin wrote: On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Docs are hitting a server error. On Sunday, 13 December 2015 at 05:21:49 UTC, Ilya Yaroshenko wrote: Docs fixed link: http://dtest.thecybershadow.net/artifact/website-13cbdcf17d84fc31328c3f517a56bea783c418d6-dd2292a424959b594956eeeba64d391f/web/phobos-prerelease/std_experimental_ndslice.html The latest docs always can be found at PullRequest's CyberShadow/DAutoTest Totally forgot reading that, thanks.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 12:57:00 UTC, Nordlöw wrote: To vote, please respond to this post. Yes I've contributed to both DMD and Phobos.
Re: Voting For std.experimental.ndslice
Yes
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. You have three options: * Yes * Yes with a single condition * No Yes! Image example is a good idea but I think it should not use external libraries. If someone starts with d/phobos probably won't download/configure another library to do some tests. I suggest you to fix the example using ppm format. https://en.wikipedia.org/wiki/Netpbm_format#PPM_example It's just a couple of "map!()" away. :) Or bmp, it's not that complex to read in its common form.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Yes
Re: Voting For std.experimental.ndslice
13.12.2015 12:01, lobo пишет: On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Yes Yes
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. Yes
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Yes.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 06:07:15 UTC, Meta wrote: Is std.experimental also on the table, or directly to phobos? Not quite sure what you're asking here, but if you're asking if this is going into stdx first, then the answer is yes. Things that go into stdx aren't supposed to change that much.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] I've played some large part in the various iterations of reviewing this, so I'm biased, but of course: yes
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. You have three options: Yes
Re: Voting For std.experimental.ndslice
Yes Hope to see a Matrix (multidimensional, if possible) class with this.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 17:07:06 UTC, Jack Stouffer wrote: On Sunday, 13 December 2015 at 06:07:15 UTC, Meta wrote: Is std.experimental also on the table, or directly to phobos? Not quite sure what you're asking here, but if you're asking if this is going into stdx first, then the answer is yes. Things that go into stdx aren't supposed to change that much. Ok, your wording about the API freezing seemed to imply that it would go directly into Phobos without spending some time in std.experimental. Anyway, I'm not familiar with the subject domain, so I'll abstain from voting. I will say however that the documentation could use some work.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 22:16:35 UTC, Tanel Tagaväli wrote: Yes Hope to see a Matrix (multidimensional, if possible) class with this. What matrix features are missing in ndslice except matrix multiplication?
Voting For std.experimental.ndslice
This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. You have three options: * Yes * Yes with a single condition * No If you vote "yes" you can still mention something you'd like improved, but please be explicit if that problem is a non starter for you and you are choosing option two. If you vote no, please state why, though you aren't required to. Everyone can vote, but the contributors to the language will have more sway. If you are a contributor, please identify yourself as such. Some things to consider when making a vote: * Is this functionality needed in Phobos? * The API is practically permanent once the module is accepted. Some minor changes can be made, but a full redesign is no longer an option. The voting will end December 27th. Github: https://github.com/D-Programming-Language/phobos/pull/3397 dub: http://code.dlang.org/packages/dip80-ndslice docs: http://dtest.thecybershadow.net/artifact/website-13cbdcf17d84fc31328c3f517a56bea783c418d6-dd2292a424959b594956eeeba64d391f/web/phobos-prerelease/std_experimental_ndslice.html/ original discussion: http://forum.dlang.org/thread/rilfmeaqkailgpxoz...@forum.dlang.org review thread: http://forum.dlang.org/post/uesnmkgniumswfclw...@forum.dlang.org
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. To vote, please respond to this post. You have three options: * Yes * Yes with a single condition * No I vote yes. I am a contributor to Phobos.
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: This is the voting thread to decide if the proposed addition to Phobos, std.experimental.ndslice, should be accepted. [...] Docs fixed link: http://dtest.thecybershadow.net/artifact/website-13cbdcf17d84fc31328c3f517a56bea783c418d6-dd2292a424959b594956eeeba64d391f/web/phobos-prerelease/std_experimental_ndslice.html The latest docs always can be found at PullRequest's CyberShadow/DAutoTest
Re: Voting For std.experimental.ndslice
On Sunday, 13 December 2015 at 05:10:17 UTC, Jack Stouffer wrote: * The API is practically permanent once the module is accepted. Some minor changes can be made, but a full redesign is no longer an option. Is std.experimental also on the table, or directly to phobos?
