Re: Setting a hard limit on slice size, is this possible?
On Sunday, 8 August 2021 at 02:00:26 UTC, Tejas wrote: On Saturday, 7 August 2021 at 19:07:04 UTC, Paul Backus wrote: On Saturday, 7 August 2021 at 15:41:24 UTC, Tejas wrote: On Saturday, 7 August 2021 at 15:21:01 UTC, Paul Backus wrote: [...] Oh wow, and here I thought I was being smart :( So, how can we work around this without assembly language magic? I'm illiterate at assembly. For stack allocations, you can use the workaround in [Vladimir Panteleev's comment][1] (ignoring the ASM part, which is unrelated to alignment). For heap allocations, I guess the easiest way would be to use [`AlignedMallocator`][2] from `std.experimental.allocator`. [1]: https://issues.dlang.org/show_bug.cgi?id=16098#c3 [2]: https://phobos.dpldocs.info/std.experimental.allocator.mallocator.AlignedMallocator.html Cool... thanks Thanks to everyone for all of the great discussion and hints on this topic! I have learned quite much, and have a gained an understanding on how to use D effectively. Fantastic forum here on a great language. Best Regards, James
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 19:07:04 UTC, Paul Backus wrote: On Saturday, 7 August 2021 at 15:41:24 UTC, Tejas wrote: On Saturday, 7 August 2021 at 15:21:01 UTC, Paul Backus wrote: [...] Oh wow, and here I thought I was being smart :( So, how can we work around this without assembly language magic? I'm illiterate at assembly. For stack allocations, you can use the workaround in [Vladimir Panteleev's comment][1] (ignoring the ASM part, which is unrelated to alignment). For heap allocations, I guess the easiest way would be to use [`AlignedMallocator`][2] from `std.experimental.allocator`. [1]: https://issues.dlang.org/show_bug.cgi?id=16098#c3 [2]: https://phobos.dpldocs.info/std.experimental.allocator.mallocator.AlignedMallocator.html Cool... thanks
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 15:41:24 UTC, Tejas wrote: On Saturday, 7 August 2021 at 15:21:01 UTC, Paul Backus wrote: The issue with `align` attributes being ignored for stack variables is apparently a known bug, first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16098 The issue with `align` attributes being ignored by `new` is also a known bug, and was also first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16508 Oh wow, and here I thought I was being smart :( So, how can we work around this without assembly language magic? I'm illiterate at assembly. For stack allocations, you can use the workaround in [Vladimir Panteleev's comment][1] (ignoring the ASM part, which is unrelated to alignment). For heap allocations, I guess the easiest way would be to use [`AlignedMallocator`][2] from `std.experimental.allocator`. [1]: https://issues.dlang.org/show_bug.cgi?id=16098#c3 [2]: https://phobos.dpldocs.info/std.experimental.allocator.mallocator.AlignedMallocator.html
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 15:21:01 UTC, Paul Backus wrote: On Saturday, 7 August 2021 at 14:34:49 UTC, Tejas wrote: [...] For the array as a whole to be aligned, not only must the spacing between the elements respect the alignment, but starting address of the array itself must be a multiple of the alignment. So it is correct to check that `x.ptr%ALIGNMENT == 0`. The issue with `align` attributes being ignored for stack variables is apparently a known bug, first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16098 The issue with `align` attributes being ignored by `new` is also a known bug, and was also first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16508 Oh wow, and here I thought I was being smart :( So, how can we work around this without assembly language magic? I'm illiterate at assembly.
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 14:34:49 UTC, Tejas wrote: Umm, the ```align array``` solution is flat out wrong, please ignore it. Most likely a bug in the compiler. Also, why will the address of the first element of the array ```modulo``` alignment be 0? The address of the array has absolutely nothing to do with the alignment. You say that the ```align array``` solution has the expected spacing, but it is default aligned, totally ignoring your 64 byte requirement. Don't use ```x.PTR%ALIGNMENT``` Use ```([1]-&[2])%ALIGNMENT``` You will get ALIGNMENT*(2 -1)(since you took difference of 2nd and 1st elem) For the array as a whole to be aligned, not only must the spacing between the elements respect the alignment, but starting address of the array itself must be a multiple of the alignment. So it is correct to check that `x.ptr%ALIGNMENT == 0`. The issue with `align` attributes being ignored for stack variables is apparently a known bug, first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16098 The issue with `align` attributes being ignored by `new` is also a known bug, and was also first reported in 2016: https://issues.dlang.org/show_bug.cgi?id=16508
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 13:36:52 UTC, james.p.leblanc wrote: On Saturday, 7 August 2021 at 12:08:00 UTC, Paul Backus wrote: [...] **First, thanks all for helping with this question!** The simple desire to arbitrarily align an array is certainly looking non-trivial. Below is a simple program of both the suggested "struct" and "align array" solutions. Unfortunately, neither is guaranteed to place the array with the desired alignnment. [...] If you don't believe me just subtract the adresses yourself: x.ptr: 78D05B90 [0]78D05B90 [1]78D05BD0 [2]78D05C10 [1] 78D05BD0 -[0] -78D05B90 _ 0040 0X40 = 64 in base 10 Your alignment requirement was 64 bytes
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 13:36:52 UTC, james.p.leblanc wrote: On Saturday, 7 August 2021 at 12:08:00 UTC, Paul Backus wrote: [...] **First, thanks all for helping with this question!** The simple desire to arbitrarily align an array is certainly looking non-trivial. Below is a simple program of both the suggested "struct" and "align array" solutions. Unfortunately, neither is guaranteed to place the array with the desired alignnment. [...] Umm, the ```align array``` solution is flat out wrong, please ignore it. Most likely a bug in the compiler. Also, why will the address of the first element of the array ```modulo``` alignment be 0? The address of the array has absolutely nothing to do with the alignment. You say that the ```align array``` solution has the expected spacing, but it is default aligned, totally ignoring your 64 byte requirement. Don't use ```x.PTR%ALIGNMENT``` Use ```([1]-&[2])%ALIGNMENT``` You will get ALIGNMENT*(2 -1)(since you took difference of 2nd and 1st elem)
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 12:08:00 UTC, Paul Backus wrote:
On Saturday, 7 August 2021 at 07:32:04 UTC, Tejas wrote:
And if it really is correct, then it seems once again that
static arrays are the answer after all:
```d
align(your_alignment) int[your_length] array;
```
No need for structs \\('_')/
The main advantage of the struct is that you can heap-allocate
it:
```d
auto array = new Aligned!(int[4], 16);
```
**First, thanks all for helping with this question!**
The simple desire to arbitrarily align an array is certainly
looking non-trivial.
Below is a simple program of both the suggested "struct" and
"align array" solutions.
Unfortunately, neither is guaranteed to place the array with the
desired alignnment.
import std.stdio;
enum ALIGNMENT=64;
enum LENGTH=10;
struct Aligned(T, size_t alignment) if (alignment >=
T.alignof)
{
align(alignment) T payload;
alias payload this;
}
void main()
{
writeln("ALIGNMENT: ", ALIGNMENT, ",\t\tLENGTH: ",
LENGTH);
int[23] junk;
align(ALIGNMENT) int[LENGTH] z;
writeln("\nusing 'align(ALIGNMENT) int[LENGTH] z;'");
writeln("\ncast(ulong) z.ptr%ALIGNMENT: ", cast(ulong)
z.ptr%ALIGNMENT);
writeln("int.alignof: ", int.alignof, ",\tint.sizeof: ",
int.sizeof);
writeln("[0]", [0]);
writeln("[1]", [1]);
writeln("[2]", [2]);
writeln("\nusing: 'Aligned!(int, ALIGNMENT)[LENGTH] x;'");
Aligned!(int, ALIGNMENT)[LENGTH] x;
writeln("x.sizeof: ", x.sizeof, "\t\tx.alignof: ",
x.alignof);
writeln("x: ", x);
writeln("\nx.ptr: ", x.ptr);
writeln("[0]", [0]);
writeln("[1]", [1]);
writeln("[2]", [2]);
writeln("\ncast(ulong) x.ptr%ALIGNMENT: ", cast(ulong)
x.ptr%ALIGNMENT);
}
---
and here is a sample output of a run:
ALIGNMENT: 64, LENGTH: 10
using 'align(ALIGNMENT) int[LENGTH] z;'
cast(ulong) z.ptr%ALIGNMENT: ***32***
int.alignof: 4,int.sizeof: 4
[0]78D05B60
[1]78D05B64
[2]78D05B68
using: 'Aligned!(int, ALIGNMENT)[LENGTH] x;'
x.sizeof: 640 x.alignof: 64
x: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
x.ptr: 78D05B90
[0]78D05B90
[1]78D05BD0
[2]78D05C10
cast(ulong) x.ptr%ALIGNMENT: ***16***
---
Notice, that neither attempt yields an array starting address as
zero modulo the alignment value ...
Also, a bit weird is that the "align array" solution yields
expected
spacing between array elements, but the "struct" solution has
wider
separations ...
--
**Again, I am very appreciative of your replies and help with
this.**
I have learned quite a bit from this discussion. However, I
remain
a bit stumped by all of this
Any ideas out there?
Best Regards,
James
Re: Setting a hard limit on slice size, is this possible?
On Saturday, 7 August 2021 at 07:32:04 UTC, Tejas wrote:
And if it really is correct, then it seems once again that
static arrays are the answer after all:
```d
align(your_alignment) int[your_length] array;
```
No need for structs \\('_')/
The main advantage of the struct is that you can heap-allocate it:
```d
auto array = new Aligned!(int[4], 16);
```
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 22:15:00 UTC, Paul Backus wrote:
On Friday, 6 August 2021 at 19:03:53 UTC, Tejas wrote:
Stealing Paul's answer now:
```d
import std;
enum your_max_length = 1000;
enum your_align = 256;
struct MySlice(T/*, size_t maxLength*/)
{
private align(your_align)T payload;
//invariant(payload.length <= maxLength);
//this(T[] slice) { payload = slice; }
//T opIndex(size_t i) { return payload[i]; }
}
void main()
{
MySlice!(int/*, 1000*/)[your_max_length] slice;
writeln(slice.sizeof);
}
```
You can actually pass the alignment as a parameter too:
```d
struct Aligned(T, size_t alignment)
if (alignment >= T.alignof)
{
align(alignment) T payload;
alias payload this;
}
void main()
{
Aligned!(int, 16)[4] x;
assert(x.alignof == 16);
assert(x.sizeof == 64);
Aligned!(int[4], 16) y;
assert(y.alignof == 16);
assert(y.sizeof == 16);
}
```
Is the second example really correct? if the alignment of
```int[4]``` really is 16, then why isn't the size of it 64 as
well? It seems that the alignment constraint is not being
satisfied _within_ the array,
And if it really is correct, then it seems once again that static
arrays are the answer after all:
```d
align(your_alignment) int[your_length] array;
```
No need for structs \\('_')/
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 19:03:53 UTC, Tejas wrote:
Stealing Paul's answer now:
```d
import std;
enum your_max_length = 1000;
enum your_align = 256;
struct MySlice(T/*, size_t maxLength*/)
{
private align(your_align)T payload;
//invariant(payload.length <= maxLength);
//this(T[] slice) { payload = slice; }
//T opIndex(size_t i) { return payload[i]; }
}
void main()
{
MySlice!(int/*, 1000*/)[your_max_length] slice;
writeln(slice.sizeof);
}
```
You can actually pass the alignment as a parameter too:
```d
struct Aligned(T, size_t alignment)
if (alignment >= T.alignof)
{
align(alignment) T payload;
alias payload this;
}
void main()
{
Aligned!(int, 16)[4] x;
assert(x.alignof == 16);
assert(x.sizeof == 64);
Aligned!(int[4], 16) y;
assert(y.alignof == 16);
assert(y.sizeof == 16);
}
```
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 18:02:01 UTC, james.p.leblanc wrote:
mes
On Friday, 6 August 2021 at 17:25:24 UTC, Tejas wrote:
Okay we were overthinking the solution.
Just use a static array
```d
int[your_max_length]/*or whatever type*/ var;
```
You're good to go!
I almost feel stupid now lol
Hello Tejas,
Kind thanks for your replies ... all are appreciated.
However, do NOT feel stupid ... the motivation behind why
I cannot use a standard int[your_max_length] (in other words,
use a static array), is because I need to do a specified
memory alignment (known at compile time) on my slice, or array.
I understand that neither a slice or an array is capable to
doing
an arbitrary memory alignment. (But, perhaps I am wrong about
this ...)
I believe structs can be aligned, but need to learn more about
the speicific of that.
In the meantime, I have written a way to get an aligned slice
from a
static array. Ugly beginner code is below. While I believe
this basic
idea should work, I would like to guarantee that my slice does
not get
moved in memory (destroying the alignment).
Ugly code here:
--
import std.stdio;
enum MAXLENGTH = 1024;
enum ALIGN = 128;
void main(){
// static array to allow creation of aligned slice
ubyte[MAXLENGTH+ALIGN] u;
writeln("u.ptr: ", u.ptr);
auto u_i = cast(ulong) u.ptr;
auto u_excess = u_i%ALIGN;
writeln("u_excess: ", u_excess);
ulong mustadd;
if(u_excess !=0){
mustadd = ALIGN-u_excess;
}
writeln("mustadd: ", mustadd);
// create aligned pointer for our needed slice
auto zp = cast(double*) (u.ptr + mustadd);
// create slice
double[] z = zp[0 .. MAXLENGTH];
writeln("z.ptr: ", z.ptr);
writeln("z.length: ", z.length);
auto z_i = cast(ulong) z.ptr;
auto z_excess = z_i%ALIGN;
writeln("z_excess: ", z_excess);
}
If you're being that specialized, don't bother with fundamental
data types, the hunt for simplicity will lead you to create very
complex things.
As HS Teoh said, take a look at ```align```:
https://dlang.org/spec/attribute.html#align
Stealing Paul's answer now:
```d
import std;
enum your_max_length = 1000;
enum your_align = 256;
struct MySlice(T/*, size_t maxLength*/)
{
private align(your_align)T payload;
//invariant(payload.length <= maxLength);
//this(T[] slice) { payload = slice; }
//T opIndex(size_t i) { return payload[i]; }
}
void main()
{
MySlice!(int/*, 1000*/)[your_max_length] slice;
writeln(slice.sizeof);
}
```
Re: Setting a hard limit on slice size, is this possible?
On Fri, Aug 06, 2021 at 06:02:01PM +, james.p.leblanc via Digitalmars-d-learn wrote: [...] > However, do NOT feel stupid ... the motivation behind why > I cannot use a standard int[your_max_length] (in other words, > use a static array), is because I need to do a specified > memory alignment (known at compile time) on my slice, or array. I've never actually tried this myself, but have you looked into the `align` keyword in the D spec? I'm not 100% sure whether it can be applied to static arrays (or arrays in general) but it might be worth a look. > I understand that neither a slice or an array is capable to doing > an arbitrary memory alignment. (But, perhaps I am wrong about this > ...) > > I believe structs can be aligned, but need to learn more about the > speicific of that. One thing that might help you is if you had a struct with the appropriate align(...) declaration that wraps around your array elements. Pretty sure that would guarantee the alignment of array elements, which in turn should guarantee alignment of the array as a whole. T -- It is the quality rather than the quantity that matters. -- Lucius Annaeus Seneca
Re: Setting a hard limit on slice size, is this possible?
mes
On Friday, 6 August 2021 at 17:25:24 UTC, Tejas wrote:
Okay we were overthinking the solution.
Just use a static array
```d
int[your_max_length]/*or whatever type*/ var;
```
You're good to go!
I almost feel stupid now lol
Hello Tejas,
Kind thanks for your replies ... all are appreciated.
However, do NOT feel stupid ... the motivation behind why
I cannot use a standard int[your_max_length] (in other words,
use a static array), is because I need to do a specified
memory alignment (known at compile time) on my slice, or array.
I understand that neither a slice or an array is capable to doing
an arbitrary memory alignment. (But, perhaps I am wrong about
this ...)
I believe structs can be aligned, but need to learn more about
the speicific of that.
In the meantime, I have written a way to get an aligned slice
from a
static array. Ugly beginner code is below. While I believe this
basic
idea should work, I would like to guarantee that my slice does
not get
moved in memory (destroying the alignment).
Ugly code here:
--
import std.stdio;
enum MAXLENGTH = 1024;
enum ALIGN = 128;
void main(){
// static array to allow creation of aligned slice
ubyte[MAXLENGTH+ALIGN] u;
writeln("u.ptr: ", u.ptr);
auto u_i = cast(ulong) u.ptr;
auto u_excess = u_i%ALIGN;
writeln("u_excess: ", u_excess);
ulong mustadd;
if(u_excess !=0){
mustadd = ALIGN-u_excess;
}
writeln("mustadd: ", mustadd);
// create aligned pointer for our needed slice
auto zp = cast(double*) (u.ptr + mustadd);
// create slice
double[] z = zp[0 .. MAXLENGTH];
writeln("z.ptr: ", z.ptr);
writeln("z.length: ", z.length);
auto z_i = cast(ulong) z.ptr;
auto z_excess = z_i%ALIGN;
writeln("z_excess: ", z_excess);
}
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 10:50:19 UTC, james.p.leblanc wrote: I am aware of the "capacity" concept with slices. But, I would like to know if it is possible to set a hard limit on a slice size. I prefer it to error and crash instead of a doing an extension or reallocation. I understand my question screams of "convoluted thinking". But, I need to align my slice according to certain criteria. (Alternatively, I could use an array ... if I could align that according to criteria known at compile time. Is this possible?). I have a working solution (ugly trick, maybe is a better description) to align my slice as desired. But, the solution would be more robust if I could guarantee that the slice is never moved in memory. Any thoughts, hints, etc are welcome! James Okay we were overthinking the solution. Just use a static array ```d int[your_max_length]/*or whatever type*/ var; ``` You're good to go! I almost feel stupid now lol
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 17:16:28 UTC, Tejas wrote: On Friday, 6 August 2021 at 16:32:59 UTC, james.p.leblanc wrote: On Friday, 6 August 2021 at 11:58:59 UTC, Paul Backus wrote: [...] ``` Paul, Thanks very much for your reply. I understand only a fraction of the suggested solution. I will need to digest this a bit, and do some more background reading on templates. My initial, naive attempts to use this in a simple main() were unsuccessful. I'll keep plugging at it ... Best Regards, James If you want something even simpler, maybe this could help: ```d void setCapacity(int[] a)// not int a[] ```
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 16:32:59 UTC, james.p.leblanc wrote:
On Friday, 6 August 2021 at 11:58:59 UTC, Paul Backus wrote:
struct MySlice(T, size_t maxLength)
{
private T[] payload;
invariant(payload.length <= maxLength);
this(T[] slice) { payload = slice; }
T opIndex(size_t i) { return payload[i]; }
// etc.
}
```
Paul,
Thanks very much for your reply. I understand only
a fraction of the suggested solution. I will need to
digest this a bit, and do some more background reading
on templates.
My initial, naive attempts to use this in a simple main()
were unsuccessful. I'll keep plugging at it ...
Best Regards,
James
If you want something even simpler, maybe this could help:
```d
void setCapacity(int/*or whatever type you want*/ a[], ulong
capacity){
assert(capacity < 100/*whatever you want to set*/);
a.length = capacity;
}
void main(){
int[] arr;
setCapacity(arr, 50);//works :D
//setCapacity(arr, 1000);// fails :(
}
```
Of course, this won't work if you want to resize the capacity
natively, ie, by using ```.length``` directly; then you will have
to use Paul's solution.
If you still have difficulties, please ping.
Also, if you're completely new, please try the book "programming
in D", it is a marvellous resource.
link:
http://ddili.org/ders/d.en/
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 11:58:59 UTC, Paul Backus wrote:
struct MySlice(T, size_t maxLength)
{
private T[] payload;
invariant(payload.length <= maxLength);
this(T[] slice) { payload = slice; }
T opIndex(size_t i) { return payload[i]; }
// etc.
}
```
Paul,
Thanks very much for your reply. I understand only
a fraction of the suggested solution. I will need to
digest this a bit, and do some more background reading
on templates.
My initial, naive attempts to use this in a simple main()
were unsuccessful. I'll keep plugging at it ...
Best Regards,
James
Re: Setting a hard limit on slice size, is this possible?
On Friday, 6 August 2021 at 10:50:19 UTC, james.p.leblanc wrote:
I am aware of the "capacity" concept with slices.
But, I would like to know if it is possible to set a
hard limit on a slice size.
I prefer it to error and crash instead of a doing an
extension or reallocation.
I don't think there's any way to make built-in slices behave like
this, but you can pretty easily define your own wrapper type:
```d
struct MySlice(T, size_t maxLength)
{
private T[] payload;
invariant(payload.length <= maxLength);
this(T[] slice) { payload = slice; }
T opIndex(size_t i) { return payload[i]; }
// etc.
}
```
