Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 13:34:42 UTC, Richard (Rikki)
Andrew Cattermole wrote:
In case you didn't know, all you need to get unittests working
in -betterC is:
```d
foreach (module_; allModules) {
foreach (unitTest; __traits(getUnitTests,
module_)) {
unitTest();
}
}
```
You'd need to provide allModules somehow, like dub does.
https://github.com/dlang/dub/blob/2ea883833adf085095b07a7dba8250fb3db79a71/source/dub/project.d#L1937
TIL, thanks for sharing, this will be very useful!
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 10:54:34 UTC, bachmeier wrote: Hate to be that guy, but I posted a link to a stackoverflow question with the exact error message you were getting, and the solution. And I told you I had experienced the same error and that question fixed it. No reason to not says something, always speak you mind my friend! I did read it and all the replies but the problem is that, I was so focused on thinking that something is going wrong with the compiler or with libc or something unexpected. For that reason, I made the mistake to not see other things that might had gotten wrong even tho you literally gave me the answer! That's another problem of mine that I am aware and trying to fix (not only when it comes to coding but in general) and I do apologize for wasting both mine and everyone else's time! Now that people explained to me, I fully understood what's going on and how I made the mistake. Have a great day my friend and I hope I can repay one day! ;)
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 09:30:10 UTC, rempas wrote: Bingo! You and Brad found out! Hate to be that guy, but I posted a link to a stackoverflow question with the exact error message you were getting, and the solution. And I told you I had experienced the same error and that question fixed it.
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 09:56:59 UTC, H. S. Teoh wrote: libc doesn't know what you intended. All it knows is that you asked it for 20 bytes (even though you actually needed 40), then later on its internal structures are corrupted (because you thought you got 40 bytes; storing data past the 20 bytes overwrote some of malloc's internal data -- this is the buffer overrun / buffer overflow I referred to). So it aborts the program instead of continuing to run in a compromised state. T Thank you! I fully realize now what's the problem! And that was indeed a very sneaky problem. The good news is that I'm mostly done with these memory structures and functions so I will probably take a while since I find something similar. I'm lucky there are people smarter, more experience that are willing to help. Bless you all and have a great day!
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 09:47:14 UTC, Steven Schveighoffer wrote: You are focusing on the wrong problem. You asked for size bytes, and malloc gave you size bytes. It doesn't "know" anything special. Then you proceeded at some point to write *past* the size bytes. What did you overwrite? Probably some internal malloc implementation structure. Then it later noticed "hey, this structure doesn't make sense, I'm going to report it to the user!" That's why you see the message. Memory problems are very difficult to find, and typically an error is triggered far away from the source, in seemingly unrelated code. This is why whenever I see an error that smells like memory corruption, I stop all other work and find it. Memory errors can come and go based on random chance or how the compiler lays out functions. So having it "just go away" isn't enough. Very very infrequently, this happens because of a codegen issue, but most of the time it's pilot error. -Steve I understand! Thank you for the valuable information. I do have lots of things to learn it seems. But that means that I also won't get bored anytime soon ;)
Re: malloc error when trying to assign the returned pointer to a struct field
On Sat, Sep 09, 2023 at 09:21:32AM +, rempas via Digitalmars-d-learn wrote: > On Saturday, 9 September 2023 at 08:54:14 UTC, Brad Roberts wrote: > > I'm pretty sure this is your problem. You're allocating size bytes > > which is only going to work where sizeof(T) == 1. Changing to > > malloc(size * sizeof(T)) is likely going to work better. > > Oh man That was it! I had forget about that! Funny enough, the > reallocation tests I do letter when expanding the vector do include > that but I had forgot to place it in the new (because I had the an old > one and it included this) constructor I had made that only allocates > memory! > > Now, if only one could expect how and why "libc" knows that and > doesn't just care to give me the memory I asked it for? Or it could be > than D does something additional without telling us? Which can explain > when this memory is only present when I assign the value to the > "this._ptr` field! libc doesn't know what you intended. All it knows is that you asked it for 20 bytes (even though you actually needed 40), then later on its internal structures are corrupted (because you thought you got 40 bytes; storing data past the 20 bytes overwrote some of malloc's internal data -- this is the buffer overrun / buffer overflow I referred to). So it aborts the program instead of continuing to run in a compromised state. T -- There are four kinds of lies: lies, damn lies, and statistics.
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 09:21:32 UTC, rempas wrote: Now, if only one could expect how and why "libc" knows that and doesn't just care to give me the memory I asked it for? Or it could be than D does something additional without telling us? Which can explain when this memory is only present when I assign the value to the "this._ptr` field! You are focusing on the wrong problem. You asked for size bytes, and malloc gave you size bytes. It doesn't "know" anything special. Then you proceeded at some point to write *past* the size bytes. What did you overwrite? Probably some internal malloc implementation structure. Then it later noticed "hey, this structure doesn't make sense, I'm going to report it to the user!" That's why you see the message. Memory problems are very difficult to find, and typically an error is triggered far away from the source, in seemingly unrelated code. This is why whenever I see an error that smells like memory corruption, I stop all other work and find it. Memory errors can come and go based on random chance or how the compiler lays out functions. So having it "just go away" isn't enough. Very very infrequently, this happens because of a codegen issue, but most of the time it's pilot error. -Steve
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 09:04:18 UTC, Steven
Schveighoffer wrote:
This is not ideal. Why? Because 99% of the time, a poster has
come here with a problem they don't know how to solve, and have
focused in on where they *think* the problem is. However, the
problem isn't there. But us reading the description can only
see what the poster sees, and either don't see a problem ("I'm
just as confused as you are!") or know there is more to the
story.
Not only that, but frequently not-complete code is... not
complete. And people tend to focus on problems they can see
(e.g. where is that `_len` defined?), frustrating the poster
with "trivial" problems that are solved "in the real code".
Inevitably, there is a subsequent post with the real code, and
that contains the problem.
The best thing to post is a minimally reproducing example. The
next best thing is a link to a complex reproducing example.
Which you have done later (I will take a look). I just wanted
to point this out because it's a frequent problem on these
forums.
Yeah... I got my lesson! Tbh, I wanted to avoid posting my source
as much as possible because I do have this "perfectionist"
mindset (which I am aware is bad and trying to get rid of, and
have already done progress) and I want the source to sound as
"professional" as possible as I take serious my project and have
hopes for it! But yeah, NEVER again!
Here, you have allocated `size` bytes for the array. Is this
what is intended? Your comments suggest otherwise! If
`T.sizeof` is 2 or more, then you still only allocate e.g. 20
bytes for a `_cap` of 20. but an array of 20 T's would require
40 bytes.
-Steve
Bingo! You and Brad found out! And like I said to him, it's funny
that I had a previous function that just allocates memory and I
removed it and then created that one and in that one, I forgot
about that. It's super funny cause I even wrote the comment that
explains it and my reallocation function (that gets triggered
when someone wants to expand the collection) has that
multiplication one it.
Thank you and everyone else for the help! Hope I can at least
finish and share something that people will enjoy using!
Re: malloc error when trying to assign the returned pointer to a struct field
On Saturday, 9 September 2023 at 08:54:14 UTC, Brad Roberts wrote: I'm pretty sure this is your problem. You're allocating size bytes which is only going to work where sizeof(T) == 1. Changing to malloc(size * sizeof(T)) is likely going to work better. Oh man That was it! I had forget about that! Funny enough, the reallocation tests I do letter when expanding the vector do include that but I had forgot to place it in the new (because I had the an old one and it included this) constructor I had made that only allocates memory! Now, if only one could expect how and why "libc" knows that and doesn't just care to give me the memory I asked it for? Or it could be than D does something additional without telling us? Which can explain when this memory is only present when I assign the value to the "this._ptr` field! Hope I don't have other memory related errors until I finish the compiler... How can I thank you
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:
I do have the following struct:
...
That's some minimal code that I do have just to showcase it.
This is not ideal. Why? Because 99% of the time, a poster has
come here with a problem they don't know how to solve, and have
focused in on where they *think* the problem is. However, the
problem isn't there. But us reading the description can only see
what the poster sees, and either don't see a problem ("I'm just
as confused as you are!") or know there is more to the story.
Not only that, but frequently not-complete code is... not
complete. And people tend to focus on problems they can see (e.g.
where is that `_len` defined?), frustrating the poster with
"trivial" problems that are solved "in the real code".
Inevitably, there is a subsequent post with the real code, and
that contains the problem.
The best thing to post is a minimally reproducing example. The
next best thing is a link to a complex reproducing example. Which
you have done later (I will take a look). I just wanted to point
this out because it's a frequent problem on these forums.
So, some times, this work will works, some others, it will give
me the following error:
`Fatal glibc error: malloc.c:2594 (sysmalloc): assertion
failed: (old_top == initial_top (av) && old_size == 0) ||
((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top)
&& ((unsigned long) old_end & (pagesize - 1)) == 0)`
This is an internal message from glibc. It seems the malloc
structure is corrupted.
Is there any possible that there is a compiler bug? I do use
ldc2 and `betterC`!
There is always a chance...
Now, critiquing your original code, I see red flags here:
```d
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
```
and then later:
```d
this(i64 size) {
this._len = 0;
this._cap = size;
```
ok, so `size` must mean the number of elements, not the number of
bytes.
```d
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
```
Here, you have allocated `size` bytes for the array. Is this what
is intended? Your comments suggest otherwise! If `T.sizeof` is 2
or more, then you still only allocate e.g. 20 bytes for a `_cap`
of 20. but an array of 20 T's would require 40 bytes.
-Steve
Re: malloc error when trying to assign the returned pointer to a struct field
On 9/8/2023 12:59 AM, rempas via Digitalmars-d-learn wrote: u64 _cap = 0; // Total amount of elements (not bytes) we can this._ptr = cast(T*)malloc(size); I'm pretty sure this is your problem. You're allocating size bytes which is only going to work where sizeof(T) == 1. Changing to malloc(size * sizeof(T)) is likely going to work better.
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 19:48:33 UTC, Basile B. wrote: My idea was that if you dont have defined a copy constructor and if an instance is assigned to another, then that other instance share the same pointer, which can cause memory errors. To eliminate that risk and to detect where default post-blitting may happen you can add ```d @disable this(this); ``` to the struct. Thank you! I did and I got a bunch of errors! Seeing where they are located, it doesn't seem to be in the file and instances that give me the error. Something really weird happens. I'll probably return to my own memory allocated and fix the bugs I had there, it will probably take me less time...
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 19:14:47 UTC, H. S. Teoh wrote: The error message looks to me like a corruption of the malloc heap. These kinds of bugs are very hard to trace, because they may go undetected and only show up in specific circumstances, so small perturbations of completely unrelated code may make the bug appear or disappear -- just because the bug doesn't show up when you disable some code does not prove that that's where the problem is; it could be that corruption is still happening, it just so happens that it goes unnoticed when the behaviour of the code changes slightly. Yep! That's what I guess as well! Tbh, I knew that playing with memory and trying to write a system library will be a touch job and I'm up for it! But these bugs really burn me out because like you said, nobody can truly help because they are so weird... My guess is that you have a double-free somewhere, or there's a buffer overrun. Or maybe some bad interaction with the GC, e.g. if you tried to free a pointer from the GC heap. (Note that this may not immediately show up; free() could've assumed that everything was OK when it has in fact messed up its internal data structures; the problem would only show up later on in code that's actually unrelated to the real problem.) I have commented out every `free` at this point just to be sure and still the problem remains. I don't know what a "buffer overrun" is, I will make my research and I will reply you when I try. The GC does not exist as I'm `betterC`. If I were in your shoes I'd use Valgrind / Memcheck to try to find the real cause of the problem. Chances are, it may have nothing to do with the bit of code you quoted at all. You could try to insert extra malloc/free's in various places around the code (in places along the code path, but unrelated to the problematic code) to see if that changes the behaviour of the bug. If it does, your corruption is likely somewhere other than the _ptr code you showed. T Thanks for the advice! I already used Valgrind before I bother you guys but because at this point of development, I didn't cared about freeing the memory, "valgrind" points so many errors that it isn't useful to help me identify what's wrong.
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 19:14:47 UTC, H. S. Teoh wrote: My guess is that you have a double-free somewhere, or there's a buffer overrun. Or maybe some bad interaction with the GC, e.g. if you tried to free a pointer from the GC heap. (Note that this may not immediately show up; free() could've assumed that everything was OK when it has in fact messed up its internal data structures; the problem would only show up later on in code that's actually unrelated to the real problem.) Wait! I did searched on the web! I cannot find anything about a "buffer overrun"! I only find about "buffer overflow".
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 16:17:15 UTC, Richard (Rikki) Andrew Cattermole wrote: I would strongly suggest that you log all memory sizes that are allocated, and double check that you do free. Also turn on ASAN in ldc. http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html Aha! In the search I made about this problem, I've seen a thread talking about "sanatizer=address" but only in C. I searched how I can use this one on LDC and I wasn't able to find it so, thank you! Unfortunately, I didn't helped because it has the same kind of output with [Valgrind](https://valgrind.org/) so it isn't very useful in my case...
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 19:14:47 UTC, H. S. Teoh wrote: [...] My guess is that you have a double-free somewhere, or there's a buffer overrun. Or maybe some bad interaction with the GC, e.g. if you tried to free a pointer from the GC heap. That cant be a GC problem as rempas project is compiled with `-betterC`
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 18:59:21 UTC, rempas wrote: On Friday, 8 September 2023 at 16:02:36 UTC, Basile B. wrote: Could this be a problem of copy construction ? I don't think so. My idea was that if you dont have defined a copy constructor and if an instance is assigned to another, then that other instance share the same pointer, which can cause memory errors. To eliminate that risk and to detect where default post-blitting may happen you can add ```d @disable this(this); ``` to the struct.
Re: malloc error when trying to assign the returned pointer to a struct field
On Fri, Sep 08, 2023 at 06:59:21PM +, rempas via Digitalmars-d-learn wrote: > On Friday, 8 September 2023 at 16:02:36 UTC, Basile B. wrote: > > > > Could this be a problem of copy construction ? > > I don't think so. The assertion seems to be violated when `malloc` is used. > And when I assert the result in the `_ptr` field. Really weird... The error message looks to me like a corruption of the malloc heap. These kinds of bugs are very hard to trace, because they may go undetected and only show up in specific circumstances, so small perturbations of completely unrelated code may make the bug appear or disappear -- just because the bug doesn't show up when you disable some code does not prove that that's where the problem is; it could be that corruption is still happening, it just so happens that it goes unnoticed when the behaviour of the code changes slightly. My guess is that you have a double-free somewhere, or there's a buffer overrun. Or maybe some bad interaction with the GC, e.g. if you tried to free a pointer from the GC heap. (Note that this may not immediately show up; free() could've assumed that everything was OK when it has in fact messed up its internal data structures; the problem would only show up later on in code that's actually unrelated to the real problem.) If I were in your shoes I'd use Valgrind / Memcheck to try to find the real cause of the problem. Chances are, it may have nothing to do with the bit of code you quoted at all. You could try to insert extra malloc/free's in various places around the code (in places along the code path, but unrelated to the problematic code) to see if that changes the behaviour of the bug. If it does, your corruption is likely somewhere other than the _ptr code you showed. T -- If the comments and the code disagree, it's likely that *both* are wrong. -- Christopher
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 16:02:36 UTC, Basile B. wrote: Could this be a problem of copy construction ? I don't think so. The assertion seems to be violated when `malloc` is used. And when I assert the result in the `_ptr` field. Really weird...
Re: malloc error when trying to assign the returned pointer to a struct field
On 08/09/2023 7:59 PM, rempas wrote: |Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed: (old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)| I would strongly suggest that you log all memory sizes that are allocated, and double check that you do free. Also turn on ASAN in ldc. http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote: I do have the following struct: [...] Is there any possible that there is a compiler bug? I do use ldc2 and `betterC`! Could this be a problem of copy construction ?
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote: [I do have ... I do use ldc2 and `betterC`!] For anyone who is still following, first of all thanks! Second, I have bad news and good news! The bad news is that I tested in an Linux Mint system (mine is an Arch Linux) and the it still doesn't work. So unless it's a bug in EVERY glibc on Linux, it will take me days to find it at best (I can feel the burnout coming!)... The good news is that I have [uploaded](https://codeberg.org/rempas/nemesis) the source code and as I have already asked for help for that codebase in the past and I'll probably ask in the future, I now have a proper codebase to showcase when the "smallest possible example" is not enough! More specifically, look at the showcases [Vec](https://codeberg.org/rempas/nemesis/src/branch/dev/source/erebos/structs/vector.d#L44) struct and the [template](https://codeberg.org/rempas/nemesis/src/branch/dev/source/erebos/structs/array.d#L38) that it expands on its body! The test that fails is for a function called [split](https://codeberg.org/rempas/nemesis/src/branch/dev/source/erebos/algorithms/split.d#L6) and it is located [here](https://codeberg.org/rempas/nemesis/src/branch/dev/source/tests/algorithms/test_split.d#L21) (I specifically point to the line that fails inside the test). Once again, thank you all for the support!
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 14:40:13 UTC, Richard (Rikki)
Andrew Cattermole wrote:
No, for this you need ModuleInfo. The order is sequential on
what it sees first.
Personally I test using full D rather than -betterC.
For dub:
```json
"configurations": [
{
"name": "library",
"targetType": "dynamicLibrary",
"versions": [
"DynamicSideroBase"
],
"buildOptions": [
"betterC"
]
},
{
"name": "static",
"targetType": "staticLibrary",
"buildOptions": [
"betterC"
]
},
{
"name": "unittest"
},
{
"name": "executable",
"targetType": "executable"
}
]
```
Thank you for all the information! I really appreciate it my
friend!
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 14:50:17 UTC, Kagamin wrote: Did you run this example program above? Does it crash? I didn't as I suppose that it would had no problems as it works for you. Either that, or it will be indeed a problem with my Glibc. I did run it now after your reply and it works without any problems!
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 13:32:00 UTC, rempas wrote:
On Friday, 8 September 2023 at 13:05:47 UTC, evilrat wrote:
```d
import core.stdc.stdlib;
import core.stdc.stdio;
alias u64 = ulong;
alias i64 = long;
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
u64 _len = 0;
public:
/* Create a vector by just allocating memory for it. The
null terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
}
ref T opIndex(size_t idx) { return _ptr[idx]; }
}
extern(C)
void main()
//unittest
{
enum el = 3;
auto vec = Vec!char(10);
assert(vec._ptr);
vec[el] = 'h';
assert(vec[el] == 'h');
printf("ptr = %p\n", vec._ptr);
printf("vec ptr = %p\n", [el]);
printf("vec local = %p\n", );
printf("vec[%d] = %c\n", el, vec[el]);
foreach (i; 0..vec._cap) {
printf("-");
}
printf("\n");
foreach (i; 0..vec._cap) {
printf("%d", vec[i]);
}
printf("\n");
printf("run ok\n");
}
```
ldc2 -betterC -run membug.d
output
```
ptr = 0x55cb701de2a0
vec ptr = 0x55cb701de2a3
vec local = 0x7fffa1542258
vec[3] = h
--
00010400
run ok
```
I have made a search on the web and I found out one thread that
pointed out that it may be a Glibc error. However, because like
I said the problem only happens when I assign the returned
value to the `_ptr` field, I just wanted to post here in case
someone has a similar experience and if it's a compiler bug in
which case, we should report it.
Did you run this example program above? Does it crash?
Re: malloc error when trying to assign the returned pointer to a struct field
On 09/09/2023 2:20 AM, rempas wrote:
Do they have automatic symbol order resolution? Which is, testing
symbols that other symbol depend on first? Or is it random?
No, for this you need ModuleInfo. The order is sequential on what it
sees first.
Personally I test using full D rather than -betterC.
For dub:
```json
"configurations": [
{
"name": "library",
"targetType": "dynamicLibrary",
"versions": [
"DynamicSideroBase"
],
"buildOptions": [
"betterC"
]
},
{
"name": "static",
"targetType": "staticLibrary",
"buildOptions": [
"betterC"
]
},
{
"name": "unittest"
},
{
"name": "executable",
"targetType": "executable"
}
]
```
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 13:34:42 UTC, Richard (Rikki)
Andrew Cattermole wrote:
In case you didn't know, all you need to get unittests working
in -betterC is:
```d
foreach (module_; allModules) {
foreach (unitTest; __traits(getUnitTests,
module_)) {
unitTest();
}
}
```
You'd need to provide allModules somehow, like dub does.
https://github.com/dlang/dub/blob/2ea883833adf085095b07a7dba8250fb3db79a71/source/dub/project.d#L1937
Oh wow!!! How did I missed that? It's either recent (past months)
or I'm blind, lol!
As I don't remember unittests however, I would like to as. Do
they have automatic symbol order resolution? Which is, testing
symbols that other symbol depend on first? Or is it random?
Re: malloc error when trying to assign the returned pointer to a struct field
In case you didn't know, all you need to get unittests working in
-betterC is:
```d
foreach (module_; allModules) {
foreach (unitTest; __traits(getUnitTests,
module_)) {
unitTest();
}
}
```
You'd need to provide allModules somehow, like dub does.
https://github.com/dlang/dub/blob/2ea883833adf085095b07a7dba8250fb3db79a71/source/dub/project.d#L1937
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 13:05:47 UTC, evilrat wrote:
You run with -unittest compiler flag? Well, that does nothing
for me with betterc (without it is ok).
I did stupid and unsafe things like malloc(0) and writing out
of bounds but still no crash, it works fine.
I guess it depends on your libc, tested this on ubuntu 23.04
with gcc12 install and ldc 1.32.2
```d
import core.stdc.stdlib;
import core.stdc.stdio;
alias u64 = ulong;
alias i64 = long;
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
u64 _len = 0;
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
}
ref T opIndex(size_t idx) { return _ptr[idx]; }
}
extern(C)
void main()
//unittest
{
enum el = 3;
auto vec = Vec!char(10);
assert(vec._ptr);
vec[el] = 'h';
assert(vec[el] == 'h');
printf("ptr = %p\n", vec._ptr);
printf("vec ptr = %p\n", [el]);
printf("vec local = %p\n", );
printf("vec[%d] = %c\n", el, vec[el]);
foreach (i; 0..vec._cap) {
printf("-");
}
printf("\n");
foreach (i; 0..vec._cap) {
printf("%d", vec[i]);
}
printf("\n");
printf("run ok\n");
}
```
ldc2 -betterC -run membug.d
output
```
ptr = 0x55cb701de2a0
vec ptr = 0x55cb701de2a3
vec local = 0x7fffa1542258
vec[3] = h
--
00010400
run ok
```
As D's "uninttest" feature is disabled on BetterC, I have wrote
my own testing suit (with is very simple). I just said that to
point out that I'm testing this data structure (along side other
things) and I cannot find anything wrong.
I have made a search on the web and I found out one thread that
pointed out that it may be a Glibc error. However, because like I
said the problem only happens when I assign the returned value to
the `_ptr` field, I just wanted to post here in case someone has
a similar experience and if it's a compiler bug in which case, we
should report it.
I will use a VM to test my code in another environment and I'll
be back to report!
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:
I do have the following struct:
```d
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
}
}
```
That's some minimal code that I do have just to showcase it.
So, some times, this work will works, some others, it will give
me the following error:
`Fatal glibc error: malloc.c:2594 (sysmalloc): assertion
failed: (old_top == initial_top (av) && old_size == 0) ||
((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top)
&& ((unsigned long) old_end & (pagesize - 1)) == 0)`
The problem seems to happen when the pointer that is returned
from `malloc` is assigned to the `_ptr` field. If I just assign
it to a variable and don't assign anything to `_ptr`, it will
work!
Is there any possible that there is a compiler bug? I do use
ldc2 and `betterC`!
I've had an error message like that before. This was the answer:
https://stackoverflow.com/questions/46803671/sysmalloc-assertion
Without additional code it's hard to say if that's your problem.
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 11:50:52 UTC, rempas wrote:
That's interesting, I wasn't able to find something else! The
bug happens when I run the testing suit and well... the tests
before pass so I cannot find anything that goes wrong except
for the fact that I do not free the memory that is allocated
(on purpose).
You run with -unittest compiler flag? Well, that does nothing for
me with betterc (without it is ok).
I did stupid and unsafe things like malloc(0) and writing out of
bounds but still no crash, it works fine.
I guess it depends on your libc, tested this on ubuntu 23.04 with
gcc12 install and ldc 1.32.2
```d
import core.stdc.stdlib;
import core.stdc.stdio;
alias u64 = ulong;
alias i64 = long;
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we can
store
u64 _len = 0;
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional space
for the null terminator
this._ptr = cast(T*)malloc(size);
}
ref T opIndex(size_t idx) { return _ptr[idx]; }
}
extern(C)
void main()
//unittest
{
enum el = 3;
auto vec = Vec!char(10);
assert(vec._ptr);
vec[el] = 'h';
assert(vec[el] == 'h');
printf("ptr = %p\n", vec._ptr);
printf("vec ptr = %p\n", [el]);
printf("vec local = %p\n", );
printf("vec[%d] = %c\n", el, vec[el]);
foreach (i; 0..vec._cap) {
printf("-");
}
printf("\n");
foreach (i; 0..vec._cap) {
printf("%d", vec[i]);
}
printf("\n");
printf("run ok\n");
}
```
ldc2 -betterC -run membug.d
output
```
ptr = 0x55cb701de2a0
vec ptr = 0x55cb701de2a3
vec local = 0x7fffa1542258
vec[3] = h
--
00010400
run ok
```
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 09:25:59 UTC, Hipreme wrote: Hello, not completely unrelated to your problem, I have also done something like that, and when you're in D, don't use simply a pointer and length like that, use the `slice` operator. See references: https://tour.dlang.org/tour/en/basics/slices https://dlang.org/spec/operatoroverloading.html#array-ops For example, you can make your pointer a lot safer by doing: ```d size_t length = 5; int* pointer = cast(int*)malloc(int.sizeof * length); //Don't int[] mallocArray = (cast(int*)malloc(int.sizeof * length))[0..length]; //Do ``` On the second way, you'll get bound checks, thus, making it safer. Also, no need to keep track of your length separately anymore. This is good practice in D language and you'll find yourself using this instead in the future. And yes, this works in betterC, it is a simple runtime check, completely `@nogc @safe nothrow` and every other kind of thing you would want. Thank you for the reply! Using slices will give me the same result (my code has another field that I forgot to mention btw and it's `_len` (which holds the length that is used from the capacity). However, this will just make the API worse to use so I choose to not use slices in this case as using separate fields is more convenient.
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 09:07:12 UTC, evilrat wrote: Hard to tell from that code but it is quite unlikely there is a compiler bug in such simple use case. I assume you already tried debugging your program, right? Yep! I have spent days and it's these kinds of bugs that burn me off and make me want to give up. So how about to diagnose a bit more, what if you enforce check before malloc that size>0, In the real code, there is a check! I just removed it from this example to make the code more clear. and second - from that example it is unclear how you are using that struct, so maybe add else statement static assert to see if it is misused somewhere else in your codebase? That's interesting, I wasn't able to find something else! The bug happens when I run the testing suit and well... the tests before pass so I cannot find anything that goes wrong except for the fact that I do not free the memory that is allocated (on purpose). But still, why would the bug be there when I assign the `_ptr` on the field and it won't happen if I don't assign it and just use a regular local variable instead (and don't assign anything to the `_ptr` field)? Also this example doesn't have len field, depending on how you use with regard to cap this could be a source of problems too. Oh right, I forgot about that! The `_len` field normally exists and it's it's also an unsigned 64-bit (u64) number! Yeah, the code is a little bit changed from the original to make it more simple for you guys...
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:
I do have the following struct:
```d
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
}
}
```
That's some minimal code that I do have just to showcase it.
So, some times, this work will works, some others, it will give
me the following error:
`Fatal glibc error: malloc.c:2594 (sysmalloc): assertion
failed: (old_top == initial_top (av) && old_size == 0) ||
((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top)
&& ((unsigned long) old_end & (pagesize - 1)) == 0)`
The problem seems to happen when the pointer that is returned
from `malloc` is assigned to the `_ptr` field. If I just assign
it to a variable and don't assign anything to `_ptr`, it will
work!
Is there any possible that there is a compiler bug? I do use
ldc2 and `betterC`!
Hello, not completely unrelated to your problem, I have also done
something like that, and when you're in D, don't use simply a
pointer and length like that, use the `slice` operator.
See references:
https://tour.dlang.org/tour/en/basics/slices
https://dlang.org/spec/operatoroverloading.html#array-ops
For example, you can make your pointer a lot safer by doing:
```d
size_t length = 5;
int* pointer = cast(int*)malloc(int.sizeof * length); //Don't
int[] mallocArray = (cast(int*)malloc(int.sizeof *
length))[0..length]; //Do
```
On the second way, you'll get bound checks, thus, making it
safer. Also, no need to keep track of your length separately
anymore.
This is good practice in D language and you'll find yourself
using this instead in the future.
And yes, this works in betterC, it is a simple runtime check,
completely `@nogc @safe nothrow` and every other kind of thing
you would want.
Re: malloc error when trying to assign the returned pointer to a struct field
On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:
I do have the following struct:
```d
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we
can store
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional
space for the null terminator
this._ptr = cast(T*)malloc(size);
}
}
```
That's some minimal code that I do have just to showcase it.
So, some times, this work will works, some others, it will give
me the following error:
`Fatal glibc error: malloc.c:2594 (sysmalloc): assertion
failed: (old_top == initial_top (av) && old_size == 0) ||
((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top)
&& ((unsigned long) old_end & (pagesize - 1)) == 0)`
The problem seems to happen when the pointer that is returned
from `malloc` is assigned to the `_ptr` field. If I just assign
it to a variable and don't assign anything to `_ptr`, it will
work!
Is there any possible that there is a compiler bug? I do use
ldc2 and `betterC`!
Hard to tell from that code but it is quite unlikely there is a
compiler bug in such simple use case.
I assume you already tried debugging your program, right?
So how about to diagnose a bit more, what if you enforce check
before malloc that size>0, and second - from that example it is
unclear how you are using that struct, so maybe add else
statement static assert to see if it is misused somewhere else in
your codebase?
Also this example doesn't have len field, depending on how you
use with regard to cap this could be a source of problems too.
malloc error when trying to assign the returned pointer to a struct field
I do have the following struct:
```d
struct Vec(T) {
private:
T* _ptr = null; // The pointer to the data
u64 _cap = 0; // Total amount of elements (not bytes) we can
store
public:
/* Create a vector by just allocating memory for it. The null
terminator is not set for
strings as, the vector is considered empty and we should
first push something to it
in order to use it! */
this(i64 size) {
this._len = 0;
this._cap = size;
static if (is(T == char)) { size += 1; } // Additional space
for the null terminator
this._ptr = cast(T*)malloc(size);
}
}
```
That's some minimal code that I do have just to showcase it. So,
some times, this work will works, some others, it will give me
the following error:
`Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed:
(old_top == initial_top (av) && old_size == 0) || ((unsigned
long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned
long) old_end & (pagesize - 1)) == 0)`
The problem seems to happen when the pointer that is returned
from `malloc` is assigned to the `_ptr` field. If I just assign
it to a variable and don't assign anything to `_ptr`, it will
work!
Is there any possible that there is a compiler bug? I do use ldc2
and `betterC`!
