Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 25 July 2022 at 02:59:41 UTC, electricface wrote: https://forum.dlang.org/post/gsztclsvxdhnvfhbi...@forum.dlang.org Looking forward to part 2 and part 3 Me, too! I'm hoping to make time for this and a couple of other series that are long overdue for updates before the end of the year.
From the D Blog -- Interfacing D with C: Strings Part One
https://forum.dlang.org/post/gsztclsvxdhnvfhbi...@forum.dlang.org On Thursday, 27 May 2021 at 03:54:11 UTC, Виталий Фадеев wrote: On Thursday, 27 May 2021 at 03:52:32 UTC, Виталий Фадеев wrote: On Thursday, 27 May 2021 at 03:40:02 UTC, Виталий Фадеев wrote: On Wednesday, 26 May 2021 at 16:35:36 UTC, Ali Çehreli wrote: On 5/25/21 9:00 PM, Виталий Фадеев wrote: // cString[ $-1 ] != '\'0'; // cString[ $ ] != '\'0'; Looking forward to part 2 and part 3
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Thursday, 27 May 2021 at 03:52:32 UTC, Виталий Фадеев wrote: On Thursday, 27 May 2021 at 03:40:02 UTC, Виталий Фадеев wrote: On Wednesday, 26 May 2021 at 16:35:36 UTC, Ali Çehreli wrote: On 5/25/21 9:00 PM, Виталий Фадеев wrote: // cString[ $-1 ] != '\'0'; // cString[ $ ] != '\'0';
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Thursday, 27 May 2021 at 03:40:02 UTC, Виталий Фадеев wrote:
On Wednesday, 26 May 2021 at 16:35:36 UTC, Ali Çehreli wrote:
On 5/25/21 9:00 PM, Виталий Фадеев wrote:
> immutable(char)* toStringz( ref string s )
> {
> if ( s.capacity <= s.length )
> s.reserve( s.length + 1 );
>
> char* cptr = cast( char* ) s.ptr; // C ptr
> char* zptr = cptr + s.length; // zero ptr
> *zptr = '\0';
That's undefined behavior because that location does not
belong to the string. Here is an example that defeats the
proposed toStringz:
void main()
{
string s;
s = "D string";
auto c_string = toStringz( s );
auto other = s;
other ~= 'X';// <-- Seemingly unrelated operation
// ...
}
puts accesses that unrelated 'X' and more bytes after that:
C string: D stringX1^
Ali
Yes. True.
reserve/capacity - not for all cases.
Zero terminator not keeped after concatenate source string with
other string.
auto dString = "D string" ~ 2.to!string;
auto cString = dString.toStringz();
dString = dString ~ "new tail";
// cString[ $-1 ] != '\'0';
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Wednesday, 26 May 2021 at 16:35:36 UTC, Ali Çehreli wrote:
On 5/25/21 9:00 PM, Виталий Фадеев wrote:
> immutable(char)* toStringz( ref string s )
> {
> if ( s.capacity <= s.length )
> s.reserve( s.length + 1 );
>
> char* cptr = cast( char* ) s.ptr; // C ptr
> char* zptr = cptr + s.length; // zero ptr
> *zptr = '\0';
That's undefined behavior because that location does not belong
to the string. Here is an example that defeats the proposed
toStringz:
void main()
{
string s;
s = "D string";
auto c_string = toStringz( s );
auto other = s;
other ~= 'X';// <-- Seemingly unrelated operation
// ...
}
puts accesses that unrelated 'X' and more bytes after that:
C string: D stringX1^
Ali
Yes. True.
reserve/capacity - not for all cases.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On 5/25/21 9:00 PM, Виталий Фадеев wrote:
> immutable(char)* toStringz( ref string s )
> {
> if ( s.capacity <= s.length )
> s.reserve( s.length + 1 );
>
> char* cptr = cast( char* ) s.ptr; // C ptr
> char* zptr = cptr + s.length; // zero ptr
> *zptr = '\0';
That's undefined behavior because that location does not belong to the
string. Here is an example that defeats the proposed toStringz:
void main()
{
string s;
s = "D string";
auto c_string = toStringz( s );
auto other = s;
other ~= 'X';// <-- Seemingly unrelated operation
// ...
}
puts accesses that unrelated 'X' and more bytes after that:
C string: D stringX1^
Ali
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Wednesday, 26 May 2021 at 04:27:16 UTC, Виталий Фадеев wrote: On Wednesday, 26 May 2021 at 04:00:17 UTC, Виталий Фадеев wrote: On Monday, 24 May 2021 at 14:02:14 UTC, Mike Parker wrote: The blog: https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/ Test code: https://run.dlang.io/is/xZwwtw Example for using reserve. Test code 2: https://run.dlang.io/is/aQsr8n Pull request to std.string: https://github.com/dlang/phobos/pull/8111 Review code, please.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Wednesday, 26 May 2021 at 04:00:17 UTC, Виталий Фадеев wrote: On Monday, 24 May 2021 at 14:02:14 UTC, Mike Parker wrote: The blog: https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/ Test code: https://run.dlang.io/is/xZwwtw Example for using reserve. Test code 2: https://run.dlang.io/is/aQsr8n
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 14:02:14 UTC, Mike Parker wrote:
The blog:
https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/
Good!
toStringz()
Technically we can use 'reserve()' for reserve memory. Possible,
memory already reserved.
s.reserve( s.length + 1 );
Then we can set trailing zero.
s[ $ ] = '\0';
And return pointer.
return s.ptr;
In this case we prevent memory allocation. Operations will be
faster.
In other case we cam:
auto copy = new char[s.length + 1];
copy[0 .. s.length] = s[];
copy[s.length] = 0;
return (copy)[0];
Example:
immutable(char)* toStringz( ref string s )
{
if ( s.capacity <= s.length )
s.reserve( s.length + 1 );
char* cptr = cast( char* ) s.ptr; // C ptr
char* zptr = cptr + s.length; // zero ptr
*zptr = '\0';
return cast( immutable(char)* ) cptr;
}
Test code: https://run.dlang.io/is/xZwwtw
Re: From the D Blog -- Interfacing D with C: Strings Part One
哦,原来如此.谢谢.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Tuesday, 25 May 2021 at 23:57:24 UTC, zjh wrote: On Tuesday, 25 May 2021 at 21:14:24 UTC, Vinod K Chandran wrote: On Tuesday, 25 May 2021 at 00:41:50 UTC, zjh wrote: switch to ie. other websites is no problem. I don't know why. I too faced this problem in my laptop's chrome. The `js` of this website must have problems. I faced the problem too long. other websites no problem. the website is ok, it's because, you and me in China mainland, and we just behind that wall, you just need a VPN proxy! as in Chinese: 网站没问题，你需要梯子！
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Tuesday, 25 May 2021 at 21:14:24 UTC, Vinod K Chandran wrote: On Tuesday, 25 May 2021 at 00:41:50 UTC, zjh wrote: switch to ie. other websites is no problem. I don't know why. I too faced this problem in my laptop's chrome. The `js` of this website must have problems. I faced the problem too long. other websites no problem.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Tuesday, 25 May 2021 at 00:41:50 UTC, zjh wrote: switch to ie. other websites is no problem. I don't know why. I too faced this problem in my laptop's chrome. I started reading the article in the middle of my work (In my PC's chrome browser). Then after read half of the page, I bookmarked that page and after a few hours, at the mid night, I try to open the bookmarked page in my laptop's chrome browser. But page is partially opened and no text from the article is visible. Only the links in side bar is visible, but they are not clickable. The progress bar is still rotating. So I gave up and did some other tasks. Then next morning, I have opened the page in my PC's chrome and saved the web page with "Save As" option and send those file to my laptop via dropbox. Then I tried to open that saved webpage in laptop's chrome. But it did the same thing. No text is visible. It's some time showing a alert that this web page is taking too many time to load, wait or kill ?. Then I tested that saved web page in Internet Explorer and it opened without any problem.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Tuesday, 25 May 2021 at 00:58:31 UTC, Steven Schveighoffer wrote: On 5/24/21 8:38 PM, Mike Parker wrote: OK, I'm just concerned people will see the pattern: ```d somecfunc(str.toStringz); ``` and think that's the end of it. Yeah. Good point. I've updated the post.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On 5/24/21 11:30 PM, surlymoor wrote: On Tuesday, 25 May 2021 at 00:58:31 UTC, Steven Schveighoffer wrote: OK, I'm just concerned people will see the pattern: ```d somecfunc(str.toStringz); ``` and think that's the end of it. Pretty sure its documentation has a conspicuous warning regarding that. It does, and that's [sometimes ignored](https://forum.dlang.org/post/bogwusbqfewqifjlf...@forum.dlang.org), even by [seasoned veterans](https://forum.dlang.org/post/rorsrk$1tdu$1...@digitalmars.com). Anyway, it doesn't hurt to identify the possible memory problems that a function can have when recommending it. -Steve
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Tuesday, 25 May 2021 at 00:58:31 UTC, Steven Schveighoffer wrote: OK, I'm just concerned people will see the pattern: ```d somecfunc(str.toStringz); ``` and think that's the end of it. -Steve Pretty sure its documentation has a conspicuous warning regarding that.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On 5/24/21 8:38 PM, Mike Parker wrote: On Monday, 24 May 2021 at 16:16:53 UTC, Steven Schveighoffer wrote: This leaves a GC-collectible allocation in C land. For `puts`, it's fine, as the data is not used past the call, but in something else that might keep it somewhere not accessible to the GC, you'll want to assign that to a variable that lasts as long as the resource is used. That's what I'm referring to in the conclusion where I say this about what's going to be in Part Two: how to avoid a potential problem spot that can arise when passing GC-allocated D strings to C I'll cover approaches to maintaining a reference, like `GC.addRoot`, and emphasize that it applies to any GC-allocated memory, not just strings. OK, I'm just concerned people will see the pattern: ```d somecfunc(str.toStringz); ``` and think that's the end of it. -Steve
Re: From the D Blog -- Interfacing D with C: Strings Part One
switch to ie. other websites is no problem. I don't know why.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 16:16:53 UTC, Steven Schveighoffer wrote: Nice article! Thanks! Note that there is a huge pitfall awaiting you if you use `toStringz`: garbage collection. You may want to amend the article to identify this pitfall. And I'm not talking about requiring `@nogc`, I'm talking about the GC collecting the data while C is still using it. In your example: ```d puts(s1.toStringz()); ``` This leaves a GC-collectible allocation in C land. For `puts`, it's fine, as the data is not used past the call, but in something else that might keep it somewhere not accessible to the GC, you'll want to assign that to a variable that lasts as long as the resource is used. That's what I'm referring to in the conclusion where I say this about what's going to be in Part Two: how to avoid a potential problem spot that can arise when passing GC-allocated D strings to C I'll cover approaches to maintaining a reference, like `GC.addRoot`, and emphasize that it applies to any GC-allocated memory, not just strings.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 16:16:53 UTC, Steven Schveighoffer
wrote:
On 5/24/21 10:02 AM, Mike Parker wrote:
The latest post in the D and C series dives into the weeds of
D and C strings: how they're implemented, when you need to
NUL-terminate your D strings and when you don't, and how the
storage of literals in memory allows you to avoid NUL
termination in one case you might not have considered and
another case that you shouldn't rely on but can in practice
with the current compilers.
There are at least two more posts worth of information to go
into on this topic, but everything in this post is enough to
cover many use cases of D to C string interop.
The blog:
https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/
Reddit:
https://www.reddit.com/r/programming/comments/njyf76/interfacing_d_with_c_strings_part_one/
Nice article!
Note that there is a huge pitfall awaiting you if you use
`toStringz`: garbage collection. You may want to amend the
article to identify this pitfall.
And I'm not talking about requiring `@nogc`, I'm talking about
the GC collecting the data while C is still using it.
In your example:
```d
puts(s1.toStringz());
```
This leaves a GC-collectible allocation in C land. For `puts`,
it's fine, as the data is not used past the call, but in
something else that might keep it somewhere not accessible to
the GC, you'll want to assign that to a variable that lasts as
long as the resource is used.
-Steve
It’s worse than that, no? If the only reference to GC data isn’t
on the stack of a tracked thread, in GC allocated memory or in a
tracked root then it can be freed. Even in D:
void foo(int* a) {
int** b = cast(int**) malloc((int*).sizeof);
*b = a;
a = null;
GC.collect();
**b = 4; // whoops!!
}
foo(new int);
Right? Obviously that collection could be from calling another
function (e.g. a callback from C to D code) or from another
thread. Or am I missing something?
Re: From the D Blog -- Interfacing D with C: Strings Part One
On 5/24/21 10:02 AM, Mike Parker wrote: The latest post in the D and C series dives into the weeds of D and C strings: how they're implemented, when you need to NUL-terminate your D strings and when you don't, and how the storage of literals in memory allows you to avoid NUL termination in one case you might not have considered and another case that you shouldn't rely on but can in practice with the current compilers. There are at least two more posts worth of information to go into on this topic, but everything in this post is enough to cover many use cases of D to C string interop. The blog: https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/ Reddit: https://www.reddit.com/r/programming/comments/njyf76/interfacing_d_with_c_strings_part_one/ Nice article! Note that there is a huge pitfall awaiting you if you use `toStringz`: garbage collection. You may want to amend the article to identify this pitfall. And I'm not talking about requiring `@nogc`, I'm talking about the GC collecting the data while C is still using it. In your example: ```d puts(s1.toStringz()); ``` This leaves a GC-collectible allocation in C land. For `puts`, it's fine, as the data is not used past the call, but in something else that might keep it somewhere not accessible to the GC, you'll want to assign that to a variable that lasts as long as the resource is used. -Steve
Re: From the D Blog -- Interfacing D with C: Strings Part One
On 25/05/2021 3:39 AM, zjh wrote: use chrome ,cannot open the DEV. press `F12` of no use. `Alt+U`of no use. continue rotating the small circle.You can do nothing. There is something seriously wrong with your install then. It isn't related to the website itself.
Re: From the D Blog -- Interfacing D with C: Strings Part One
Use the dev tools, network + performance tabs could be very useful information for debugging this.
Re: From the D Blog -- Interfacing D with C: Strings Part One
use chrome ,cannot open the DEV. press `F12` of no use. `Alt+U`of no use. continue rotating the small circle.You can do nothing.
Re: From the D Blog -- Interfacing D with C: Strings Part One
but ie can open it. it's strange.
Re: From the D Blog -- Interfacing D with C: Strings Part One
loading all the time, loading for a long time, but also only part of the load, and still turn the circle. just like there is a BIG `js` file .
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 14:02:14 UTC, Mike Parker wrote: The latest post in the D and C series dives into the weeds of D and C strings: Thanks, I wasn't even aware of this. However, I wish the behavior would be the same between string bounded storage variables and string literals. I think this would be a good use case for multiple alias this where we would map d types to c types and vice versa. For strings, it would in case of D->C just add the \0 at the end and for C->D it would be subtracted from the string. There are at least two more posts worth of information to go into on this topic, but everything in this post is enough to cover many use cases of D to C string interop. Which posts did you think of?
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 15:10:46 UTC, sighoya wrote: There are at least two more posts worth of information to go into on this topic, but everything in this post is enough to cover many use cases of D to C string interop. Which posts did you think of? I describe them in the Conclusion: In Part Two, we’ll look at how mutability, immutability, and constness come into the picture, how to avoid a potential problem spot that can arise when passing GC-allocated D strings to C, and how to get D strings from C strings. We’ll save encoding for Part Three.
Re: From the D Blog -- Interfacing D with C: Strings Part One
On Monday, 24 May 2021 at 15:03:20 UTC, zjh wrote: I always think there is something wrong with the JS of ` D blog site`. I can't use chrome to open it. I'm looking at it in Chrome right now. Are you saying it doesn't open at all for you or there are rendering issues, or...?
Re: From the D Blog -- Interfacing D with C: Strings Part One
I always think there is something wrong with the JS of ` D blog site`. I can't use chrome to open it.
Re: From the D Blog -- Interfacing D with C: Strings Part One
I always think there is something wrong with the JS of ` D blog site`. I can't use chrome to open it.
From the D Blog -- Interfacing D with C: Strings Part One
The latest post in the D and C series dives into the weeds of D and C strings: how they're implemented, when you need to NUL-terminate your D strings and when you don't, and how the storage of literals in memory allows you to avoid NUL termination in one case you might not have considered and another case that you shouldn't rely on but can in practice with the current compilers. There are at least two more posts worth of information to go into on this topic, but everything in this post is enough to cover many use cases of D to C string interop. The blog: https://dlang.org/blog/2021/05/24/interfacing-d-with-c-strings-part-one/ Reddit: https://www.reddit.com/r/programming/comments/njyf76/interfacing_d_with_c_strings_part_one/
