Re: Dealing with property function and operator overloads
On 2011-10-03 20:57, Andrej Mitrovic wrote:
Looks like I can use some D tricks for this:
import std.stdio;
struct Point
{
int x, y;
void opOpAssign(string op)(int rhs)
{
mixin ("x = x " ~ op ~ " rhs;");
mixin ("y = y " ~ op ~ " rhs;");
}
}
struct Wrapped(T)
{
T payload;
alias payload this;
void delegate() dg;
@property void changed(void delegate() dg)
{
this.dg = dg;
}
void opOpAssign(string op)(int rhs)
{
payload.opOpAssign!op(rhs);
dg();
}
}
struct Wrapper
{
this(int x)
{
point.changed =¬ifyChanged;
}
void notifyChanged()
{
writeln("changed!");
}
public Wrapped!Point point;
}
void main()
{
auto wrap = Wrapper(1);
wrap.point = Point(1, 1);
assert(wrap.point == Point(1, 1));
wrap.point += 1;
assert(wrap.point == Point(2, 2));
}
Pretty cool. I might even be able to write a Wrapped() template that
searches for all operator overloads of a type and creates forwarding
functions.
No "alias this" in the wrapper?
--
/Jacob Carlborg
Re: Dealing with property function and operator overloads
Steven Schveighoffer: > I suspect operator overloads are going to be a large hole in the interface > design of many objects, but at least they won't be exploitable once > compiled. Just after the introduction of the new operator overload syntax I have suggested to introduce strict tests to statically forbid strings that don't contain one of the correct accepted operators. Walter didn't appreciate this idea :-( Bye, bearophile
Re: Dealing with property function and operator overloads
On 10/3/11, Steven Schveighoffer wrote: > Even so, it's better for people who are not familiar with the language to > see correct code vs. code open to exploitation. I can see your point. I'll make some simple constraints when I post code samples like this from now on.
Re: Dealing with property function and operator overloads
On Mon, 03 Oct 2011 16:07:19 -0400, Andrej Mitrovic wrote: On 10/3/11, Steven Schveighoffer wrote: Probably slightly off topic, but be very careful with operator overloads without using constraints. For example, I can do some weird things to your struct: Point p; p.opOpAssign!"*x; y+="(5); I suspect operator overloads are going to be a large hole in the interface design of many objects, but at least they won't be exploitable once compiled. -Steve Generally when I post examples I won't add any constraints, this is to make it easier for other people to read the code. All of the ops have constraints in my code. Even so, it's better for people who are not familiar with the language to see correct code vs. code open to exploitation. I've argued in the past that we need some boilerplate constraints for operators (like isValidOpString). Can't remember where that stands. -Steve
Re: Dealing with property function and operator overloads
On 10/3/11, Steven Schveighoffer wrote: > Probably slightly off topic, but be very careful with operator overloads > without using constraints. > > For example, I can do some weird things to your struct: > > Point p; > > p.opOpAssign!"*x; y+="(5); > > I suspect operator overloads are going to be a large hole in the interface > design of many objects, but at least they won't be exploitable once > compiled. > > -Steve > Generally when I post examples I won't add any constraints, this is to make it easier for other people to read the code. All of the ops have constraints in my code.
Re: Dealing with property function and operator overloads
On Mon, 03 Oct 2011 14:57:33 -0400, Andrej Mitrovic
wrote:
Looks like I can use some D tricks for this:
import std.stdio;
struct Point
{
int x, y;
void opOpAssign(string op)(int rhs)
{
mixin ("x = x " ~ op ~ " rhs;");
mixin ("y = y " ~ op ~ " rhs;");
}
}
Probably slightly off topic, but be very careful with operator overloads
without using constraints.
For example, I can do some weird things to your struct:
Point p;
p.opOpAssign!"*x; y+="(5);
I suspect operator overloads are going to be a large hole in the interface
design of many objects, but at least they won't be exploitable once
compiled.
-Steve
Re: Dealing with property function and operator overloads
Forgot to add an opAssign in Wrapped as well:
void opAssign(T)(T rhs)
{
payload = rhs;
dg();
}
That takes care of assigning and ops that change the object's state.
Re: Dealing with property function and operator overloads
Looks like I can use some D tricks for this:
import std.stdio;
struct Point
{
int x, y;
void opOpAssign(string op)(int rhs)
{
mixin ("x = x " ~ op ~ " rhs;");
mixin ("y = y " ~ op ~ " rhs;");
}
}
struct Wrapped(T)
{
T payload;
alias payload this;
void delegate() dg;
@property void changed(void delegate() dg)
{
this.dg = dg;
}
void opOpAssign(string op)(int rhs)
{
payload.opOpAssign!op(rhs);
dg();
}
}
struct Wrapper
{
this(int x)
{
point.changed = ¬ifyChanged;
}
void notifyChanged()
{
writeln("changed!");
}
public Wrapped!Point point;
}
void main()
{
auto wrap = Wrapper(1);
wrap.point = Point(1, 1);
assert(wrap.point == Point(1, 1));
wrap.point += 1;
assert(wrap.point == Point(2, 2));
}
Pretty cool. I might even be able to write a Wrapped() template that
searches for all operator overloads of a type and creates forwarding
functions.
Re: std.socket - problems closing socket
On 03.10.2011 20:41, simendsjo wrote: It seems nginx is to blame here, and not me. I tried Lighttp and it works. It gives several EWOULDBLOCK, but I can just handle these again with no problem. I should have tried this sooner... I've used a lot of time trying to track down these problems :| Thanks for all your help - I'll update this thread if I find a solution to the nginx issue. Well, that was quick... Seems I was running a development version of nginx. I downloaded the stable version, and things work as expected - I can finally try to get some actual coding done :)
Re: std.socket - problems closing socket
On 03.10.2011 20:02, Regan Heath wrote: On Mon, 03 Oct 2011 17:33:57 +0100, simendsjo wrote: Yes. I've coded the client as follows: 1) start listening socket 2) wait for incoming connections or incoming data 3) receive(). If a socket returns 0 or -1, close it and process next with data 4) read fastcgi request from server 5) write fastcgi response 6) write fastcgi EndRequest (the server should now end the request) 7) if the application should close the request, send shutdown(send) 8) accept incoming connection 9) back to 2) FastCGI connections works in one of two ways: the server is responsible for closing the connections (supports mulitplexing) or the application should close the connection after a request has been sent. For the latter I send SocketShutdown.SEND after writing EndRequest in step 6), but it doesn't really matter as nginx doesn't support multiplexing. It closes the connection after each request anyway. I see the same result no matter what option I use. Ok, so your "client" (that you have coded) is also the "application" you refer to in the bit about FastCGI above? Or are there 2 components here, and are both written in D? It's just one component to handle FastCGI requests. I didn't want to rely on the external libfcgi. Does the fastcgi "EndRequest" close the socket/connection? If so, doing a socket.Shutdown /after/ this is not going to work as the socket has already been closed (which implicitly does a shutdown(BOTH)). In that case, try doing the shutdown /before/ the EndRequest, and make sure you also read any/all data remaining on the socket before doing the EndRequest/close. EndRequest doesn't really close the socket, it's just a message to the server telling that the full response is written (request handled). If the server (nginx) is responsible, it can reuse the connection to give other requests. If the server says that the application is responsible, shutdown(send) is called. This is part of the specification. The key question seems to be, at which point does nginx close the connection? and therefore, is there any unread data on the socket (at either end) when it does. If, for example, it flushes the response to the other end, but does not wait for it to be read, and closes the socket, you will get CONNRESET/ABORTED errors on the other end. > I'm running the exact same request and writing the exact same response for all queries, so there shouldn't be any unknown fields. I didn't mean unknown "field" I mean extra data of any kind, but I suspect you're using an API to form the requests etc so this is probably not the case. I also only get an error on <1/5 of the requests, and even when the error occurs, the response has been written completely to the browser. Ahh, ok, I believe the problem is simply the timing of the 'close/EndRequest'. Sometimes it happens /before/ the data has been completely read (1/5), other times after (4/5). R It seems nginx is to blame here, and not me. I tried Lighttp and it works. It gives several EWOULDBLOCK, but I can just handle these again with no problem. I should have tried this sooner... I've used a lot of time trying to track down these problems :| Thanks for all your help - I'll update this thread if I find a solution to the nginx issue.
Re: std.socket - problems closing socket
On Mon, 03 Oct 2011 17:33:57 +0100, simendsjo wrote: Yes. I've coded the client as follows: 1) start listening socket 2) wait for incoming connections or incoming data 3) receive(). If a socket returns 0 or -1, close it and process next with data 4) read fastcgi request from server 5) write fastcgi response 6) write fastcgi EndRequest (the server should now end the request) 7) if the application should close the request, send shutdown(send) 8) accept incoming connection 9) back to 2) FastCGI connections works in one of two ways: the server is responsible for closing the connections (supports mulitplexing) or the application should close the connection after a request has been sent. For the latter I send SocketShutdown.SEND after writing EndRequest in step 6), but it doesn't really matter as nginx doesn't support multiplexing. It closes the connection after each request anyway. I see the same result no matter what option I use. Ok, so your "client" (that you have coded) is also the "application" you refer to in the bit about FastCGI above? Or are there 2 components here, and are both written in D? Does the fastcgi "EndRequest" close the socket/connection? If so, doing a socket.Shutdown /after/ this is not going to work as the socket has already been closed (which implicitly does a shutdown(BOTH)). In that case, try doing the shutdown /before/ the EndRequest, and make sure you also read any/all data remaining on the socket before doing the EndRequest/close. The key question seems to be, at which point does nginx close the connection? and therefore, is there any unread data on the socket (at either end) when it does. If, for example, it flushes the response to the other end, but does not wait for it to be read, and closes the socket, you will get CONNRESET/ABORTED errors on the other end. I'm running the exact same request and writing the exact same response for all queries, so there shouldn't be any unknown fields. I didn't mean unknown "field" I mean extra data of any kind, but I suspect you're using an API to form the requests etc so this is probably not the case. I also only get an error on <1/5 of the requests, and even when the error occurs, the response has been written completely to the browser. Ahh, ok, I believe the problem is simply the timing of the 'close/EndRequest'. Sometimes it happens /before/ the data has been completely read (1/5), other times after (4/5). R
Re: std.socket - problems closing socket
On 03.10.2011 16:16, Regan Heath wrote: On Mon, 03 Oct 2011 12:57:56 +0100, simendsjo wrote: (...) To help me understand (I know nothing about fastcgi or nginx) can you clarify... 1. Your D code is the client side, connecting to the web server and sending GET/POST style requests? 2. You get these ABORTED and RESET errors on the client side? 3. As #3 even after doing as I described, shutdown(SEND), recv, then close? If yes to all the above, then it sounds like the web server/fastcgi is closing the socket without reading all the data you're sending, which probably means you're sending something it's not expecting. I would start by verifying exactly what data you're sending, and that it's all expected by the remote end. Yes. I've coded the client as follows: 1) start listening socket 2) wait for incoming connections or incoming data 3) receive(). If a socket returns 0 or -1, close it and process next with data 4) read fastcgi request from server 5) write fastcgi response 6) write fastcgi EndRequest (the server should now end the request) 7) if the application should close the request, send shutdown(send) 8) accept incoming connection 9) back to 2) FastCGI connections works in one of two ways: the server is responsible for closing the connections (supports mulitplexing) or the application should close the connection after a request has been sent. For the latter I send SocketShutdown.SEND after writing EndRequest in step 6), but it doesn't really matter as nginx doesn't support multiplexing. It closes the connection after each request anyway. I see the same result no matter what option I use. I'm running the exact same request and writing the exact same response for all queries, so there shouldn't be any unknown fields. I also only get an error on <1/5 of the requests, and even when the error occurs, the response has been written completely to the browser. I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all my data and reading until I receive 0 or -1. (doesn't really matter as sending the FastCGI EndRequest makes the server shut it down as it doesn't handle multiplexing) So, the socket closure is initiated by fastcgi/the web server. This supports the theory that it's not reading some of your data, because it's not expecting it, and this is likely the cause of the ABORT/RESET errors you're seeing. I have tried with linger too, but it doesn't help: socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER, std.socket.linger(1, 30)); The default LINGER options should be fine, as-is. But, double check the D socket code just in case it is setting different LINGER options by default (I haven't used it, or looked myself, sorry). Linger is default off, but it doesn't help to turn it on. RCV/SNDTIMO is also set to 0. Could this be caused by some bad settings on the webserver? It is possible, but I would double check your requests first. There may be a setting, or settings for aborting connections which take too long, or fail to send certain data, or connect from the wrong IP, or... If your requests are otherwise working, then I suspect you're sending some 'extra' data which is not being read. The requests are handled in ~1msec, so there shouldn't be any timeouts. The default timeout on nginx for fastcgi is 60 seconds too. I can easily process ~200 requests per second (and nginx and my server doesn't break a sweat, it's my curl spammers that's using all the cpu) PS: Seems my computer can handle about 16000 TIME_WAIT before it starts "hanging". You'll be running out of operating system handles or similar at that point :p Yup. I'll probably never have that problem in a production environment though :)
Re: TickDuration = Divide by Zero
On Mon, 03 Oct 2011 19:09:31 +0300, Chris Williams wrote: I presume that TickDuration is trying to access something that it doesn't have access to yet, during startup. http://d.puremagic.com/issues/show_bug.cgi?id=6631 Is there another way to make my constant a constant? How about using something with a resolution known at compilation time (e.g. a Duration)? -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
TickDuration = Divide by Zero
I have the following code:
import std.datetime;
const TickDuration a;
static this() {
a = TickDuration.from!"msecs"(15_000);
}
int main() {
writefln("Hello world!"); // is never printed
return 0;
}
When I run it, I get the following output:
object.Error: Integer Divide by Zero
I presume that TickDuration is trying to access something that it
doesn't have access to yet, during startup. Is there another way to
make my constant a constant?
Dealing with property function and operator overloads
Sample code:
struct Point
{
int x, y;
void opOpAssign(string op)(int rhs)
{
mixin("x = x " ~ op ~ " rhs;");
mixin("y = y " ~ op ~ " rhs;");
}
}
struct Wrapper
{
void notifyChanged() { }
@property void point(Point newpoint) { _p = newpoint; notifyChanged(); }
@property Point point() { return _p; }
private Point _p;
}
void main()
{
auto wrap = Wrapper();
wrap.point = Point(1, 1);
assert(wrap.point == Point(1, 1));
wrap.point += 1;
assert(wrap.point != Point(2, 2)); // oops
}
I want to get notified when the _p field is changed. A property
function works for assignments, but by using property functions I lose
any operator overloads Point might have.
I was thinking I could provide some sort of injecting mechanism into
the Point definition, so I could get rid of property functions and
instead do something like this:
struct Wrapper
{
this(...)
{
point.changed = ¬ifyChanged;
}
public Point point;
}
Has anyone ran into this issue before, and if so how did you work around it?
Re: std.socket - problems closing socket
This might be a useful read.. http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx
Re: std.socket - problems closing socket
On Mon, 03 Oct 2011 12:57:56 +0100, simendsjo wrote: On 03.10.2011 11:36, Regan Heath wrote: For a "graceful" close you're supposed to ensure there is no data pending. To do that you: shutdown(SD_SEND); // send only, not recv close(); The loop should read until recv returns 0. If recv returns -1 and the socket is blocking it should error/exit. If recv returns -1 and the socket is non-blocking it should check for [WSA]EWOULDBLOCK (and select/sleep + loop) or error/exit. The reason to do this is to flush all the data from the socket buffers on the remote and local ends, otherwise a close can cause remote buffered data to cause a "connection broken" error on the remote end, and/or (I am guessing a little here) may cause the socket to close while negotiating a graceful close, and/or remain in a TIME_WAIT state due to buffered data or data "in flight". ... are you setting any close options/timeouts i.e. LINGER? Thanks. :) recv returns -1 for many requests. The errors are only WSAECONNABORTED and WSAECONNRESET as described here: http://msdn.microsoft.com/en-us/library/ms740668.aspx To help me understand (I know nothing about fastcgi or nginx) can you clarify... 1. Your D code is the client side, connecting to the web server and sending GET/POST style requests? 2. You get these ABORTED and RESET errors on the client side? 3. As #3 even after doing as I described, shutdown(SEND), recv, then close? If yes to all the above, then it sounds like the web server/fastcgi is closing the socket without reading all the data you're sending, which probably means you're sending something it's not expecting. I would start by verifying exactly what data you're sending, and that it's all expected by the remote end. I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all my data and reading until I receive 0 or -1. (doesn't really matter as sending the FastCGI EndRequest makes the server shut it down as it doesn't handle multiplexing) So, the socket closure is initiated by fastcgi/the web server. This supports the theory that it's not reading some of your data, because it's not expecting it, and this is likely the cause of the ABORT/RESET errors you're seeing. I have tried with linger too, but it doesn't help: socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER, std.socket.linger(1, 30)); The default LINGER options should be fine, as-is. But, double check the D socket code just in case it is setting different LINGER options by default (I haven't used it, or looked myself, sorry). Could this be caused by some bad settings on the webserver? It is possible, but I would double check your requests first. There may be a setting, or settings for aborting connections which take too long, or fail to send certain data, or connect from the wrong IP, or... If your requests are otherwise working, then I suspect you're sending some 'extra' data which is not being read. PS: Seems my computer can handle about 16000 TIME_WAIT before it starts "hanging". You'll be running out of operating system handles or similar at that point :p -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Passing ref through a template chain
Works like a charm, and it's so obvious a thing to try that I'm kicking myself. Thanks Steve
Re: Passing ref through a template chain
On 2011-10-03 15:08, Steve Teale wrote:
To bind variables, the MySQL api wants their addresses - in my tiny example
below, these are represented by the void*[].
If I just use something like setTarget in the example, it works fine, but then
when I try to set a bunch of them in one go I'm hosed because I can't get the
right addresses through the chain.
import std.stdio;
void*[2] vpa;
void setTarget(T)(ref T t, int i)
{
vpa[i] =&t;
}
void setSeveral(T...)(ref T args)
{
foreach (int i, arg; args)
setTarget(&arg, i);
}
//ref.d|19|Error: variable i cannot be read at compile time|
//ref.d|19|Error: variable i cannot be read at compile time|
//ref.d|19|Error: Integer constant expression expected instead of cast(uint)i|
/*
void setSeveral2(T...)(ref T args)
{
for (int i = 0; i< args.length; i++)
setTarget(args[i], i);
}
*/
void main()
{
bool a;
int b;
setSeveral(a, b);
writefln("orig %x %x, set %x %x",&a,&b, vpa[0], vpa[1]); // addresses differ
}
The problem arises I believe in 'foreach (int i, arg; args)' which makes a
copy for 'arg'. Usually in a foreach I can get round that by making it 'ref
arg'. But in the template function that does not work.
I can't use the setSeveral2() form, because that provokes the errors shown in
the comments.
How can I force the real addresses through the chain?
Thanks
Steve
Does this work:
void setSeveral(T...)(ref T args)
{
foreach (i, _; args)
setTarget(args[i], i);
}
Or you already tried that in setSeveral2.
--
/Jacob Carlborg
Passing ref through a template chain
To bind variables, the MySQL api wants their addresses - in my tiny example
below, these are represented by the void*[].
If I just use something like setTarget in the example, it works fine, but then
when I try to set a bunch of them in one go I'm hosed because I can't get the
right addresses through the chain.
import std.stdio;
void*[2] vpa;
void setTarget(T)(ref T t, int i)
{
vpa[i] = &t;
}
void setSeveral(T...)(ref T args)
{
foreach (int i, arg; args)
setTarget(&arg, i);
}
//ref.d|19|Error: variable i cannot be read at compile time|
//ref.d|19|Error: variable i cannot be read at compile time|
//ref.d|19|Error: Integer constant expression expected instead of cast(uint)i|
/*
void setSeveral2(T...)(ref T args)
{
for (int i = 0; i < args.length; i++)
setTarget(args[i], i);
}
*/
void main()
{
bool a;
int b;
setSeveral(a, b);
writefln("orig %x %x, set %x %x", &a, &b, vpa[0], vpa[1]); // addresses
differ
}
The problem arises I believe in 'foreach (int i, arg; args)' which makes a
copy for 'arg'. Usually in a foreach I can get round that by making it 'ref
arg'. But in the template function that does not work.
I can't use the setSeveral2() form, because that provokes the errors shown in
the comments.
How can I force the real addresses through the chain?
Thanks
Steve
Re: std.socket - problems closing socket
On 03.10.2011 11:36, Regan Heath wrote: For a "graceful" close you're supposed to ensure there is no data pending. To do that you: shutdown(SD_SEND); // send only, not recv close(); The loop should read until recv returns 0. If recv returns -1 and the socket is blocking it should error/exit. If recv returns -1 and the socket is non-blocking it should check for [WSA]EWOULDBLOCK (and select/sleep + loop) or error/exit. The reason to do this is to flush all the data from the socket buffers on the remote and local ends, otherwise a close can cause remote buffered data to cause a "connection broken" error on the remote end, and/or (I am guessing a little here) may cause the socket to close while negotiating a graceful close, and/or remain in a TIME_WAIT state due to buffered data or data "in flight". ... are you setting any close options/timeouts i.e. LINGER? Thanks. recv returns -1 for many requests. The errors are only WSAECONNABORTED and WSAECONNRESET as described here: http://msdn.microsoft.com/en-us/library/ms740668.aspx I'm doing socket.shutdown(SocketShutdown.SEND) now after sending all my data and reading until I receive 0 or -1. (doesn't really matter as sending the FastCGI EndRequest makes the server shut it down as it doesn't handle multiplexing) I have tried with linger too, but it doesn't help: socket.setOption(SocketOptionLevel.SOCKET, SocketOption.LINGER, std.socket.linger(1, 30)); Could this be caused by some bad settings on the webserver? PS: Seems my computer can handle about 16000 TIME_WAIT before it starts "hanging".
Re: std.socket - problems closing socket
On Sat, 01 Oct 2011 00:26:35 +0100, simendsjo wrote: Not sure if this is a problem with std.socket, nginx or my knowledge of sockets. I'm pretty sure it's the last one. I'm experimenting with fastcgi on nginx, and the socket stays in TIME_WAIT even after I call socket.shutdown(SocketShutdown.BOTH); socket.close(); (Crossposted from SO: http://stackoverflow.com/questions/7616601/nginx-fastcgi-and-open-sockets) For a "graceful" close you're supposed to ensure there is no data pending. To do that you: shutdown(SD_SEND); // send only, not recv close(); The loop should read until recv returns 0. If recv returns -1 and the socket is blocking it should error/exit. If recv returns -1 and the socket is non-blocking it should check for [WSA]EWOULDBLOCK (and select/sleep + loop) or error/exit. The reason to do this is to flush all the data from the socket buffers on the remote and local ends, otherwise a close can cause remote buffered data to cause a "connection broken" error on the remote end, and/or (I am guessing a little here) may cause the socket to close while negotiating a graceful close, and/or remain in a TIME_WAIT state due to buffered data or data "in flight". .. are you setting any close options/timeouts i.e. LINGER? -- Using Opera's revolutionary email client: http://www.opera.com/mail/
