Re: Tips on TCP socket to postgresql middleware

2022-02-24 Thread eugene via Digitalmars-d-learn
On Thursday, 24 February 2022 at 11:27:56 UTC, Tejas wrote: Wagner F. et al Modeling Software with Finite State Machines: A Practical Approach I've adopted some ideas from this book to POSIX/Linux API. see also http://www.stateworks.com/ Thank you so much! Also there is [very nice discussi

Re: Tips on TCP socket to postgresql middleware

2022-02-24 Thread eugene via Digitalmars-d-learn
On Thursday, 24 February 2022 at 09:11:01 UTC, eugene wrote: In my case (I was working with REDIS KVS at the moment) exact scenario was as follows: * prog gets EPOLLOUT (write() won't block) * prog writes()'s data to REDIS ("successfully") * prog gets EPOLLERR|EPOLLHUP After this I see that I n

Re: Tips on TCP socket to postgresql middleware

2022-02-24 Thread Tejas via Digitalmars-d-learn
On Thursday, 24 February 2022 at 06:54:07 UTC, eugene wrote: On Thursday, 24 February 2022 at 06:30:51 UTC, Tejas wrote: On Wednesday, 23 February 2022 at 09:34:56 UTC, eugene wrote: On Tuesday, 22 February 2022 at 20:19:39 UTC, Chris Piker wrote: [...] As you might have been already noted,

Re: Tips on TCP socket to postgresql middleware

2022-02-24 Thread eugene via Digitalmars-d-learn
On Thursday, 24 February 2022 at 08:46:35 UTC, eugene wrote: On Saturday, 19 February 2022 at 20:13:01 UTC, Chris Piker wrote: 3. Update/insert to a postgresql database as data arrive. I've remembered one not so obvious feature of TCP sockets behavour. If the connection is closed on the serv

Re: Tips on TCP socket to postgresql middleware

2022-02-24 Thread eugene via Digitalmars-d-learn
On Saturday, 19 February 2022 at 20:13:01 UTC, Chris Piker wrote: 3. Update/insert to a postgresql database as data arrive. I've remembered one not so obvious feature of TCP sockets behavour. If the connection is closed on the server side (i.e. on the client side the socket is in CLOSE_WAIT s

Re: Tips on TCP socket to postgresql middleware

2022-02-23 Thread eugene via Digitalmars-d-learn
On Thursday, 24 February 2022 at 06:54:07 UTC, eugene wrote: Wagner F. et al Modeling Software with Finite State Machines: A Practical Approach I've adopted some ideas from this book to POSIX/Linux API. Ah! I also have EDSM for bare metal (AVR8 to be exact) There is [some description](http:

Re: Tips on TCP socket to postgresql middleware

2022-02-23 Thread eugene via Digitalmars-d-learn
On Thursday, 24 February 2022 at 06:30:51 UTC, Tejas wrote: On Wednesday, 23 February 2022 at 09:34:56 UTC, eugene wrote: On Tuesday, 22 February 2022 at 20:19:39 UTC, Chris Piker wrote: [...] As you might have been already noted, the key idea is to implement SM explicitly, i.e we have states

Re: Tips on TCP socket to postgresql middleware

2022-02-23 Thread Tejas via Digitalmars-d-learn
On Wednesday, 23 February 2022 at 09:34:56 UTC, eugene wrote: On Tuesday, 22 February 2022 at 20:19:39 UTC, Chris Piker wrote: [...] As you might have been already noted, the key idea is to implement SM explicitly, i.e we have states, messages, actions, transitions and extremely simple engine

Re: Tips on TCP socket to postgresql middleware

2022-02-23 Thread eugene via Digitalmars-d-learn
On Tuesday, 22 February 2022 at 20:19:39 UTC, Chris Piker wrote: credit you for the basic ideas As you might have been already noted, the key idea is to implement SM explicitly, i.e we have states, messages, actions, transitions and extremely simple engine (reactTo() method) Switch-based imple

Re: Tips on TCP socket to postgresql middleware

2022-02-23 Thread eugene via Digitalmars-d-learn
On Tuesday, 22 February 2022 at 20:19:39 UTC, Chris Piker wrote: On Monday, 21 February 2022 at 07:00:52 UTC, eugene wrote: On Monday, 21 February 2022 at 04:46:53 UTC, Chris Piker wrote: On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: I'm adverse to reading it closely since there wa

Re: Tips on TCP socket to postgresql middleware

2022-02-22 Thread Chris Piker via Digitalmars-d-learn
On Monday, 21 February 2022 at 07:00:52 UTC, eugene wrote: On Monday, 21 February 2022 at 04:46:53 UTC, Chris Piker wrote: On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: I'm adverse to reading it closely since there was no license file and don't want to accidentally violate copyrigh

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread eugene via Digitalmars-d-learn
On Monday, 21 February 2022 at 04:46:53 UTC, Chris Piker wrote: On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: Yes, here is my engine with example (echo client/server pair): - [In D (for Linux & FreeBSD)] The code is terse and clean, thanks for sharing :) Nice to hear, thnx! Act

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread eugene via Digitalmars-d-learn
On Monday, 21 February 2022 at 04:48:56 UTC, Chris Piker wrote: On Sunday, 20 February 2022 at 18:36:21 UTC, eugene wrote: I often use two connections, one for perform main task (upload some data and alike) and the second for getting notifications from PG, 'cause it very incovinient to do both

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread Chris Piker via Digitalmars-d-learn
On Sunday, 20 February 2022 at 18:36:21 UTC, eugene wrote: I often use two connections, one for perform main task (upload some data and alike) and the second for getting notifications from PG, 'cause it very incovinient to do both in a single connection. Ah, a very handy tip. It would be conv

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread Chris Piker via Digitalmars-d-learn
On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: Yes, here is my engine with example (echo client/server pair): - [In D (for Linux & FreeBSD)](http://zed.karelia.ru/0/e/edsm-2022-02-20.tar.gz) The code is terse and clean, thanks for sharing :) I'm adverse to reading it closely si

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread Chris Piker via Digitalmars-d-learn
On Sunday, 20 February 2022 at 17:58:41 UTC, Ali Çehreli wrote: Another one is to set the message box sizes to throttle. Message sizes and rates are relatively well know so it will be easy to pick a throttle point that's unlikely to backup the source yet provide for some quick DB maintenance i

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread eugene via Digitalmars-d-learn
On Saturday, 19 February 2022 at 20:13:01 UTC, Chris Piker wrote: In general it should buffer data in RAM to avoid exerting back pressure on the input socket and to allow for dropped connections to the PG database. If I get it right you want to restore connection if it was closed by server for

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread eugene via Digitalmars-d-learn
On Sunday, 20 February 2022 at 16:55:44 UTC, Chris Piker wrote: But I would like to return to your idea in a couple months so that I can try a fiber based implementation instead. I thougt about implementing my engine using fibers but... it seemed to me they are not very convinient because corou

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread Ali Çehreli via Digitalmars-d-learn
On 2/19/22 12:13, Chris Piker wrote: >* gotchas you've run into in your multi-threaded (or just concurrent) > programs, I use the exact scenario that you describe: Multiple threads process data and pass the results to a "writer" thread that persist it in a file. The main gotcha is your th

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread Chris Piker via Digitalmars-d-learn
On Sunday, 20 February 2022 at 15:20:17 UTC, eugene wrote: Most people will probably say this is crazy, but as to PG, one can do without libraries. I am doing so during years (in C, not D) and did not expierienced extremely complex troubles. I mean I do not use libpq - instead I implement some

Re: Tips on TCP socket to postgresql middleware

2022-02-20 Thread eugene via Digitalmars-d-learn
On Saturday, 19 February 2022 at 20:13:01 UTC, Chris Piker wrote: * general tips on which libraries to examine Most people will probably say this is crazy, but as to PG, one can do without libraries. I am doing so during years (in C, not D) and did not expierienced extremely complex troubles

Tips on TCP socket to postgresql middleware

2022-02-19 Thread Chris Piker via Digitalmars-d-learn
Hi D I'm about to start a small program to whose job is: 1. Connect to a server over a TCP socket 2. Read a packetized real-time data stream 3. Update/insert to a postgresql database as data arrive. In general it should buffer data in RAM to avoid exerting back pressure on the input socket and