Re: Middleware Messages for FE/BE

2021-08-20 Thread Jesper Pedersen
Hi Simon, On 8/20/21 10:39 AM, Simon Riggs wrote: Yeah, but it is a change to the protocol which means that the client drivers and middleware ecosystem needs to be updated to support that message. No, because FE messages don't need to be handled by the client, they just send them. Yes, but

Re: Middleware Messages for FE/BE

2021-08-20 Thread Simon Riggs
On Fri, 20 Aug 2021 at 13:59, Jesper Pedersen wrote: > > On 8/19/21 3:52 PM, Hannu Krosing wrote: > > Jesper, please don't confuse my ramblings with Simon's initial proposal. > > > > As I understand, the original proposal was just about adding a new > > wire protocol message type, which then

Re: Middleware Messages for FE/BE

2021-08-20 Thread Jesper Pedersen
Hi Hannu, On 8/19/21 3:52 PM, Hannu Krosing wrote: Jesper, please don't confuse my ramblings with Simon's initial proposal. As I understand, the original proposal was just about adding a new wire protocol message type, which then could be used for emitting custom messages by middleware support

Re: Middleware Messages for FE/BE

2021-08-19 Thread Hannu Krosing
Jesper, please don't confuse my ramblings with Simon's initial proposal. As I understand, the original proposal was just about adding a new wire protocol message type, which then could be used for emitting custom messages by middleware support extension - likely loaded as a preloaded shared

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
On Thu, 19 Aug 2021 at 19:04, Jesper Pedersen wrote: > Should the middleware guess the client driver based on the startup > message ? Or are you thinking about having a class identifier group for > each client driver and then a massive "switch/case" inside each middleware ? The question is how

Re: Middleware Messages for FE/BE

2021-08-19 Thread Jesper Pedersen
Hi, On 8/19/21 1:13 PM, Simon Riggs wrote: We need to be able to send the middleware a message. Replies can be sent as NoticeResponse, if needed, which already exists. Yeah, but you need the client driver - of all languages - to understand that notice. And, if different middleware uses

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
On Thu, 19 Aug 2021 at 17:42, Jesper Pedersen wrote: > I would say that this is a PostgreSQL protocol v4 thing, as there is a > bit more to it. > > > There is a need to trigger middleware functionality, but you need to > query the middleware stack first on what it supports - failover, load >

Re: Middleware Messages for FE/BE

2021-08-19 Thread Jesper Pedersen
Hi Simon, On 8/19/21 4:33 AM, Simon Riggs wrote: The current FE/BE protocol assumes that the client is talking to the server directly. If middleware wants to do something like load balancing, the only current option is to inspect the incoming commands and take action from that. That approach

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
On Thu, 19 Aug 2021 at 10:58, Hannu Krosing wrote: > > Actually the way I have been thinking about this would be a message > > "Hey replica x.x.x.x, please take over the write head role" > > Which would > > 1. cause the replica x.x.x.x to self-promote at this point-in-WAL > 1A - optionally the

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
On Thu, 19 Aug 2021 at 10:33, Hannu Krosing wrote: > > Need for this does come up quite often so I very much support this. > > In addition to keeping a registry there likely need to be some other > "generally agreed" rules as well, like > * it being safe to ignore any and all of the middleware

Re: Middleware Messages for FE/BE

2021-08-19 Thread Hannu Krosing
Actually the way I have been thinking about this would be a message "Hey replica x.x.x.x, please take over the write head role" Which would 1. cause the replica x.x.x.x to self-promote at this point-in-WAL 1A - optionally the old write head becomes a replica (it should mention it in message

Re: Middleware Messages for FE/BE

2021-08-19 Thread Hannu Krosing
One more set of "standard middleware messages" clients/middleware could turn on could be reporting LSNs * various local LSNs for progress of WAL persisting * reporting replication state of some or all replicas - Hannu Krosing Google Cloud - We have a long list of planned contributions and

Re: Middleware Messages for FE/BE

2021-08-19 Thread Hannu Krosing
Need for this does come up quite often so I very much support this. In addition to keeping a registry there likely need to be some other "generally agreed" rules as well, like * it being safe to ignore any and all of the middleware messages (at least with no degradation from the state of not

Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
The current FE/BE protocol assumes that the client is talking to the server directly. If middleware wants to do something like load balancing, the only current option is to inspect the incoming commands and take action from that. That approach performs poorly and has proven difficult to maintain,