Re: [HACKERS] One process per session lack of sharing

On 12 July 2016 at 09:57, > wrote: We have faced with some lack of sharing resources. So in our test memory usage per session: Oracle: about 5M MSSqlServer: about 4M postgreSql: about 160М Using shared resources also has

Re: [HACKERS] One process per session lack of sharing

On 2016-07-19 14:18:22 +0300, amatv...@bitec.ru wrote: > Hi > > > > Using TLS will slow down things noticeably though. So if we were to go > > there, we'd have to make up for some constant slowdown. > I can not understand why? > > I've read >

Re: [HACKERS] One process per session lack of sharing

On Mon, Jul 18, 2016 at 8:56 PM, Craig Ringer wrote: > Since I got started with Pg, I've taken it as given that PostgreSQL Will > Never Use Threads, Don't Even Talk About It. As taboo as query hints or more > so. Is this actually a serious option? I'm sure that depends on

Re: [HACKERS] One process per session lack of sharing

Hi > Using TLS will slow down things noticeably though. So if we were to go > there, we'd have to make up for some constant slowdown. I can not understand why? I've read https://msdn.microsoft.com/en-us/library/windows/desktop/ms686749(v=vs.85).aspx and

Re: [HACKERS] One process per session lack of sharing

On 18 July 2016 at 02:27, Robert Haas wrote: > On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer > wrote: > > I don't think anyone's considering moving from multi-processing to > > multi-threading in PostgreSQL. I really, really like the protection that

Re: [HACKERS] One process per session lack of sharing

On 2016-07-19 08:33:20 +0800, Craig Ringer wrote: > On 19 July 2016 at 03:53, Andres Freund wrote: > > > On 2016-07-18 15:47:58 -0400, Robert Haas wrote: > > > I think the risk profile is exactly the opposite of what you are > > > suggesting here. If we provide an option to

Re: [HACKERS] One process per session lack of sharing

On 19 July 2016 at 03:53, Andres Freund wrote: > On 2016-07-18 15:47:58 -0400, Robert Haas wrote: > > I think the risk profile is exactly the opposite of what you are > > suggesting here. If we provide an option to compile the server with > > all global variables converted

Re: [HACKERS] One process per session lack of sharing

On 2016-07-18 15:47:58 -0400, Robert Haas wrote: > I think the risk profile is exactly the opposite of what you are > suggesting here. If we provide an option to compile the server with > all global variables converted to thread-local variables, there's > really not a whole lot that can break,

Re: [HACKERS] One process per session lack of sharing

On Sun, Jul 17, 2016 at 10:00 PM, Jan Wieck wrote: >> I admit that it is risky, but I think there are things that could be >> done to limit the risk. I don't believe we can indefinitely continue >> to ignore the potential performance benefits of making a switch like >> this.

Re: [HACKERS] One process per session lack of sharing

On Mon, Jul 18, 2016 at 10:03 AM, Greg Stark wrote: > On Mon, Jul 18, 2016 at 2:41 PM, wrote: >> And I will be really happy when there are processors with infinite >> performance and memory with infinite size. >>:))) > > Well for what it's worth there's no

Re: [HACKERS] One process per session lack of sharing

Hi >So I think as long as process and thread have different function in OS, >use process like thread will have overhead in practice. But There are other negative things. I think parallel oriented library usually do not work with process. So Jvm integration is not exception. It is

Re: [HACKERS] One process per session lack of sharing

Hi > In other words, there's no theoretical reason you couldn't have adapt > a JVM to create a large shared memory segment using mmap or SysV I think even if I was the leader in OS development, I could not correctly answer your question. So just let discuss. Ok, I agree with you that

Re: [HACKERS] One process per session lack of sharing

On Mon, Jul 18, 2016 at 2:41 PM, wrote: > And I will be really happy when there are processors with infinite > performance and memory with infinite size. >:))) Well for what it's worth there's no theoretical difference between multi-process and multi-threaded. They're

Re: [HACKERS] One process per session lack of sharing

Hi > This https://github.com/davecramer/plj-new is a very old project > that did work at one time which attempted to do RPC calls to the jvm to > address exactly this problem. > However "cheaply" calling jvm from sql or vice-versa is not really possible. > I do like the idea of the background

Re: [HACKERS] One process per session lack of sharing

On 18 July 2016 at 06:04, wrote: > Hi > > > There's https://github.com/jnr/jnr-ffi that enables to call C > > functions without resorting to writing JNI wrappers. > I have not said that you are wrong. > It's the dark side of "like seprate process" > They can cheaply call sql

Re: [HACKERS] One process per session lack of sharing

Hi > There's https://github.com/jnr/jnr-ffi that enables to call C > functions without resorting to writing JNI wrappers. I have not said that you are wrong. It's the dark side of "like seprate process" They can cheaply call sql from jvm. And they can't cheaply call jvm from sql. Jvm in oracle

Re: [HACKERS] One process per session lack of sharing

Hi > I admit that it is risky, but I think there are things that could be > done to limit the risk. I don't believe we can indefinitely continue > to ignore the potential performance benefits of making a switch like > this. Breaking a thirty-year old code base irretrievably would be > sad, but

Re: [HACKERS] One process per session lack of sharing

Hi > The issue here is an architectural mismatch between PostgreSQL and > the JVM, made worse by the user's very stored-proc-heavy code. Some > other runtime that's designed to co-operate with a multiprocessing > environment could well be fine, but the JVM isn't. At least, the >

Re: [HACKERS] One process per session lack of sharing

Hi > Such a host delegate process could be explicitly built with > multithread support and not 'infect' the rest of the code with its > requirements. > > Using granular RPC is nice for isolation but I am concerned that the > latencies might be high. I agree with you. Moreover I think that

Re: [HACKERS] One process per session lack of sharing

On Sun, Jul 17, 2016 at 9:28 PM, Robert Haas wrote: > On Sun, Jul 17, 2016 at 3:12 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer > wrote: > >>> I don't

Re: [HACKERS] One process per session lack of sharing

On Sun, Jul 17, 2016 at 3:04 PM, Jim Nasby wrote: > On 7/14/16 12:34 AM, Craig Ringer wrote: >> Starting with a narrow scope would help. Save/restore GUCs and the other >> easy stuff, and disallow sessions that are actively LISTENing, hold >> advisory locks, have open

Re: [HACKERS] One process per session lack of sharing

On Sun, Jul 17, 2016 at 3:12 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: >>> I don't think anyone's considering moving from multi-processing to >>> multi-threading in

Re: [HACKERS] One process per session lack of sharing

Robert Haas writes: > On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: >> I don't think anyone's considering moving from multi-processing to >> multi-threading in PostgreSQL. I really, really like the protection that the >>

Re: [HACKERS] One process per session lack of sharing

On 7/14/16 12:34 AM, Craig Ringer wrote: Starting with a narrow scope would help. Save/restore GUCs and the other easy stuff, and disallow sessions that are actively LISTENing, hold advisory locks, have open cursors, etc from being saved and restored. Along the lines of narrow scope... I

Re: [HACKERS] One process per session lack of sharing

On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: > I don't think anyone's considering moving from multi-processing to > multi-threading in PostgreSQL. I really, really like the protection that the > shared-nothing-by-default process model gives us, among other things.

Re: [HACKERS] One process per session lack of sharing

On 12 July 2016 at 09:57, wrote: > We have faced with some lack of sharing resources. > So in our test memory usage per session: > Oracle: about 5M > MSSqlServer: about 4M > postgreSql: about 160М > Using shared resources also has significant problems, so care must be taken.

Re: [HACKERS] One process per session lack of sharing

On Sun, Jul 17, 2016 at 3:23 AM, Craig Ringer wrote: > > > Lots more could be shared, too. Cached plans, for example. > But the fact that PostgreSQL has transactional DDL complicates things like a shared plan cache and shared PL/pgSQL execution trees. Those things are

Re: [HACKERS] One process per session lack of sharing

On 15 July 2016 at 20:54, wrote: > Hi > > > > but parallel processing doesn't requires threading support - see > PostgreSQL 9.6 features. > > To share dynamic execution code between threads much more easy(If > sharing this code between process is possible). > There is

Re: [HACKERS] One process per session lack of sharing

On 16 July 2016 at 00:43, james wrote: > On 15/07/2016 09:28, Craig Ringer wrote: > >> I don't think anyone's considering moving from multi-processing to >> multi-threading in PostgreSQL. I really, really like the protection that >> the shared-nothing-by-default

Re: [HACKERS] One process per session lack of sharing

Hi >What I know about Oracle, PL/SQL, Java - all is executed as outprocess calls. >I am sure, so PL doesn't share process with SQL engine there You are highly not correct. Sorry, I will answer more when i come to work. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] One process per session lack of sharing

2016-07-15 19:57 GMT+02:00 Матвеев Алексей : > Hi > > >What I know about Oracle, PL/SQL, Java - all is executed as outprocess > calls. I am sure, so PL doesn't share process with SQL engine there > > You are highly not correct. > Sorry, I will answer more when i come to

Re: [HACKERS] One process per session lack of sharing

2016-07-15 18:43 GMT+02:00 james : > On 15/07/2016 09:28, Craig Ringer wrote: > >> I don't think anyone's considering moving from multi-processing to >> multi-threading in PostgreSQL. I really, really like the protection that >> the shared-nothing-by-default process

Re: [HACKERS] One process per session lack of sharing

On 15/07/2016 09:28, Craig Ringer wrote: I don't think anyone's considering moving from multi-processing to multi-threading in PostgreSQL. I really, really like the protection that the shared-nothing-by-default process model gives us, among other things. As I understand it, the main issue is

Re: [HACKERS] One process per session lack of sharing

Hi > This is true, only when data are immutable and in memory. Elsewhere it is > false idea. For case whenthe server works 24x7 and you need ability to fix bugs(or update) on the fly in any app code. It's usual. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] One process per session lack of sharing

2016-07-15 14:54 GMT+02:00 : > Hi > > > > but parallel processing doesn't requires threading support - see > PostgreSQL 9.6 features. > > To share dynamic execution code between threads much more easy(If > sharing this code between process is possible). > There is many

Re: [HACKERS] One process per session lack of sharing

Hi > but parallel processing doesn't requires threading support - see PostgreSQL > 9.6 features. To share dynamic execution code between threads much more easy(If sharing this code between process is possible). There is many other interaction techniques between threads which is

Re: [HACKERS] One process per session lack of sharing

Hi > If amatveev (username, unsure of full name) wants to improve > PL/PgSQL performance and the ability of a JVM to share resources > between backends, then it would be more productive to focus on that than on > threading. Note, I've statred this post with

Re: [HACKERS] One process per session lack of sharing

2016-07-15 13:25 GMT+02:00 : > Hi > > > > Can be nice, if we can help to all Oracle users - but it is not > > possible in this world :( - there is lot of barriers - threading is > > only one, second should be different design of PL/SQL - it is based > > on out processed, next

Re: [HACKERS] One process per session lack of sharing

Hi > Can be nice, if we can help to all Oracle users - but it is not > possible in this world :( - there is lot of barriers - threading is > only one, second should be different design of PL/SQL - it is based > on out processed, next can be libraries, JAVA integration, and lot > of others. I

Re: [HACKERS] One process per session lack of sharing

On 15 July 2016 at 18:05, Pavel Stehule wrote: > There is only few use cases - mostly related to Oracle emulation when > multi threading is necessary - and few can be solved better - PLpgSQL to C > compilation and similar techniques. > > Right. If amatveev (username,

Re: [HACKERS] One process per session lack of sharing

2016-07-15 12:20 GMT+02:00 : > Hi > > > > I disagree - there is lot of possible targets with much higher > > benefits - columns storage, effective execution - compiled > > execution, implementation of temporal databases, better support for > > dynamic structures, better support

Re: [HACKERS] One process per session lack of sharing

Hi > I disagree - there is lot of possible targets with much higher > benefits - columns storage, effective execution - compiled > execution, implementation of temporal databases, better support for > dynamic structures, better support for XML, JSON, integration of connection > pooling, ... Off

Re: [HACKERS] One process per session lack of sharing

thread based database - is better than Postgres, or there is more users migrated from Orace? Not. Regards Pavel > > And may be the right start is to fix the Faq > > https://wiki.postgresql.org/wiki/FAQ#Why_does_PostgreSQL_use_so_much_memory.3F > >Why does PostgreSQL use so much memory

Re: [HACKERS] One process per session lack of sharing

y? >Despite appearances, this is absolutely normal It's not normal. It's "as is". You should use pgBouncer. See "Re: [HACKERS] One process per session lack of sharing" And it is why >there are workloads where it >fails badly - and competing database products survive a n

Re: [HACKERS] One process per session lack of sharing

On 14 July 2016 at 16:41, wrote: > Hi > > > On Tue, Jul 12, 2016 at 9:18 AM, Tom Lane wrote: > >> amatv...@bitec.ru writes: > >>> Is there any plan to implement "session per thread" or "shared > >>> sessions between thread"? > >>... > >> so > >>

Re: [HACKERS] One process per session lack of sharing

Hi > It's mostly working, but there are workloads where it > fails badly - and competing database products survive a number of > scenarios where we just fall on our face. > So, I actually think it would be a good idea to think about this. Just to think. http://www.tiobe.com/tiobe_index The

Re: [HACKERS] One process per session lack of sharing

Hi > On Tue, Jul 12, 2016 at 9:18 AM, Tom Lane wrote: >> amatv...@bitec.ru writes: >>> Is there any plan to implement "session per thread" or "shared >>> sessions between thread"? >>... >> so >> there's not that much motivation to do a ton of work inside the database

Re: [HACKERS] One process per session lack of sharing

On 14 July 2016 at 14:28, Vladimir Sitnikov wrote: > Craig>That moves work further away from the DB, which has its own costs, > and isn't something you're likely to be happy with if you're looking at > things like optimising PL/PgSQL with a bytecode compiler. But

Re: [HACKERS] One process per session lack of sharing

Craig>That moves work further away from the DB, which has its own costs, and isn't something you're likely to be happy with if you're looking at things like optimising PL/PgSQL with a bytecode compiler. But it's the best we have right now. What if JVM was started within a background worker? Then

Re: [HACKERS] One process per session lack of sharing

On 12 July 2016 at 21:57, wrote: > Hi > > Is there any plan to implement "session per thread" or "shared > sessions between thread"? > As has been noted by others, there isn't any such plan right now. PostgreSQL isn't threaded. It uses a multi-processing

Re: [HACKERS] One process per session lack of sharing

On 14 July 2016 at 03:59, Robert Haas wrote: > I agree that there's not really a plan to implement this, but I don't > agree that connection pooling solves the whole problem. Most people > can't get by with statement pooling, so in practice you are looking at >

Re: [HACKERS] One process per session lack of sharing

On Tue, Jul 12, 2016 at 9:18 AM, Tom Lane wrote: > amatv...@bitec.ru writes: >> Is there any plan to implement "session per thread" or "shared >> sessions between thread"? > > No, not really. The amount of overhead that would add --- eg, the need > for locking on what

Re: [HACKERS] One process per session lack of sharing

Hi > amatv...@bitec.ru writes: >> Is there any plan to implement "session per thread" or "shared >> sessions between thread"? > No, not really. The amount of overhead that would add --- eg, the need > for locking on what used to be single-use caches --- makes the benefit > highly

Re: [HACKERS] One process per session lack of sharing

amatv...@bitec.ru writes: > Is there any plan to implement "session per thread" or "shared > sessions between thread"? No, not really. The amount of overhead that would add --- eg, the need for locking on what used to be single-use caches --- makes the benefit highly questionable. Also,