Re: [HACKERS] [sqlsmith] Failed assertion in parallel worker (ExecInitSubPlan)

2016-05-13 Thread Dilip Kumar
On Fri, May 13, 2016 at 10:31 AM, Amit Kapila 
wrote:

> Here I want to ask base rels which are plain rels?
>
> It might be that I am missing something, but if we debug the serial plan
> for original query [1] for which this issue is reported, we have noticed
> that PlaceHolderVars that contain subplans are added to base rels for which
> RTE kind is (RTE_RELATION).
>
>
> [1] - select ref_68.is_trigger_deletable as c0, (select d from
>

Finally, I could reproduce this issue, with just three tables.

If targetlist of top query has subquery references and if subquery is part
of OUTER join, then while pulling up the subquery,
it will create PlaceHolderVar, which will have expressions.

After applying prohibit_parallel_clause_below_rel_v1.patch, it is not
selecting parallel plan.
So now its clear that because of sub query pullup, we may get expression in
targetlist while creating single table path list. So we need to avoid
parallel plan if it contains expression.

Below is my test:
--

postgres=# CREATE TABLE t1(c1, c2) AS SELECT g, repeat('x', 5) FROM
postgres-# generate_series(1, 1000) g;

SELECT 1000
postgres=#
postgres=# CREATE TABLE t2(c1, c2) AS SELECT g, repeat('x', 5) FROM
postgres-# generate_series(1, 100) g;
SELECT 100
postgres=# CREATE TABLE t3(c1, c2) AS SELECT g, repeat('x', 5) FROM
generate_series(1, 100) g;
SELECT 100
postgres=# analyze t1;
ANALYZE
postgres=# analyze t2;
ANALYZE
postgres=# analyze t3;
ANALYZE

postgres=# explain select t1.c1, y.x from t1 left join (select (select
t3.c1 from t3 where t3.c1=t2.c1) as x from t2)y on (y.x=t1.c1);
 QUERY PLAN

-
 Hash Right Join  (cost=319113.85..96991333895.85 rows=860 width=8)
   Hash Cond: ((SubPlan 2) = t1.c1)
   ->  Gather  (cost=1000.00..7460943906.00 rows=100 width=8)
 Workers Planned: 2
 ->  Parallel Seq Scan on t2  (cost=0.00..7460842906.00 rows=416667
width=8)
   SubPlan 1
 ->  Seq Scan on t3  (cost=0.00..17906.00 rows=1 width=4)
   Filter: (c1 = t2.c1)
   ->  Hash  (cost=154053.60..154053.60 rows=860 width=4)
 ->  Seq Scan on t1  (cost=0.00..154053.60 rows=860 width=4)
   SubPlan 2
 ->  Seq Scan on t3 t3_1  (cost=0.00..17906.00 rows=1 width=4)
   Filter: (c1 = t2.c1)
(13 rows)

postgres=# select t1.c1, y.x from t1 left join (select (select t3.c1 from
t3 where t3.c1=t2.c1) as x from t2)y on (y.x=t1.c1);
LOG:  worker process: parallel worker for PID 109446 (PID 109483) was
terminated by signal 11: Segmentation fault
LOG:  terminating any other active server processes
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.


-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com


Re: [HACKERS] Perf Benchmarking and regression.

2016-05-13 Thread Amit Kapila
On Fri, May 13, 2016 at 11:13 PM, Andres Freund  wrote:
>
> On 2016-05-13 10:20:04 -0400, Robert Haas wrote:
> > On Fri, May 13, 2016 at 7:08 AM, Ashutosh Sharma 
wrote:
> > > Following are the performance results for read write test observed
with
> > > different numbers of "backend_flush_after".
> > >
> > > 1) backend_flush_after = 256kb (32*8kb), tps = 10841.178815
> > > 2) backend_flush_after = 512kb (64*8kb), tps = 11098.702707
> > > 3) backend_flush_after = 1MB (128*8kb), tps = 11434.964545
> > > 4) backend_flush_after = 2MB (256*8kb), tps = 13477.089417
> >
> > So even at 2MB we don't come close to recovering all of the lost
> > performance.  Can you please test these three scenarios?
> >
> > 1. Default settings for *_flush_after
> > 2. backend_flush_after=0, rest defaults
> > 3. backend_flush_after=0, bgwriter_flush_after=0,
> > wal_writer_flush_after=0, checkpoint_flush_after=0
>
> 4) 1) + a shared_buffers setting appropriate to the workload.
>

If by 4th point, you mean to test the case when data fits in shared
buffers, then Mithun has already reported above [1] that it didn't see any
regression for that case


[1] -
http://www.postgresql.org/message-id/cad__ouiobznvtt_ho__p5aenu4inqcfwgarxr4tblke-uxy...@mail.gmail.com
Read line - Even for READ-WRITE when data fits into shared buffer
(scale_factor=300 and shared_buffers=8GB) performance has improved.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


[HACKERS] Losing memory references - SRF + SPI

2016-05-13 Thread Anderson Carniel
I am writing a function that returns a set of tuples by using also the
PostGIS. Thuis, I am using SRF too. It successfully returns the expected
result when it has at most 4 tuples. However, this is not the case when
more than 4 tuples have to be returned. When I debug the code, I found that
the problem is in my function that transforms a cstring after a
SPI_connection. It seems that this cstring is not valid anymore in the
moment of this conversion (see my comment below). I know that the SPI uses
different contexts when it init and finish its process. But, I don't
understand why I have this problem here. Please, note that I tried to copy
the values of the whole tuple, but I have the same problem: system crash
after the forth call of the function. Also note that I call this function
only in the init call of the SRF. Please I would appreciate any suggestion
and help.

--- code of the problematic function here ---

LWGEOM *retrieve_geom_from_postgis(int row_id) {
char query[100];
int err;
char *wkt;
int srid;
LWGEOM *lwgeom = NULL;
HeapTuple cop;
bool null;
TupleDesc tupdesc;

//refin is a prepared select command that returns 2 columns
sprintf(query, "EXECUTE refinplan(%d);", row_id);

if (SPI_OK_CONNECT != SPI_connect()) {
SPI_finish();
_DEBUG(ERROR, "retrieve_geom_from_postgis: could not connect to SPI
manager");
return NULL;
}
err = SPI_execute(query, false, 1);
if (err < 0) {
SPI_finish();
_DEBUG(ERROR, "retrieve_geom_from_postgis: could not execute the
EXECUTE command");
return NULL;
}

if (SPI_processed <= 0) {
SPI_finish();
_DEBUGF(ERROR, "the row_id (%d) does not exist in the table",
row_id)
return NULL;
}
cop = SPI_copytuple(SPI_tuptable->vals[0]);
tupdesc = SPI_tuptable->tupdesc;

/* disconnect from SPI */
SPI_finish();

wkt = text2cstring(DatumGetTextP(heap_getattr(cop, 1, tupdesc, )));
srid = DatumGetInt32(heap_getattr(cop, 2, tupdesc, ));

lwgeom = lwgeom_from_wkt(wkt, LW_PARSER_CHECK_NONE); //error here...
only after the forth call
lwgeom_set_srid(lwgeom, srid);

lwfree(wkt);

return lwgeom;
}


Re: [HACKERS] 10.0

2016-05-13 Thread Mark Dilger

> On May 13, 2016, at 8:26 PM, David G. Johnston  
> wrote:
> 
> On Fri, May 13, 2016 at 10:18 PM, Mark Dilger  wrote:
> 
> My main concern is that a commitment to never, ever break backwards
> compatibility is a commitment to obsolescence. 
> 
> ​​You started this sub-thread with:
> 
> "If I understand correctly..."
> 
> ​I'm not sure that you do...​
> 
> Our scheme is, in your terms, basically:
> 
> .micro
> 
> where  is a decimal.
> 
> You cannot reason about the whole and fraction portions of the decimal 
> independently.

There is no point in having them as separate parts of the version number
unless you can do precisely that.  If the only difference between choosing 9.7.0
vs. 10.0.0 is that you consulted a numerologist who moonlights as an astrologer,
then, yes, you can't tell anything from the first number independent from
the second.  I was simply arguing against the numerology/astrology approach
to version numbering.  The only other way out of the numerology/astrology
approach is the one Tom Lane suggested, and that you seem to support.

This whole conversation makes me think the community has done a poor job
of communicating the nature of the (major,minor) portion of the 
(major,minor,micro)
numbering scheme.  I always assumed it was more respectable that it now
appears to have been.

mark

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David G. Johnston
On Fri, May 13, 2016 at 10:18 PM, Mark Dilger 
wrote:

>
> My main concern is that a commitment to never, ever break backwards
> compatibility is a commitment to obsolescence.


​​You started this sub-thread with:

"If I understand correctly..."

​I'm not sure that you do...​

Our scheme is, in your terms, basically:

.micro

where  is a decimal.

You cannot reason about the whole and fraction portions of the decimal
independently.

When  changes backward compatibility can be broken - with respect to
both API and implementation.

It therefore makes sense to
> reserve room in the numbering scheme to be clear and honest about when
> backwards compatibility has been broken.  The major number is the normal
> place to do that.


​I'm not convinced there is enough risk here to compromise the present in
order to accommodate some unknown ​scenario that may never even come to
pass.

David J.


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread David G. Johnston
On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake 
wrote:

> On 05/13/2016 07:42 AM, Robert Haas wrote:
>
>> On Sat, Apr 30, 2016 at 8:46 PM, Joshua Drake 
>> wrote:
>>
>>> Oh, absolutely. I was just pointing out how a lot of companies are
>>> hoarding
>>> talent internally for no productive purpose.
>>>
>>
>> Wow, really?
>>
>
​[...]
​


>
>
>> Let's respect people and companies for what they do contribute, rather
>> than labeling it as not good enough.
>>
>>
> There was no disrespect intended. I was trying to push forth an idea that
> multi-company team collaboration is better for the community than single
> company team collaboration. I will stand by that assertion.


​Yeah, that failed...

I find it much easier to debate the assertion "multi-company team
collaboration is better ..." than debating ​whether "companies are
hoarding...".

I'll disagree - such a blanket statement is nearly worthless.  Three
one-person companies working together would not automatically produce
better outputs than a single company dedicating 3 people to an endeavor.
But even that retort requires me to cheat because I'm able to draw a
conclusion about the fallacy of generalization without needing to be any
more precise as to my concept of "better".

When it comes to the first statement nothing can be done to avoid the fact
that no frame-of-reference has been established for "productive purpose".

Contrary to my earlier point the nature of our setup makes the whole "scold
privately" aspect difficult - and maybe this approach works better than I
can imagine.  Though the down-thread from the quoted post does seem to
indicate that considerable non-lethal friendly-fire is one of its
byproducts.

David J.


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread David G. Johnston
On Fri, May 13, 2016 at 6:05 PM, Joshua D. Drake 
wrote:

> On 05/13/2016 01:42 PM, Josh berkus wrote:
>
>> On 05/13/2016 01:04 PM, Joshua D. Drake wrote:
>>
>>> On 05/13/2016 12:03 PM, Josh berkus wrote:
>>>
 On 05/13/2016 11:48 AM, Robert Haas wrote:

> On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake
>  wrote:
>

>>> Anyway, all of this is a moot point, because nobody has the power to
 tell the various companies what to do.  We're just lucky that everyone
 is still committed to writing stuff which adds to PostgreSQL.

>>>
>>> Lucky? No. We earned it. We earned it through years and years of hard
>>> work. Should we be thankful? Absolutely. Should we be grateful that we
>>> have such a powerful and engaged commercial contribution base? 100%.
>>>
>>
>> Lucky.  Sure there was work and personal integrity involved, but like
>> any success story, there was luck.
>>
>> But we've also been fortunate in not spawning hostile-but-popular forks
>> by people who left the project, and that none of the companies who
>> created hostile forks were very successful with them, and that nobody
>> has seriously tried using lawyers to control/ruin the project.
>>
>
> I can't get behind you on this. Everything you have said above has to do
> with the hard work and integrity of the people in this project. It isn't
> luck or divine intervention.
>
>
>> And, most importantly, we've been lucky that a lot of competing projects
>> have self-immolated instead of being successful and brain-draining our
>> contributors (MySQL, ANTS, MonetDB, etc.)
>>
>>
> Actually there are people that have been drained out, I won't name them
> but it is pretty easy to figure out who they are. The people that are left,
> especially the long timers are here because of their integrity and
> attachment to the project.
>
> This project builds good people, and the good people build a good project.
>
> I am not going to buy into a luck story for something I and many others
> have invested decades of their life into.


​I'm not sure how this ended up a philosophical/religious argument but
given that we are simply lucky that a meteor didn't wipe out our species 20
thousand years ago our present reality is a combination of determination
and luck and trying to look at shorter term slices and assign weights to
how much luck influenced something and how much was determination seems
like something best done over drinks.

​In the vein of "praise publicly, sold privately" if anyone here thinks
that someone else should be acting differently I'd suggest sending them a
private message to that effect.  More useful would be to point out on these
lists things you find are done well.

David J.


Re: [HACKERS] 10.0

2016-05-13 Thread Mark Dilger

> On May 13, 2016, at 6:33 PM, David G. Johnston  
> wrote:
> 
> On Fri, May 13, 2016 at 7:32 PM, Mark Dilger  wrote:
> 
> Any project that starts inflating its numbering scheme sends a message to
> users of the form, "hey, we've just been taken over by marketing people, and
> software quality will go down from now on."
> 
> ​Tom brought up my own thoughts on the rest - but, really, this is a cynical 
> way of looking at things.

Every company I have worked for that has been taken over by pointy haired
bosses has almost immediately begun playing games with the version numbers,
product names, etc, and dropped the focus on quality, performance, bug fixes,
etc.  I don't expect that from the postgresql community, but I find discussion
on the mailing lists about using a "wow, 10.x.x release! how amazing" marketing
line rather crass.  I doubt many people in the tech industry generally feel all
that differently about it.  I've never worked with a programmer or DBA who was
impressed with these kinds of version number bumps.

I fully agree mine is a cynical point of view.

My main concern is that a commitment to never, ever break backwards
compatibility is a commitment to obsolescence.  It therefore makes sense to
reserve room in the numbering scheme to be clear and honest about when
backwards compatibility has been broken.  The major number is the normal
place to do that.

Just my .2 cents.

mark

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David G. Johnston
On Fri, May 13, 2016 at 7:32 PM, Mark Dilger 
wrote:

>
> Any project that starts inflating its numbering scheme sends a message to
> users of the form, "hey, we've just been taken over by marketing people,
> and
> software quality will go down from now on."
>

​Tom brought up my own thoughts on the rest - but, really, this is a
cynical way of looking at things.  At least insofar as whether marketing
has say over what exact version number is assigned to a release should, and
here likely does, have almost zero influence on the quality of the code
that went into said release.  This entire discussion and timeline is proof
of exactly that.  We have the option to market version 10.0.0 if we so
choose but during the entire past year -core hasn't cared whether the
release they are working toward is eventually numbered 9.6.0 or 10.0.0​ -
and I posit that nothing would have changed had it been decided a year ago
that we were going to release 10.0.0.

I'll accept that addressing the broader community's confusion regarding our
version numbering scheme is a problem worth addressing.  Therefore I
support moving to a simple "version.patch" system for next year's release.
Having accepted the importance of accepting the broader community's
opinions changing to 10 now after we've released beta1 would be
counter-productive.

Personally, having come in around the 8.2 release I didn't, and quite
honestly still don't, care why the decision was made to release 9.0 instead
of 8.5.  Mainly because the feature oriented decisions to increment seems
to be largely focused on the DBA - which is it not my primary concern.  For
me, the inclusion of window functions would have been enough to warrant a
major version bump - and CTEs as well.  Both of those went into 8.4.  So
not only is it getting harder to gain critical mass in a single release
(which is good - it means we are not settling for the easy stuff and are
properly breaking down harder features into small releaseables) but the
stuff we are keying on are likely of secondary importance to a large number
of users.

David J.


Re: [HACKERS] 10.0

2016-05-13 Thread Mark Dilger

> On May 13, 2016, at 5:49 PM, Josh berkus  wrote:
> 
> On 05/13/2016 05:22 PM, Mark Dilger wrote:
>> Any project that starts inflating its numbering scheme sends a message to
>> users of the form, "hey, we've just been taken over by marketing people, 
>> and
>> software quality will go down from now on."
 
 I don't think this is about version number inflation, but actually more
 the opposite.  What you're calling the major number is really a marketing
 number.  There is not a technical distinction between major releases where
 we choose to bump the first number and those where we choose to bump the
 second.  It's all about marketing.  So to me, merging those numbers would
 be an anti-marketing move.  I think it's a good move: it would be more
 honest and transparent about what the numbers mean, not less so.
>> I find your argument persuasive if there is no possibility of ever needing
>> a major number to bump.  But if anything like what I described above could
>> someday happen, it seems the major.minor.micro format would come in
>> handy.  Perhaps the problem (from my perspective) is that the major number
>> has been used for purely marketing purposes in the past, and I've tried to
>> avert my eyes to that.  But going forward, my vote (worth less than half a
>> cent I'm sure) is to stop using it for marketing reasons.
> 
> Per a long discussion on -advocacy, nobody has any specific plans to do
> substantial breakage of backwards compatibility.  Theoretically we might
> someday want to change the on-disk format, but nobody has plans to do so
> in the immediate future.  How long should we hold out for that?  Until 9.27?
> 
> And I don't find dropping the "money" type to be substantial breakage.

If minor number bumps mean, "dump and restore is necessary, but client
programs are guaranteed to still work", then dropping the "money" type
is a different kind of action.  Maybe you don't think it matters much, but
for somebody who is using it in a client application, that matters.  Likewise,
somebody who has a struct in a client program that expects Oid to fit into
32 bits *needs* to go through their client programs if Oid ever gets changed
to 64 bit.  (Pigs will probably fly first, but just for instance.)

I'm not advocating any of these changes today.  I just think they are different
kinds of changes from adding new functions while leaving all old functions
in tact.  As cool as parallel query is (and I'd happily buy Robert or Amit a
beer) it doesn't break any functionality in client applications that ignore it,
so far as I can see.  Using parallel query as a motivation for increasing the
major number stems from a marketing instinct, not from sound engineering
practices.

Now that I've said that, I'll sit back and wait for the REL10_0_DEVEL
branch to appear in the repo.

mark





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David G. Johnston
On Fri, May 13, 2016 at 6:44 PM, Michael Banck  wrote:

> On Fri, May 13, 2016 at 05:31:00PM -0400, David G. Johnston wrote:
> > The underlying premise, for me, of choosing .4 or .5  was that presently
> we
> > discontinue support after 5 years/releases.  A new .0 would come out just
> > as we discontinue the previous .0
>
> Well maybe the 5 year support cycle would be nice to encode, but how is
> .0 different from .1 then?  You make sound like .0 would be similar to
> Ubuntu's LTS which is not the case, unless you want to propose that only
> .0 releases get 5 years and not the in-betweens? That'd be a shame.
>

​The opinion seems to be that major.0 is some kind of magic incantation in
the broader world of users...so its basically "since the last time we
incremented major this is what we've done - 80% of which is new to you only
if you've lived under a rock for the past four years".

David J.
​


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 05:22 PM, Mark Dilger wrote:
>>> >> Any project that starts inflating its numbering scheme sends a message to
>>> >> users of the form, "hey, we've just been taken over by marketing people, 
>>> >> and
>>> >> software quality will go down from now on."
>> > 
>> > I don't think this is about version number inflation, but actually more
>> > the opposite.  What you're calling the major number is really a marketing
>> > number.  There is not a technical distinction between major releases where
>> > we choose to bump the first number and those where we choose to bump the
>> > second.  It's all about marketing.  So to me, merging those numbers would
>> > be an anti-marketing move.  I think it's a good move: it would be more
>> > honest and transparent about what the numbers mean, not less so.
> I find your argument persuasive if there is no possibility of ever needing
> a major number to bump.  But if anything like what I described above could
> someday happen, it seems the major.minor.micro format would come in
> handy.  Perhaps the problem (from my perspective) is that the major number
> has been used for purely marketing purposes in the past, and I've tried to
> avert my eyes to that.  But going forward, my vote (worth less than half a
> cent I'm sure) is to stop using it for marketing reasons.

Per a long discussion on -advocacy, nobody has any specific plans to do
substantial breakage of backwards compatibility.  Theoretically we might
someday want to change the on-disk format, but nobody has plans to do so
in the immediate future.  How long should we hold out for that?  Until 9.27?

And I don't find dropping the "money" type to be substantial breakage.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Mark Dilger

> On May 13, 2016, at 5:00 PM, Tom Lane  wrote:
> 
> Mark Dilger  writes:
>> A major number change should indicate that something even bigger than on-disk
>> compatibility has changed, such as a change that precludes even a dump and
>> restore from working, or that breaks network communication protocols, etc.
> 
> If that were the standard, we'd never have bumped the major version at
> all, and would still be on 4.something (or whatever Berkeley was using
> when they tossed it over the wall; I'm not too clear on whether there was
> ever a 5.x release).  I can't imagine that we'd ever intentionally break
> dump/restore compatibility.  While we have changed the network protocol,
> and likely will again, we kept on supporting the old protocol alongside,
> and almost certainly would do so again.  There are too many compatibility
> constraints that we have to meet in order to be a usable database at all,
> so we're never going to just blow it up and start over.

Well, those are just examples.  Other candidate examples would be dropping
datatypes such as 'money' that were maybe not a good idea to begin with,
or somehow ditching the 'timestamp' vs. 'timestamptz' distinction, or 
changing to 64-bit Oids, such that software that interacts with postgresql
has to change accordingly.  Another such idea would be supporting 64-bit
varlena lengths and dropping large object support.  Another would be
dropping support for older communication protocols whose presence in the
server out of the box makes the server a security vulnerability.  (I am not
asserting the presence of such vulnerabilities, but just speculating on the
possibility that such vulnerabilities might be discovered in the future that
make it useful to ditch older, insecure protocols.)

I suspect other folks could add lots of other stuff to this list.


>> Any project that starts inflating its numbering scheme sends a message to
>> users of the form, "hey, we've just been taken over by marketing people, and
>> software quality will go down from now on."
> 
> I don't think this is about version number inflation, but actually more
> the opposite.  What you're calling the major number is really a marketing
> number.  There is not a technical distinction between major releases where
> we choose to bump the first number and those where we choose to bump the
> second.  It's all about marketing.  So to me, merging those numbers would
> be an anti-marketing move.  I think it's a good move: it would be more
> honest and transparent about what the numbers mean, not less so.

I find your argument persuasive if there is no possibility of ever needing
a major number to bump.  But if anything like what I described above could
someday happen, it seems the major.minor.micro format would come in
handy.  Perhaps the problem (from my perspective) is that the major number
has been used for purely marketing purposes in the past, and I've tried to
avert my eyes to that.  But going forward, my vote (worth less than half a
cent I'm sure) is to stop using it for marketing reasons.

mark



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Mark Dilger  writes:
> A major number change should indicate that something even bigger than on-disk
> compatibility has changed, such as a change that precludes even a dump and
> restore from working, or that breaks network communication protocols, etc.

If that were the standard, we'd never have bumped the major version at
all, and would still be on 4.something (or whatever Berkeley was using
when they tossed it over the wall; I'm not too clear on whether there was
ever a 5.x release).  I can't imagine that we'd ever intentionally break
dump/restore compatibility.  While we have changed the network protocol,
and likely will again, we kept on supporting the old protocol alongside,
and almost certainly would do so again.  There are too many compatibility
constraints that we have to meet in order to be a usable database at all,
so we're never going to just blow it up and start over.

> Any project that starts inflating its numbering scheme sends a message to
> users of the form, "hey, we've just been taken over by marketing people, and
> software quality will go down from now on."

I don't think this is about version number inflation, but actually more
the opposite.  What you're calling the major number is really a marketing
number.  There is not a technical distinction between major releases where
we choose to bump the first number and those where we choose to bump the
second.  It's all about marketing.  So to me, merging those numbers would
be an anti-marketing move.  I think it's a good move: it would be more
honest and transparent about what the numbers mean, not less so.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Mark Dilger

> On May 13, 2016, at 11:31 AM, Alvaro Herrera  wrote:
> 
> Josh berkus wrote:
> 
>> Anyway, can we come up with a consensus of some minimum changes it will
>> take to make the next version 10.0?
> 
> I think the next version should be 10.0 no matter what changes we put
> in.

-1

If I understand correctly, changing the micro version means that one or more
bugs have been fixed, but that the on-disk representation has not changed.  So
if I am running 9.3.2, I am at liberty to upgrade to 9.3.3 without a dump and
restore.

If the minor number has changed, new features have been added that require
a dump and restore.  As such, on 9.3.2, I would not be at liberty to upgrade to
9.4.0 without some extra effort.

A major number change should indicate that something even bigger than on-disk
compatibility has changed, such as a change that precludes even a dump and
restore from working, or that breaks network communication protocols, etc.

Any project that starts inflating its numbering scheme sends a message to
users of the form, "hey, we've just been taken over by marketing people, and
software quality will go down from now on."

mark

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres_fdw join pushdown - getting server crash in left outer join of three table

2016-05-13 Thread Michael Paquier
On Fri, May 13, 2016 at 11:14 PM, Robert Haas  wrote:
> So, barring objections, I intend to apply the attached fixup patch,
> which replaces Michael's logic with Ashutosh's logic and rewrites the
> comments such to be much more explicit.

Re-oops. I didn't check what was committed to be honest. And it should
not have been my version, definitely.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Greg Stark
On Fri, May 13, 2016 at 9:15 PM, Robert Haas  wrote:
> Man, I hate version number inflation.  I'm running Firefox 45.0.2, and
> I think that's crazy.  It hit 1.0 when were at aversion 7.4!

I don't see what's wrong with large numbers, it's not like there's a
shortage of numbers.

And for what it's worth the only reason we we were on 7.x at all is
that we went from 1.0.5 to 6.0 -- precisely because it was our sixth
release. If we had kept on that track we would have been on the same
plan Tom is proposing now.

One alternative to really drive home that the new scheme is a change
to people would be to do the same again. By my count 9.6 will be our
28th release. We could jump from 9.6 to Postgres 29.0. (That would
have the side benefit that we could retroactively number all the
previous releases in a consistent way if we wanted to)





-- 
greg


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Gavin Flower

On 14/05/16 09:31, David G. Johnston wrote:
On Friday, May 13, 2016, Tom Lane > wrote:


Robert Haas > writes:
> On Fri, May 13, 2016 at 2:49 PM, Tom Lane > wrote:

> If we don't want to stick with the current practice of debating when
> to bump the same digit, then let's agree that 10.0 will follow
9.6 and
> after that we'll bump the first digit after X.4, as we did with 7.X
> and 8.X.

It was absolute, utter chance that the 7.x and 8.x series both ended
with .4.  I see no reason to turn that into some kind of standard,
especially when it didn't work like that for 6.x or 9.x.


The underlying premise, for me, of choosing .4 or .5  was that 
presently we discontinue support after 5 years/releases.  A new .0 
would come out just as we discontinue the previous .0


As an added idea, if switching to major.patch, would be to make 10.0.x 
but then go to 11.x. Though that's somewhat easier cognitively it 
would probably be harder for developers that just ripping off the bandaid.


David J.

I think the idea of incrementing the first digit when ever the last 
major version reaches EOL, is probably the best compromise. If that was 
adopted, then people could plan changes that triggered major breakage in 
established usage accordingly.


I like the version 'x.y.z' approach, where 'z' mostly means bug fixes 
and minor tweaks, 'y' has more significant enhancements, and 'x' may 
mean major breakages.  Lets not go down the Mozilla version inflation route.


For now, probably best if we have 9.5.n, 9.6.n, 10.0.n (where obviously 
n starts at zero for each series!).



Cheers,

Gavin




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Academic help for Postgres

2016-05-13 Thread Thomas Munro
On Sat, May 14, 2016 at 7:13 AM, Bruce Momjian  wrote:
> I have incorporated suggestions from this email thread into my IEEE talk
> for next week:
>
> http://momjian.us/main/writings/pgsql/ieee.pdf
>
> You will see most of it in the new slides toward the end.  Please let me
> know if it needs more additions/changes.  Thanks.

Maybe slide 7 (NoSQL Sacrifices) should have a bullet point for
"transaction isolation"?

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

2016-05-13 Thread Michael Paquier
On Fri, May 13, 2016 at 11:49 PM, Alex Ignatov  wrote:
>
> On 13.05.2016 9:39, Michael Paquier wrote:
> Do we have any confidence that data file is not being corrupted? I.e
> contains some corrupted page? Can pg_basebackup check page checksum (db init
> with initdb -k) while backing up files?

That may be an idea to consider, for one class of users, though this
patch is not aimed at doing something regarding that.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 02:36 PM, Tom Lane wrote:

"Joshua D. Drake"  writes:

I mean we haven't yet actually identified a problem we are trying to
solve have we?


The problem I'd like to solve is not having to have this type of
discussion again in future years ...



Amen.

JD


regards, tom lane




--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 01:42 PM, Josh berkus wrote:

On 05/13/2016 01:04 PM, Joshua D. Drake wrote:

On 05/13/2016 12:03 PM, Josh berkus wrote:

On 05/13/2016 11:48 AM, Robert Haas wrote:

On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake
 wrote:



Anyway, all of this is a moot point, because nobody has the power to
tell the various companies what to do.  We're just lucky that everyone
is still committed to writing stuff which adds to PostgreSQL.


Lucky? No. We earned it. We earned it through years and years of hard
work. Should we be thankful? Absolutely. Should we be grateful that we
have such a powerful and engaged commercial contribution base? 100%.


Lucky.  Sure there was work and personal integrity involved, but like
any success story, there was luck.

But we've also been fortunate in not spawning hostile-but-popular forks
by people who left the project, and that none of the companies who
created hostile forks were very successful with them, and that nobody
has seriously tried using lawyers to control/ruin the project.


I can't get behind you on this. Everything you have said above has to do 
with the hard work and integrity of the people in this project. It isn't 
luck or divine intervention.




And, most importantly, we've been lucky that a lot of competing projects
have self-immolated instead of being successful and brain-draining our
contributors (MySQL, ANTS, MonetDB, etc.)



Actually there are people that have been drained out, I won't name them 
but it is pretty easy to figure out who they are. The people that are 
left, especially the long timers are here because of their integrity and 
attachment to the project.


This project builds good people, and the good people build a good project.

I am not going to buy into a luck story for something I and many others 
have invested decades of their life into.


JD




--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Andrew Dunstan  writes:
> I don't have any strong opinions about this. It's essentially a 
> marketing decision, and I'm happy to leave that to others. If and when 
> we do change, I'd like to put in a modest request that we add an extra _ 
> to the branch names, like this: REL_10_0_STABLE. That would mean they 
> would sort nicely, which would make my life simpler in a few places in 
> the buildfarm code. If not, I'd like a little advance notice so I can 
> check all the places where we compare branch names.

If we do decide to change the numbering strategy, there are quite a
few small details that probably ought to be fixed while we're at it.
I think it'd be a good idea to start separating "devel" or "betaN"
with a dot, for instance, like "10.devel" not "10devel".  But it's
likely premature to get into those sorts of details, since it's not
clear to me that we have a consensus to change at all.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Bruce Momjian  writes:
> On Fri, May 13, 2016 at 05:36:50PM -0400, Tom Lane wrote:
>> "Joshua D. Drake"  writes:
>>> I mean we haven't yet actually identified a problem we are trying to 
>>> solve have we?

>> The problem I'd like to solve is not having to have this type of
>> discussion again in future years ...

> If pg_logical and/or more parallelism are in 9.7, there will be no need
> for a discussion, just like 8.0 and 9.0 decisions.

If you're framing the problem as "how to decide whether next year is 9.7
or 10.0", you're not thinking very far ahead.  If we just leave the
process at status quo, we'll be having the exact same type of discussion
not later than about 10.3, because that's about how long it took for
people to start asking about 9.0:
http://www.postgresql.org/message-id/200704231224.15429.j...@agliodbs.com
or 10.0:
http://www.postgresql.org/message-id/512e8ef8.3000...@agliodbs.com

(I'm not trying to pick on Josh here, those were just the first hits
I found while scanning the -advocacy archives.)

I agree with the opinion that we waste an inordinate amount of effort
on this type of discussion, and have been doing so for a couple of
decades now.  I'd like a permanent fix for that.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Andrew Dunstan



On 05/13/2016 05:12 PM, Tom Lane wrote:

An analogy that might get some traction among database geeks is that
version numbers are a sort of surrogate key, and assigning meaning to
surrogate keys is a bad idea.




:-)

I agree year-based numbers will cause us grief.

I don't have any strong opinions about this. It's essentially a 
marketing decision, and I'm happy to leave that to others. If and when 
we do change, I'd like to put in a modest request that we add an extra _ 
to the branch names, like this: REL_10_0_STABLE. That would mean they 
would sort nicely, which would make my life simpler in a few places in 
the buildfarm code. If not, I'd like a little advance notice so I can 
check all the places where we compare branch names.


cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 05:36:50PM -0400, Tom Lane wrote:
> "Joshua D. Drake"  writes:
> > I mean we haven't yet actually identified a problem we are trying to 
> > solve have we?
> 
> The problem I'd like to solve is not having to have this type of
> discussion again in future years ...

If pg_logical and/or more parallelism are in 9.7, there will be no need
for a discussion, just like 8.0 and 9.0 decisions.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
"Joshua D. Drake"  writes:
> I mean we haven't yet actually identified a problem we are trying to 
> solve have we?

The problem I'd like to solve is not having to have this type of
discussion again in future years ...

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Magnus Hagander
On May 13, 2016 23:27, "Joshua D. Drake"  wrote:
>
> On 05/13/2016 02:22 PM, Magnus Hagander wrote:
>>
>>
>
>> Using something like .2.0 for the second one in the same year
>> could be suggested, but to me that sounds like the worst of both worlds.
>
>
> The amount of brain cycles, electricity, taxes on internet connectivity
and transcontinental data spent on the discussion of version numbers could
have finished the release by now.
>
> How about we just call it 9.6?
>

I'm pretty sure most people aren't talking about this one anymore, it's
about the next one. I certainly am.

9.6 was pretty much decided when we put out the beta, tbh.

/Magnus


Re: [HACKERS] 10.0

2016-05-13 Thread David G. Johnston
On Friday, May 13, 2016, Tom Lane  wrote:

> Robert Haas > writes:
> > On Fri, May 13, 2016 at 2:49 PM, Tom Lane  > wrote:
>
> > If we don't want to stick with the current practice of debating when
> > to bump the same digit, then let's agree that 10.0 will follow 9.6 and
> > after that we'll bump the first digit after X.4, as we did with 7.X
> > and 8.X.
>
> It was absolute, utter chance that the 7.x and 8.x series both ended
> with .4.  I see no reason to turn that into some kind of standard,
> especially when it didn't work like that for 6.x or 9.x.
>

The underlying premise, for me, of choosing .4 or .5  was that presently we
discontinue support after 5 years/releases.  A new .0 would come out just
as we discontinue the previous .0

As an added idea, if switching to major.patch, would be to make 10.0.x but
then go to 11.x. Though that's somewhat easier cognitively it would
probably be harder for developers that just ripping off the bandaid.

David J.


Re: [HACKERS] 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 02:22 PM, Magnus Hagander wrote:





Using something like .2.0 for the second one in the same year
could be suggested, but to me that sounds like the worst of both worlds.


The amount of brain cycles, electricity, taxes on internet connectivity 
and transcontinental data spent on the discussion of version numbers 
could have finished the release by now.


How about we just call it 9.6?

I mean we haven't yet actually identified a problem we are trying to 
solve have we?


Sincerely,

JD
--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Magnus Hagander
On Fri, May 13, 2016 at 11:14 PM, Bruce Momjian  wrote:

> On Fri, May 13, 2016 at 02:06:26PM -0700, Josh Berkus wrote:
> > On 05/13/2016 02:00 PM, Tom Lane wrote:
> > > I still don't like that much, and just thought of another reason why:
> > > it would foreclose doing two major releases per year.  We have debated
> > > that sort of schedule in the past.  While I don't see any reason to
> > > think we'd try to do it in the near future, it would be sad if we
> > > foreclosed the possibility by a poor choice of versioning scheme.
> >
> > Well, we have done two major releases in a year before, mostly due to
> > one release being late and the succeeding one being on time.
>
> Uh, guys, we just did it:
>
> 9.5 2016-01-07
> 9.6 2016-09-??
>

Let's not get ahead of ourselves, we haven't actually released 9.6 yet. It
could slip, let's not tempt fate :P

That said, count me in the -1 camp for using a year number. Because it
limits us.

Using something like .2.0 for the second one in the same year could
be suggested, but to me that sounds like the worst of both worlds.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: [HACKERS] 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 02:06:26PM -0700, Josh Berkus wrote:
> On 05/13/2016 02:00 PM, Tom Lane wrote:
> > I still don't like that much, and just thought of another reason why:
> > it would foreclose doing two major releases per year.  We have debated
> > that sort of schedule in the past.  While I don't see any reason to
> > think we'd try to do it in the near future, it would be sad if we
> > foreclosed the possibility by a poor choice of versioning scheme.
> 
> Well, we have done two major releases in a year before, mostly due to
> one release being late and the succeeding one being on time.

Uh, guys, we just did it:

9.5 2016-01-07
9.6 2016-09-??

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 03:48:39PM -0500, Merlin Moncure wrote:
> Any versioning system that removes subjective criteria is good.  These
> debates in interminable and always have been.  Personally I would go
> with something even more antiseptic like basing the version on the
> year, where year is defined at the 'point of no return' -- going beta
> for example.

Uh, the switch to 8.0 and 9.0 were quite clear discussions, unlike 10.0.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Josh berkus  writes:
> On 05/13/2016 02:00 PM, Tom Lane wrote:
>> I still don't like that much, and just thought of another reason why:
>> it would foreclose doing two major releases per year.  We have debated
>> that sort of schedule in the past.  While I don't see any reason to
>> think we'd try to do it in the near future, it would be sad if we
>> foreclosed the possibility by a poor choice of versioning scheme.

> Well, we have done two major releases in a year before, mostly due to
> one release being late and the succeeding one being on time.

What I was on about in this case was the idea of a six-month major release
cycle, which I definitely remember being discussed more-or-less-seriously
in the past.  The question of what to do with a release that slips past
December 31st is distinct from that, though it would also be annoying
if we're using year-based numbers.

An analogy that might get some traction among database geeks is that
version numbers are a sort of surrogate key, and assigning meaning to
surrogate keys is a bad idea.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Justin Clift
On 13 May 2016, at 21:42, Josh berkus  wrote:
> On 05/13/2016 01:04 PM, Joshua D. Drake wrote:
>> On 05/13/2016 12:03 PM, Josh berkus wrote:
>>> On 05/13/2016 11:48 AM, Robert Haas wrote:
 On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake
  wrote:
>> 
>>> Anyway, all of this is a moot point, because nobody has the power to
>>> tell the various companies what to do.  We're just lucky that everyone
>>> is still committed to writing stuff which adds to PostgreSQL.
>> 
>> Lucky? No. We earned it. We earned it through years and years of hard
>> work. Should we be thankful? Absolutely. Should we be grateful that we
>> have such a powerful and engaged commercial contribution base? 100%.
> 
> Lucky.  Sure there was work and personal integrity involved, but like
> any success story, there was luck.
> 
> But we've also been fortunate in not spawning hostile-but-popular forks
> by people who left the project, and that none of the companies who
> created hostile forks were very successful with them, and that nobody
> has seriously tried using lawyers to control/ruin the project.
> 
> And, most importantly, we've been lucky that a lot of competing projects
> have self-immolated instead of being successful and brain-draining our
> contributors (MySQL, ANTS, MonetDB, etc.)

Oracle buying MySQL (via Sun) seems to have helped things along pretty
well too. :)

+ Justin

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 02:00 PM, Tom Lane wrote:
> I still don't like that much, and just thought of another reason why:
> it would foreclose doing two major releases per year.  We have debated
> that sort of schedule in the past.  While I don't see any reason to
> think we'd try to do it in the near future, it would be sad if we
> foreclosed the possibility by a poor choice of versioning scheme.

Well, we have done two major releases in a year before, mostly due to
one release being late and the succeeding one being on time.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Merlin Moncure  writes:
> Any versioning system that removes subjective criteria is good.  These
> debates in interminable and always have been.

Yeah, I got bored of the topic after about 8.0 ;-)

> Personally I would go
> with something even more antiseptic like basing the version on the
> year, where year is defined at the 'point of no return' -- going beta
> for example.

I still don't like that much, and just thought of another reason why:
it would foreclose doing two major releases per year.  We have debated
that sort of schedule in the past.  While I don't see any reason to
think we'd try to do it in the near future, it would be sad if we
foreclosed the possibility by a poor choice of versioning scheme.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Christian Ullrich

* Tom Lane wrote:


So I think we should solve these problems at a stroke, and save ourselves
lots of breath in the future, by getting rid of the whole "major major"
idea and going over to a two-part version numbering scheme.  To be
specific:

* This year's major release will be 9.6.0, with minor updates 9.6.1,
9.6.2, etc.  It's too late to do otherwise for this release cycle.

* Next year's major release will be 10.0, with minor updates 10.1,
10.2, etc.

* The year after, 11.0.  Etc cetera.


When threaded Postgres is introduced in 2021, the version goes from 13 
to 14. Not quite worthy of that momentous occasion.


Losing a third of the version number means losing the ability to 
differentiate between significant-but-still-incremental improvements and 
(never-again)-stop-the-presses-this-is-great improvements; it's all just 
another major version. (Unless you'd want to jump from 13 to 20, 
effectively making the first digit special.)


Finally, from the, er, visual point of view, 10.1 and 10.2 look good and 
modern, but what about 10.13, 10.17, 10.22?


* Elsewhere, Tom Lane wrote:

> There's also the problem that the current numbering scheme confuses
> people who aren't familiar with the project.  How many times have
> you seen people say "I'm using Postgres 8" or "Postgres 9" when asked
> what version they're on?

Knowing only the first component would be very nearly as useless with 
two-part version numbers as it is today. The conversation will continue 
just the same:


- "No, what is the exact version?"
- ...
- "There have been several updates for that version since. Can you
  please update to the latest one and see if the problem persists?"

In short, -1 from me, but I don't matter much ...


PS: Three of the five 9.x micro versions are prime right now. Any
chance an extra 9.1 release could be arranged?

--
Christian



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Robert Haas  writes:
> On Fri, May 13, 2016 at 2:49 PM, Tom Lane  wrote:
>> So I think we should solve these problems at a stroke, and save ourselves
>> lots of breath in the future, by getting rid of the whole "major major"
>> idea and going over to a two-part version numbering scheme.

> Man, I hate version number inflation.  I'm running Firefox 45.0.2, and
> I think that's crazy.  It hit 1.0 when were at aversion 7.4!  Granted,
> this wouldn't be that bad, but I have always thought that burning
> through a first digit a few times a decade is much more sensible than
> doing it every year.

Firefox's scheme is weird because they're not really using the back half
of the number at all (not to mention that they then throw in a third part
anyway, some of the time).  If we use up one major-release number per
year, that seems fine to me.

> If we don't want to stick with the current practice of debating when
> to bump the same digit, then let's agree that 10.0 will follow 9.6 and
> after that we'll bump the first digit after X.4, as we did with 7.X
> and 8.X.

It was absolute, utter chance that the 7.x and 8.x series both ended
with .4.  I see no reason to turn that into some kind of standard,
especially when it didn't work like that for 6.x or 9.x.  If we're to go
with a scheme like that, I'd agree with what somebody suggested upthread:
run up to .9 before bumping the first number (ie, 10.0.x would follow
9.9.x).  But this would still leave us with somebody, as sure as the sun
rises in the east, agitating every year to rethink the next release's
number.  That argument is only going to go away if we change the version
number layout so that all major releases look the same; otherwise there
is just too much temptation to argue about it.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Merlin Moncure
On Fri, May 13, 2016 at 1:49 PM, Tom Lane  wrote:
> Alvaro Herrera  writes:
>> Josh berkus wrote:
>>> Anyway, can we come up with a consensus of some minimum changes it will
>>> take to make the next version 10.0?
>
>> I think the next version should be 10.0 no matter what changes we put
>> in.
>
> Here's my two cents: we called 8.0 that on the basis of the native Windows
> port, and 9.0 that on the basis of getting in-core replication support.
> Both of those were game-changing features that deserved a "major major"
> version number bump.  But as the project matures, it gets harder and
> harder to come up with game-changing features in the span of a single
> release.  Parallel query is a great example: a fully mature parallel query
> feature would, IMO, easily justify a 10.0 moniker.  But what we have today
> is more like half or two-thirds of a feature.  If we call this release
> 10.0 on the strength of that, we could justifiably be accused of
> overselling it.  On the other hand, if we wait till next year when
> parallelism presumably will be much more fully baked, it'll be a bit
> anticlimactic to call it 10.0 then.  This isn't going to get better with
> other major features that can be expected to appear in future.  So we can
> expect to continue to waste lots of time debating the "what to call it"
> question, in pretty much every year except for the one or two immediately
> after a "major major" bump.
>
> There's also the problem that the current numbering scheme confuses
> people who aren't familiar with the project.  How many times have
> you seen people say "I'm using Postgres 8" or "Postgres 9" when asked
> what version they're on?
>
> So I think we should solve these problems at a stroke, and save ourselves
> lots of breath in the future, by getting rid of the whole "major major"
> idea and going over to a two-part version numbering scheme.  To be
> specific:
>
> * This year's major release will be 9.6.0, with minor updates 9.6.1,
> 9.6.2, etc.  It's too late to do otherwise for this release cycle.
>
> * Next year's major release will be 10.0, with minor updates 10.1,
> 10.2, etc.
>
> * The year after, 11.0.  Etc cetera.

+1

Any versioning system that removes subjective criteria is good.  These
debates in interminable and always have been.  Personally I would go
with something even more antiseptic like basing the version on the
year, where year is defined at the 'point of no return' -- going beta
for example.

merlin


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Vitaly Burovoy
On 5/13/16, Robert Haas  wrote:
> On Fri, May 13, 2016 at 2:49 PM, Tom Lane  wrote:
>> So I think we should solve these problems at a stroke, and save ourselves
>> lots of breath in the future, by getting rid of the whole "major major"
>> idea and going over to a two-part version numbering scheme.  To be
>> specific:
>>
>> * This year's major release will be 9.6.0, with minor updates 9.6.1,
>> 9.6.2, etc.  It's too late to do otherwise for this release cycle.
>>
>> * Next year's major release will be 10.0, with minor updates 10.1,
>> 10.2, etc.
>>
>> * The year after, 11.0.  Etc cetera.
>>
>> No confusion, no surprises, no debate ever again about what the next
>> version number is.
>>
>> This is by no means a new idea, but I think its time has come.
>
> Man, I hate version number inflation.

+1

> I'm running Firefox 45.0.2, and I think that's crazy. It hit 1.0 when were
> at aversion 7.4! Granted, this wouldn't be that bad, but

+1
Big numbers in version don't show big (or even equal) changes. How to
compare 7.0…8.0 and 8.0…9.0 and 9.0…9.6(10.0?)?
Is difference between FireFox 10.0…20.0 the same as FireFox 20.0…30.0?

> I have always thought that burning through a first digit a few times a decade
> is much more sensible than doing it every year.
> We just have to remember to bump the first digit occasionally.

+1
Better once a decade. =)

> If we don't want to stick with the current practice of debating when
> to bump the same digit, then let's agree that 10.0 will follow 9.6 and
> after that we'll bump the first digit after X.4, as we did with 7.X
> and 8.X.

Why just don't agree that since each release has good features and we
can't release multiple backward-incompatible features at one release
there is no sense of bumping the first digit earlier than the second
one reaches the value "9"?

Pretend the current version is not 9.5, but 95 (PostgreSQL 95),
current beta not 9.6 but 96 etc, but just divided by 10.

In fact it is similar to Tom's offer but avoided proposed bump to 10.0
and left increasing by 0.1 instead of 1.0:
curnt   Tom's proposed
 9.5.X  9.5.X
 9.6.X  9.6.X
 9.7.X 10.X
 9.8.X 11.X
 9.9.X 12.X
10.0.X 13.X
10.1.X 14.X
10.2.X 15.X
...
10.6.X 19.X
10.7.X 20.X
10.8.X 21.X
10.9.X 22.X
11.0.X 23.X
11.1.X 24.X

Etc.

> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company

--
Best regards,
Vitaly Burovoy


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 01:04 PM, Joshua D. Drake wrote:
> On 05/13/2016 12:03 PM, Josh berkus wrote:
>> On 05/13/2016 11:48 AM, Robert Haas wrote:
>>> On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake
>>>  wrote:
> 
>> Anyway, all of this is a moot point, because nobody has the power to
>> tell the various companies what to do.  We're just lucky that everyone
>> is still committed to writing stuff which adds to PostgreSQL.
> 
> Lucky? No. We earned it. We earned it through years and years of hard
> work. Should we be thankful? Absolutely. Should we be grateful that we
> have such a powerful and engaged commercial contribution base? 100%.

Lucky.  Sure there was work and personal integrity involved, but like
any success story, there was luck.

But we've also been fortunate in not spawning hostile-but-popular forks
by people who left the project, and that none of the companies who
created hostile forks were very successful with them, and that nobody
has seriously tried using lawyers to control/ruin the project.

And, most importantly, we've been lucky that a lot of competing projects
have self-immolated instead of being successful and brain-draining our
contributors (MySQL, ANTS, MonetDB, etc.)

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 01:15 PM, Robert Haas wrote:
> On Fri, May 13, 2016 at 2:49 PM, Tom Lane  wrote:
>> So I think we should solve these problems at a stroke, and save ourselves
>> lots of breath in the future, by getting rid of the whole "major major"
>> idea and going over to a two-part version numbering scheme.  To be
>> specific:
>>
>> * This year's major release will be 9.6.0, with minor updates 9.6.1,
>> 9.6.2, etc.  It's too late to do otherwise for this release cycle.
>>
>> * Next year's major release will be 10.0, with minor updates 10.1,
>> 10.2, etc.
>>
>> * The year after, 11.0.  Etc cetera.
>>
>> No confusion, no surprises, no debate ever again about what the next
>> version number is.
>>
>> This is by no means a new idea, but I think its time has come.
> 
> Man, I hate version number inflation.  I'm running Firefox 45.0.2, and
> I think that's crazy.  It hit 1.0 when were at aversion 7.4!  Granted,
> this wouldn't be that bad, but I have always thought that burning
> through a first digit a few times a decade is much more sensible than
> doing it every year.  We just have to remember to bump the first digit
> occasionally.

Well, FF has this issue because they release a new version every 6
weeks.  Even bumping once per year, we wouldn't hit version 20 until 2027.

> If we don't want to stick with the current practice of debating when
> to bump the same digit, then let's agree that 10.0 will follow 9.6 and
> after that we'll bump the first digit after X.4, as we did with 7.X
> and 8.X.

Why X.4?  Seems arbitrary.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Merlin Moncure
On Fri, May 13, 2016 at 2:05 PM, Robert Haas  wrote:
> Now, where this gets tricky is when it comes down to whether the
> end-product of that effort is something the community wants.  We all
> need to be careful not to make our corporate priorities into community
> priorities.  Features shouldn't get committed without a consensus that
> they are both useful and well-implemented, and prior discussion is a
> good way to achieve that.  On the whole, I think we've done reasonably
> well in this area.  There is often disagreement but in the end I think
> usually end up in a place that is good for PostgreSQL.  Hopefully that
> will continue.

PostgreSQL is a miracle of good development practices.  Companies can
collaborate or contribute any manner the see fit along with the
community at large.  Demands made on others based on opinion and/of
selfish benefit gets you nowhere fast and that's a good thing.

Open source projects, especially the bigger/better run ones, tend to
migrate towards development processes that are highly informal and
great in terms of the ratio of effort to progress in all things,
including and especially, collaboration. I regularly emulate the way
things are done here as best I can which tends to completely freak out
the more corporate type developers that run most of the bigger shops.
If I'm writing emails around here, it's probably because five minutes
earlier some developer refused to fix a bug because the resolution
wasn't notated properly in some byzantine requirements document.
Solace.

merlin


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Robert Haas
On Fri, May 13, 2016 at 2:49 PM, Tom Lane  wrote:
> So I think we should solve these problems at a stroke, and save ourselves
> lots of breath in the future, by getting rid of the whole "major major"
> idea and going over to a two-part version numbering scheme.  To be
> specific:
>
> * This year's major release will be 9.6.0, with minor updates 9.6.1,
> 9.6.2, etc.  It's too late to do otherwise for this release cycle.
>
> * Next year's major release will be 10.0, with minor updates 10.1,
> 10.2, etc.
>
> * The year after, 11.0.  Etc cetera.
>
> No confusion, no surprises, no debate ever again about what the next
> version number is.
>
> This is by no means a new idea, but I think its time has come.

Man, I hate version number inflation.  I'm running Firefox 45.0.2, and
I think that's crazy.  It hit 1.0 when were at aversion 7.4!  Granted,
this wouldn't be that bad, but I have always thought that burning
through a first digit a few times a decade is much more sensible than
doing it every year.  We just have to remember to bump the first digit
occasionally.

If we don't want to stick with the current practice of debating when
to bump the same digit, then let's agree that 10.0 will follow 9.6 and
after that we'll bump the first digit after X.4, as we did with 7.X
and 8.X.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 12:03 PM, Josh berkus wrote:

On 05/13/2016 11:48 AM, Robert Haas wrote:

On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake  
wrote:



Anyway, all of this is a moot point, because nobody has the power to
tell the various companies what to do.  We're just lucky that everyone
is still committed to writing stuff which adds to PostgreSQL.


Lucky? No. We earned it. We earned it through years and years of hard 
work. Should we be thankful? Absolutely. Should we be grateful that we 
have such a powerful and engaged commercial contribution base? 100%.



JD



--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 12:05 PM, Robert Haas wrote:

On Fri, May 13, 2016 at 12:35 PM, Joshua D. Drake  
wrote:

Hey, if I am wrong that's awesome. The impression I have is the general
workflow is this:



The difference being one of coopetition versions competition for the
betterment of the community. If there are companies that are doing that
already, that is awesome and I applaud it. I was just trying to further
drive that idea home.


I think that's already happening.  I'm happy to see more of it.  In
practical terms, though, it's harder to collaborate between companies
because then you need two management teams to be on-board with it, and
there can be other competing priorities.


Yep, that's true.



If either company needs to
pull staff of a project because of some competing priority (say,
fixing a broken customer or addressing an urgent customer need), then
the whole project can stall.  The whole wagon train moves at the pace
of the slowest camel.  It's nice when we can collaborate across
companies and I'm all for it, but sometimes it's faster to for a
single company to just assign a couple of people to a project and tell
them to go do it.

Now, where this gets tricky is when it comes down to whether the
end-product of that effort is something the community wants.  We all
need to be careful not to make our corporate priorities into community
priorities.  Features shouldn't get committed without a consensus that
they are both useful and well-implemented, and prior discussion is a
good way to achieve that.  On the whole, I think we've done reasonably
well in this area.  There is often disagreement but in the end I think
usually end up in a place that is good for PostgreSQL.  Hopefully that
will continue.



+1

Sincerely,

JD

--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Andres Freund  writes:
> I'm in favor of doing something more radical than just stripping one
> digit off. We've tried (and only partially failed) to educate users that
> $major.$minor updates are the big ones; if we change that to essentially
> be $major.$micro, we'll have people not updating and such.

> So I'd rather go with 2016.01v0 or something.

Meh.  We could go with using the year of release as the major version;
that wouldn't be functionally different from what I suggested.  But
I don't want to invent some whole new notation.  That would break
version-comparison scripts for no good reason.  (As an ex-packager for
Red Hat, I know that people who invent their very own version notations
are cordially hated by all distros everywhere.  Let's stick to decimal
numbers with dots between, please.)

Although year-of-release would definitely have the advantage of making
it clear to everybody that Something Changed, I think it would still
be confusing over time.  "Why are you releasing 2017.6 in 2019?"
This is basically the same reason we got rid of the "Postgres95" name,
I believe, though I wasn't around at the time.  There's also the issue
I mentioned upthread that it becomes problematic if a release slips into
January.

My feeling is that going from 9 to 10 will in itself be a pretty good
boundary marker, in a way that wouldn't apply if we were trying to
introduce this scheme starting with 9 or 11.  So this is an ideal
opportunity to get it done.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Perf Benchmarking and regression.

2016-05-13 Thread Andres Freund
On 2016-05-13 14:43:15 -0400, Robert Haas wrote:
> On Fri, May 13, 2016 at 1:43 PM, Andres Freund  wrote:
> > I just want to emphasize what we're discussing here is a bit of an
> > extreme setup. A workload that's bigger than shared buffers, but smaller
> > than the OS's cache size; with a noticeable likelihood of rewriting
> > individual OS page cache pages within 30s.
>
> You're just describing pgbench with a scale factor too large to fit in
> shared_buffers.

Well, that *and* a scale factor smaller than 20% of the memory
available, *and* a scale factor small enough that make re-dirtying of
already written out pages likely.


> I think it's unfair to paint that as some kind of niche use case.

I'm not saying we don't need to do something about it. Just that it's a
hard tradeoff to make. The massive performance / latency we've observed
originate from the kernel caching too much dirty IO. The fix is making
is cache fewer dirty pages.  But there's workloads where the kernel's
buffer cache works as an extension of our page cache.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Josh berkus  writes:
> On 05/13/2016 11:49 AM, Tom Lane wrote:
>> So I think we should solve these problems at a stroke, and save ourselves
>> lots of breath in the future, by getting rid of the whole "major major"
>> idea and going over to a two-part version numbering scheme.

> I'm for it.

> Note that we will need to do a *bunch* of education around the change in
> version numbering schemes.  And a bunch of people and packagers will
> need to change their version comparison scripts (while everyone should
> be using the sortable version numbers, not everyone does).

Indeed.

> So if we're going to make that change, I suggest doing it *now* to get
> the word out.

Well, actually, part of the reason for proposing that we start it with the
next release cycle is that I think we need lead time to make it happen.
If we try to replace "9.6" with "10" at this stage of the cycle, it's
going to be a mess.  But if we start using that numbering scheme when
we fork the next development branch, there will be time for people to
get used to it.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Andres Freund
Hi,

On 2016-05-13 12:36:00 -0700, Josh berkus wrote:
> On 05/13/2016 11:49 AM, Tom Lane wrote:
> > Alvaro Herrera  writes:
> >> Josh berkus wrote:
> > No confusion, no surprises, no debate ever again about what the next
> > version number is.
> > 
> > This is by no means a new idea, but I think its time has come.
> 
> I'm for it.

I'm not against evolving the numbering scheme. There's a good number of
reasons why the current one is problematic (for me foremost these
discussions ;)).

But because of:

> Note that we will need to do a *bunch* of education around the change in
> version numbering schemes.  And a bunch of people and packagers will
> need to change their version comparison scripts (while everyone should
> be using the sortable version numbers, not everyone does).

I'm in favor of doing something more radical than just stripping one
digit off. We've tried (and only partially failed) to educate users that
$major.$minor updates are the big ones; if we change that to essentially
be $major.$micro, we'll have people not updating and such.

So I'd rather go with 2016.01v0 or something.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 11:49 AM, Tom Lane wrote:
> Alvaro Herrera  writes:
>> Josh berkus wrote:
>>> Anyway, can we come up with a consensus of some minimum changes it will
>>> take to make the next version 10.0?
> 
>> I think the next version should be 10.0 no matter what changes we put
>> in.
> 
> Here's my two cents: we called 8.0 that on the basis of the native Windows
> port, and 9.0 that on the basis of getting in-core replication support.
> Both of those were game-changing features that deserved a "major major"
> version number bump.  But as the project matures, it gets harder and
> harder to come up with game-changing features in the span of a single
> release.  Parallel query is a great example: a fully mature parallel query
> feature would, IMO, easily justify a 10.0 moniker.  But what we have today
> is more like half or two-thirds of a feature.  If we call this release
> 10.0 on the strength of that, we could justifiably be accused of
> overselling it.  On the other hand, if we wait till next year when
> parallelism presumably will be much more fully baked, it'll be a bit
> anticlimactic to call it 10.0 then.  This isn't going to get better with
> other major features that can be expected to appear in future.  So we can
> expect to continue to waste lots of time debating the "what to call it"
> question, in pretty much every year except for the one or two immediately
> after a "major major" bump.
> 
> There's also the problem that the current numbering scheme confuses
> people who aren't familiar with the project.  How many times have
> you seen people say "I'm using Postgres 8" or "Postgres 9" when asked
> what version they're on?
> 
> So I think we should solve these problems at a stroke, and save ourselves
> lots of breath in the future, by getting rid of the whole "major major"
> idea and going over to a two-part version numbering scheme.  To be
> specific:
> 
> * This year's major release will be 9.6.0, with minor updates 9.6.1,
> 9.6.2, etc.  It's too late to do otherwise for this release cycle.
> 
> * Next year's major release will be 10.0, with minor updates 10.1,
> 10.2, etc.
> 
> * The year after, 11.0.  Etc cetera.
> 
> No confusion, no surprises, no debate ever again about what the next
> version number is.
> 
> This is by no means a new idea, but I think its time has come.

I'm for it.

Note that we will need to do a *bunch* of education around the change in
version numbering schemes.  And a bunch of people and packagers will
need to change their version comparison scripts (while everyone should
be using the sortable version numbers, not everyone does).

So if we're going to make that change, I suggest doing it *now* to get
the word out.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Academic help for Postgres

2016-05-13 Thread Bruce Momjian
On Thu, May 12, 2016 at 09:47:02AM +0200, Michael Banck wrote:
> On Thu, May 12, 2016 at 08:57:34AM +0800, Craig Ringer wrote:
> > On 11 May 2016 at 22:20, Bruce Momjian  wrote:
> > > I am giving a keynote at an IEEE database conference in Helsinki next
> > > week (http://icde2016.fi/).  (Yes, I am not attending PGCon Ottawa
> > > because I accepted the Helsinki conference invitation before the PGCon
> > > Ottawa date was changed from June to May).
> > >
> > > As part of the keynote, I would like to mention areas where academia can
> > > help us.  The topics I can think of are:
> > >
> > > Any others?
> > >
> > 
> > When publishing work, publish source code somewhere stable that won't just
> > vanish. And build on the latest stable release, don't build your prototype
> > on Pg 8.0. Don't just publish a tarball with no information about what
> > revision it's based on, publish a git tree or a patch series.
> > 
> > While academic prototype source is rarely usable directly, it can serve a
> > valuable role with helping to understand the changes that were made,
> > reproducing results, exploring further related work, etc
> > 
> > Include your dummy data or data generators, setup scripts, etc.
> 
> That is all sound advise, but if they do all of the above, then they
> should also make sure the source (or parts of it) is potentially usable
> by the project, i.e. (joint?) PGDG copyright, if their academic
> institution allows that.

I have incorporated suggestions from this email thread into my IEEE talk
for next week:

http://momjian.us/main/writings/pgsql/ieee.pdf

You will see most of it in the new slides toward the end.  Please let me
know if it needs more additions/changes.  Thanks.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Robert Haas
On Fri, May 13, 2016 at 12:35 PM, Joshua D. Drake  
wrote:
> Hey, if I am wrong that's awesome. The impression I have is the general
> workflow is this:
>
> * Company(1) discusses feature with community
> * Company(1) works on patch/feature for a period of time
> * Company(1) delivers patch to community
> * Standard operation continues (patch review, discussion, etc..)
>
> This is not "bad" but it isn't as productive as something like this would
> be:
>
> * Company(1) + Company(2) get together and discuss using their
> respective resources to collaboratively develop X (multi-master for
> example).
>
> * Company(1) + Company(2) discuss feature with community
> * Company(1) + Company(2) work on patch/feature in the open,
> together
> * Company(1) + Company(2) deliver patch to community
> * Standard operation continues (patch review, discussion, etc...)
>
> The difference being one of coopetition versions competition for the
> betterment of the community. If there are companies that are doing that
> already, that is awesome and I applaud it. I was just trying to further
> drive that idea home.

I think that's already happening.  I'm happy to see more of it.  In
practical terms, though, it's harder to collaborate between companies
because then you need two management teams to be on-board with it, and
there can be other competing priorities.  If either company needs to
pull staff of a project because of some competing priority (say,
fixing a broken customer or addressing an urgent customer need), then
the whole project can stall.  The whole wagon train moves at the pace
of the slowest camel.  It's nice when we can collaborate across
companies and I'm all for it, but sometimes it's faster to for a
single company to just assign a couple of people to a project and tell
them to go do it.

Now, where this gets tricky is when it comes down to whether the
end-product of that effort is something the community wants.  We all
need to be careful not to make our corporate priorities into community
priorities.  Features shouldn't get committed without a consensus that
they are both useful and well-implemented, and prior discussion is a
good way to achieve that.  On the whole, I think we've done reasonably
well in this area.  There is often disagreement but in the end I think
usually end up in a place that is good for PostgreSQL.  Hopefully that
will continue.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 11:48 AM, Robert Haas wrote:
> On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake  
> wrote:
>> Singular point contribution is not the point of my argument. My point is
>> that if three people from EDB and three people from Citus got together and
>> worked on a project in full collaboration it would be more beneficial to the
>> project.
> 
> Well, the scalability work in 9.6 went almost exactly like this,
> assuming you count Andres as three people (which is entirely
> reasonable) and Dilip, Mithun, Amit, and myself as three people (which
> is maybe less reasonable, since I don't really want any of us counted
> as less than a whole person).

Frankly, PostgreSQL is practically a wonderland of inter-company
collaboration.  Yeah, there's some "does not play nice with others"
which happens, but that's pretty much inevitable.

Plus, it's also useful to have some companies go in different directions
sometimes; the best approach to certain problems isn't always well
defined.  We might have a little more of that than is completely ideal,
but it's rather hard to determine that.

Anyway, all of this is a moot point, because nobody has the power to
tell the various companies what to do.  We're just lucky that everyone
is still committed to writing stuff which adds to PostgreSQL.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Josh berkus  writes:
> On 05/13/2016 08:19 AM, Bruce Momjian wrote:
>> Someone mentioned how Postgres 8.5 became 9.0, but then someone else
>> said the change was made during alpha releases, not beta.  Can someone
>> dig up the details?

> /me digs through the announcement archives ...
> We changed it to 9.0 for Alpha4.  By Beta1, it was already 9.0.

Yeah, I'd come to the same conclusion by checking the git history.
7.0 and 8.0 were also rebranded that way in advance of any beta
releases.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 08:19 AM, Bruce Momjian wrote:
>> > Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
>> > actually be involved in making the change?
> Someone mentioned how Postgres 8.5 became 9.0, but then someone else
> said the change was made during alpha releases, not beta.  Can someone
> dig up the details?

/me digs through the announcement archives ...

We changed it to 9.0 for Alpha4.  By Beta1, it was already 9.0.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Kevin Grittner
On Fri, May 13, 2016 at 1:46 PM, Robert Haas  wrote:
> On Fri, May 13, 2016 at 2:45 PM, Vik Fearing  wrote:
>> On 05/13/2016 08:31 PM, Alvaro Herrera wrote:

>>> I think the next version should be 10.0 no matter what changes we put
>>> in.
>>
>> +1
>>
>> Let's even stamp it 10.0devel from the get-go.
>
> If making this one 10.0 is out, then I'm for that proposal.

+1

-- 
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Alvaro Herrera  writes:
> Josh berkus wrote:
>> Anyway, can we come up with a consensus of some minimum changes it will
>> take to make the next version 10.0?

> I think the next version should be 10.0 no matter what changes we put
> in.

Here's my two cents: we called 8.0 that on the basis of the native Windows
port, and 9.0 that on the basis of getting in-core replication support.
Both of those were game-changing features that deserved a "major major"
version number bump.  But as the project matures, it gets harder and
harder to come up with game-changing features in the span of a single
release.  Parallel query is a great example: a fully mature parallel query
feature would, IMO, easily justify a 10.0 moniker.  But what we have today
is more like half or two-thirds of a feature.  If we call this release
10.0 on the strength of that, we could justifiably be accused of
overselling it.  On the other hand, if we wait till next year when
parallelism presumably will be much more fully baked, it'll be a bit
anticlimactic to call it 10.0 then.  This isn't going to get better with
other major features that can be expected to appear in future.  So we can
expect to continue to waste lots of time debating the "what to call it"
question, in pretty much every year except for the one or two immediately
after a "major major" bump.

There's also the problem that the current numbering scheme confuses
people who aren't familiar with the project.  How many times have
you seen people say "I'm using Postgres 8" or "Postgres 9" when asked
what version they're on?

So I think we should solve these problems at a stroke, and save ourselves
lots of breath in the future, by getting rid of the whole "major major"
idea and going over to a two-part version numbering scheme.  To be
specific:

* This year's major release will be 9.6.0, with minor updates 9.6.1,
9.6.2, etc.  It's too late to do otherwise for this release cycle.

* Next year's major release will be 10.0, with minor updates 10.1,
10.2, etc.

* The year after, 11.0.  Etc cetera.

No confusion, no surprises, no debate ever again about what the next
version number is.

This is by no means a new idea, but I think its time has come.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Robert Haas
On Fri, May 13, 2016 at 12:12 PM, Joshua D. Drake  
wrote:
> Singular point contribution is not the point of my argument. My point is
> that if three people from EDB and three people from Citus got together and
> worked on a project in full collaboration it would be more beneficial to the
> project.

Well, the scalability work in 9.6 went almost exactly like this,
assuming you count Andres as three people (which is entirely
reasonable) and Dilip, Mithun, Amit, and myself as three people (which
is maybe less reasonable, since I don't really want any of us counted
as less than a whole person).

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Robert Haas
On Fri, May 13, 2016 at 2:45 PM, Vik Fearing  wrote:
> On 05/13/2016 08:31 PM, Alvaro Herrera wrote:
>> Josh berkus wrote:
>>
>>> Anyway, can we come up with a consensus of some minimum changes it will
>>> take to make the next version 10.0?
>>
>> I think the next version should be 10.0 no matter what changes we put
>> in.
>
> +1
>
> Let's even stamp it 10.0devel from the get-go.

If making this one 10.0 is out, then I'm for that proposal.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Vik Fearing
On 05/13/2016 08:31 PM, Alvaro Herrera wrote:
> Josh berkus wrote:
>  
>> Anyway, can we come up with a consensus of some minimum changes it will
>> take to make the next version 10.0?
> 
> I think the next version should be 10.0 no matter what changes we put
> in.

+1

Let's even stamp it 10.0devel from the get-go.
-- 
Vik Fearing  +33 6 46 75 15 36
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Alvaro Herrera
Josh berkus wrote:

> Well, if we adopt 2-part version numbers, it will be.  Maybe that's the
> easiest thing?  Then we never have to have this discussion again, which
> certainly appeals to me ...

-1

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Perf Benchmarking and regression.

2016-05-13 Thread Robert Haas
On Fri, May 13, 2016 at 1:43 PM, Andres Freund  wrote:
> On 2016-05-13 10:20:04 -0400, Robert Haas wrote:
>> On Fri, May 13, 2016 at 7:08 AM, Ashutosh Sharma  
>> wrote:
>> > Following are the performance results for read write test observed with
>> > different numbers of "backend_flush_after".
>> >
>> > 1) backend_flush_after = 256kb (32*8kb), tps = 10841.178815
>> > 2) backend_flush_after = 512kb (64*8kb), tps = 11098.702707
>> > 3) backend_flush_after = 1MB (128*8kb), tps = 11434.964545
>> > 4) backend_flush_after = 2MB (256*8kb), tps = 13477.089417
>>
>> So even at 2MB we don't come close to recovering all of the lost
>> performance.  Can you please test these three scenarios?
>>
>> 1. Default settings for *_flush_after
>> 2. backend_flush_after=0, rest defaults
>> 3. backend_flush_after=0, bgwriter_flush_after=0,
>> wal_writer_flush_after=0, checkpoint_flush_after=0
>
> 4) 1) + a shared_buffers setting appropriate to the workload.
>
>
> I just want to emphasize what we're discussing here is a bit of an
> extreme setup. A workload that's bigger than shared buffers, but smaller
> than the OS's cache size; with a noticeable likelihood of rewriting
> individual OS page cache pages within 30s.

You're just describing pgbench with a scale factor too large to fit in
shared_buffers.  I think it's unfair to paint that as some kind of
niche use case.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] delta relations in AFTER triggers

2016-05-13 Thread Kevin Grittner
On Fri, May 13, 2016 at 1:02 PM, David Fetter  wrote:
> On Thu, Jan 22, 2015 at 02:41:42PM +, Kevin Grittner wrote:

>> [ideas on how to pass around references to ephemeral relations]
>
> [almost 17 months later]
>
> It seems like now is getting close to the time to get this into
> master.  The patch might have suffered from some bit rot, but the
> design so far seems sound.
>
> What say?

I had a talk with Tom in Brussels about this.  As I understood it,
he wasn't too keen on the suggestion by Heikki (vaguely
sorta-endorsed by Robert) of passing ephemeral named relations such
as these tuplestores around in the structures currently used for
parameter values.  He intuitively foresaw the types of problems I
had run into trying to use the same structure to pass a relation
(with structure and rows and columns of data) as is used to pass,
say, an integer.

After discussing a while, he suggested that this patch should be
looked at as an opportunity to refactor the existing handling of
the data used by AFTER triggers.  He pointed out that the existing
technique is unbounded in RAM use, with no ability to spill to
temporary files, and regularly generates complaints.  This patch is
putting all that same data into a pair of tuplestores (for old and
new row versions) that would spill to disk as needed and would
likely perform better.

I think that still leaves the question open of how best to pass
around information about ephemeral relations.  It seems to me that
threads about other features  have brushed up against similar
needs, and we should consider those in conjunction with this to
make sure we solve the problem once in a sufficiently general way
to cover all the needs.  (For example, the "asynchronous and
vectorized execution" brushes against related concepts, and I seem
to recall others.)

Unfortunately for those eager to have incremental maintenance of
materialized views (for which this patch was a milestone on the
way), Tom's suggestions seem to put additional work on that path.
When I pointed that out, he pointed out that doing features well
can take a lot of time.  I think that was about the time the next
round of drinks arrived and we decided we'd better not try to take
the discussion much further that night.  ;-)

I welcome any other thoughts on this, particularly from Tom (and
*especially* if I've misrepresented his position in any way!).
Perhaps we can get a design we all like and split the work so that
the refactoring Tom wants and the feature this patch is intended to
implement can get done in the next CF or two, and some initial
work on IMMV can still make it into the next development cycle,
building on this.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David Fetter
On Fri, May 13, 2016 at 03:31:37PM -0300, Alvaro Herrera wrote:
> Josh berkus wrote:
>  
> > Anyway, can we come up with a consensus of some minimum changes it will
> > take to make the next version 10.0?
> 
> I think the next version should be 10.0 no matter what changes we put
> in.

+1

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 11:31 AM, Alvaro Herrera wrote:
> Josh berkus wrote:
>  
>> Anyway, can we come up with a consensus of some minimum changes it will
>> take to make the next version 10.0?
> 
> I think the next version should be 10.0 no matter what changes we put
> in.
> 

Well, if we adopt 2-part version numbers, it will be.  Maybe that's the
easiest thing?  Then we never have to have this discussion again, which
certainly appeals to me ...

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Petr Jelinek

On 13/05/16 20:31, Alvaro Herrera wrote:

Josh berkus wrote:


Anyway, can we come up with a consensus of some minimum changes it will
take to make the next version 10.0?


I think the next version should be 10.0 no matter what changes we put
in.



+1

--
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Alvaro Herrera
Josh berkus wrote:
 
> Anyway, can we come up with a consensus of some minimum changes it will
> take to make the next version 10.0?

I think the next version should be 10.0 no matter what changes we put
in.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Josh berkus
On 05/13/2016 09:30 AM, Tom Lane wrote:

> More generally, rebranding after beta1 sends a very public signal that
> we're a bunch of losers who couldn't make up our minds in a timely
> fashion.  We should have discussed this last month; now I think we're
> stuck with a decision by default.

Although maybe we could use some controversy to get us back in the press ;-)

Anyway, can we come up with a consensus of some minimum changes it will
take to make the next version 10.0?  Here's my thinking:

1. pglogical is accepted into core, with docs/scripts to make it a hot
upgrade option.

2. parallel continues to make progress

My argument is that even if we get nothing else, the above two are
enough to "bump" it to 10.0.  And if we can have argreement on that now,
then we can avoid a month-long argument about version numbers next year.


-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] delta relations in AFTER triggers

2016-05-13 Thread David Fetter
On Thu, Jan 22, 2015 at 02:41:42PM +, Kevin Grittner wrote:
> Robert Haas  wrote:
> 
> > Another idea is to change what actually gets passed to the parser
> > callback.  Right now we just pass the PLpgSQL_expr.  If we created a
> > new structure that contained that plus the PLpgSQL_execstate, we'd be
> > in fine shape.  But this sort of gets at the root of the problem here:
> > with variables, the parser callback doesn't return the actual *value*,
> > it returns a Param node that will later, at execution time, be looked
> > up to find the actual value.  With relations, we're sort of doing the
> > same thing - the tuplestore RTE doesn't need to contain the actual
> > data, just the tuple descriptor.  But the data structures from which
> > we can get that information also contain the actual execution-time
> > information, so passing them into parse-time callbacks seems like it
> > might be, if nothing else, a recipe for future bugs.
> 
> That was, of course, why this patch evolved to using this structure
> during parsing:
> 
> typedef struct TsrmdData
> {
> char   *name;  /* name used to identify the 
> tuplestore */
> TupleDesc   tupdesc;/* description of result rows */
> } TsrmdData;
> 
> typedef TsrmdData *Tsrmd;
> 
> ... and this during execution:
> 
> typedef struct TsrData
> {
> TsrmdData  md;
> Tuplestorestate*tstate; /* data (or tids) */
> } TsrData;
> 
> typedef TsrData *Tsr;
> 
> The big problem, as I see it, is how to deliver these to where they
> are needed.  I didn't think it was that hard to do with the parser
> hook; it's what to do to get the execution time structure to where
> it's needed that I can't figure out.  Passing it with the
> parameters is tricky because we're often passing a NULL for the
> reference to the parameter list when we need these.  Trying to coax
> the executor to pass in a parameter list when there are no
> parameters, just these ephemeral relations, seems very tricky and
> all solutions I have tried (other than the one Heikki and others
> have objected to) very fragile.
> 
> In short, the only solution which I've been able to come up with
> that works (and seems to me solid enough to commit) is the one that
> Hekki, Tom, and Robert seem to think should be made more like
> parameter handling; and every attempt at handling these relations
> like parameters seems to me too fragile for me to feel it is worthy
> of commit.
> 
> We're really down to the wire on getting this feature into 9.5; and
> we're way past what was initially my goal, which was to build on
> this to get some incremental maintenance of (some types of simple)
> materialized views into 9.5.  IMO we need to be able to build up
> tuplestores and easily reference them from within complex queries
> to create any sane and efficient implementation of incremental
> maintenance of materialized views.  This patch was mainly intended
> to make progress on the MV front, with an incidental benefit of
> providing a standard feature that I have seen requested a few times.

[almost 17 months later]

It seems like now is getting close to the time to get this into
master.  The patch might have suffered from some bit rot, but the
design so far seems sound.

What say?

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Vitaly Burovoy
On 5/13/16, Bruce Momjian  wrote:
> On Fri, May 13, 2016 at 11:05:23AM -0400, Robert Haas wrote:
>> The major arguments advanced in favor of 10.0 are:
>>
>> - There are a lot of exciting features in this release.

If I'm mot mistaken each release introduced exciting features.
In my opinion:
9.5 - kind of UPSERT & RLS;
9.4 - reduced locks for ALTER TABLE, concurrent matview updates and JSONB(!!!);
9.3 - autoupdatable views & LATERAL joins & event trigger;
9.2 - RANGE data types & index-only scans & planner improvements;
9.1 - FDW & extensions & true SERIALIZABLE isolation level & data
modification in CTE.

Each version is a candidate to be 10.0 by the same reasons as people
try to show now.
Since each version has quite big changes and you can't predict how
great changes will be in the next release why don't just increase the
second digit up to 9? As for me version 9.9 is beautiful enough (and
version 9.9.9 is even more beautiful). =)

>> - Even if you aren't super-excited by the features in this release,
>> PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
>
> I think you meant "a world away from 9.0".
>
> Actually, I don't see the distance from 9.0 as a valid argument as 9.5
> was probably also a world away from 9.0.
>
> I prefer calling 9.7 as 10.0 because there will be near-zero-downtime
> major upgrades with pg_logical (?),

I dream there is time of zero-downtime when admins are instantiate new
version of PostgreSQL, make it slave, wait until it finishes
synchronization (prepare to multimaster), replace both of them as
multimaster online (without downtime), then reconnect clients to the
new instance, then leave new instance as master and shutdown old
version.

If it appears in the release after next one (which is expected be 10.1
or 10.0 or 9.8, i.e. after the next year), whether we update major
number again (11.0) or leave numeration as is (it will be
10.2/10.1/9.9) or it is the real reason to bump 9.8->10.0 (but in this
particular case I'd leave 9.9)?

> and parallelism will cover more cases.
> Built-in logical replication in 9.7 would be big too, and
> another reason to do 9.7 as 10.0.
>
> On the other hand, the _start_ of parallelism in 9.6 could be enough of
> a reason to call it 10.0, with the idea that the 10-series is
> increasingly parallel-aware.  You could argue that parallelism is a much
> bigger deal than near-zero-downtime upgrades.

I think each developer/DBA search for different benefit from each release.
E.g. parallelism is not so important for OLTP developers and
near-zero-downtime is mostly for system administrators/DBA.

> I think the fundamental issue is whether we want to lead the 10.0 branch
> with parallelism, or wait for an administrative change like
> near-zero-downtime major upgrades and built-in logical replication.
>
>   Bruce Momjian  http://momjian.us
>   EnterpriseDB http://enterprisedb.com

I'd vote for 9.6 up to 9.9 then 10.0…

-- 
Best regards,
Vitaly Burovoy


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Perf Benchmarking and regression.

2016-05-13 Thread Andres Freund
On 2016-05-13 10:20:04 -0400, Robert Haas wrote:
> On Fri, May 13, 2016 at 7:08 AM, Ashutosh Sharma  
> wrote:
> > Following are the performance results for read write test observed with
> > different numbers of "backend_flush_after".
> >
> > 1) backend_flush_after = 256kb (32*8kb), tps = 10841.178815
> > 2) backend_flush_after = 512kb (64*8kb), tps = 11098.702707
> > 3) backend_flush_after = 1MB (128*8kb), tps = 11434.964545
> > 4) backend_flush_after = 2MB (256*8kb), tps = 13477.089417
> 
> So even at 2MB we don't come close to recovering all of the lost
> performance.  Can you please test these three scenarios?
>
> 1. Default settings for *_flush_after
> 2. backend_flush_after=0, rest defaults
> 3. backend_flush_after=0, bgwriter_flush_after=0,
> wal_writer_flush_after=0, checkpoint_flush_after=0

4) 1) + a shared_buffers setting appropriate to the workload.


I just want to emphasize what we're discussing here is a bit of an
extreme setup. A workload that's bigger than shared buffers, but smaller
than the OS's cache size; with a noticeable likelihood of rewriting
individual OS page cache pages within 30s.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Petr Jelinek

On 13/05/16 17:42, Andreas Joseph Krogh wrote:

På fredag 13. mai 2016 kl. 17:05:23, skrev Robert Haas
>:

 From a non-hacker...
 From a DBA/application-developer perspective  while there are many exiting
features in 9.6 I'd expect more from 10.0, like some of these features:
- Built in "Drop-in replacement" Multi-master replication
- Built-in per-database replication with sequences and DDL-changes
   (future versions of pglogical might solve this)
- Full (and effective) parallelism "everywhere"
- Improved executor (like Robert Haas suggested), more use of LLVM or
similar
- All of Postgres Pro's GIN-improvements for really fast FTS (with
proper, index-backed, sorting etc.)
- Pluggable storage-engines


I think this is actually nice example why we'll have to go the 
accumulation route when deciding to bump to 10, 11, etc. We can't 
possibly get all the major features people want into single release 
given the yearly release cycle.


That being said, I think that once we released beta1 as 9.6 it's too 
late for bumping it.


--
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 09:40 AM, Bruce Momjian wrote:

On Fri, May 13, 2016 at 09:35:40AM -0700, Joshua Drake wrote:

On 05/13/2016 09:28 AM, Bruce Momjian wrote:

On Fri, May 13, 2016 at 09:12:23AM -0700, Joshua Drake wrote:

There was no disrespect intended. I was trying to push forth an idea that
multi-company team collaboration is better for the community than single
company team collaboration. I will stand by that assertion.


Uh, we are already doing that.  EDB and NTT are working on FDWs and
sharding, PostgresPro and someone else is working on a transaction
manager, and EDB and 2nd Quadrant worked on parallelism.

What is the problem you are trying to solve?


Hey, if I am wrong that's awesome. The impression I have is the general
workflow is this:

* Company(1) discusses feature with community
* Company(1) works on patch/feature for a period of time
* Company(1) delivers patch to community
* Standard operation continues (patch review, discussion, etc..)


Yes, there are some cases of that.  I assume it is due to efficiency and
the belief that others aren't interested in helping.  In a way is a
company working on something alone different from a person working on a
patch alone?


No but I also think we should discourage that when reasonable as well. 
Obviously some patches just don't need more than one person but when we 
are talking about anything that is taking X time (month or more?) then

we should actively encourage collaboration.

That is all I am really talking about here. A more assertive 
collaboration for the betterment of the community. When I think about 
the size of the brain trust we have as a whole, I imagine the great 
things we could do even better. It isn't magical or overnight but a long 
term goal.


Sincerely,

JD

--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 09:35:40AM -0700, Joshua Drake wrote:
> On 05/13/2016 09:28 AM, Bruce Momjian wrote:
> >On Fri, May 13, 2016 at 09:12:23AM -0700, Joshua Drake wrote:
> >>There was no disrespect intended. I was trying to push forth an idea that
> >>multi-company team collaboration is better for the community than single
> >>company team collaboration. I will stand by that assertion.
> >
> >Uh, we are already doing that.  EDB and NTT are working on FDWs and
> >sharding, PostgresPro and someone else is working on a transaction
> >manager, and EDB and 2nd Quadrant worked on parallelism.
> >
> >What is the problem you are trying to solve?
> 
> Hey, if I am wrong that's awesome. The impression I have is the general
> workflow is this:
> 
>   * Company(1) discusses feature with community
>   * Company(1) works on patch/feature for a period of time
>   * Company(1) delivers patch to community
>   * Standard operation continues (patch review, discussion, etc..)

Yes, there are some cases of that.  I assume it is due to efficiency and
the belief that others aren't interested in helping.  In a way is a
company working on something alone different from a person working on a
patch alone?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David Fetter
On Fri, May 13, 2016 at 12:30:47PM -0400, Tom Lane wrote:
> 
> I think you could, though, make an argument that breaking such code
> after beta1 is a bit unfair.  People expect to be able to do
> compatibility testing with a new major version starting with beta1.

One could, but I wouldn't find it terribly persuasive.  As Thom
pointed out, we have actually done this before.

> More generally, rebranding after beta1 sends a very public signal
> that we're a bunch of losers who couldn't make up our minds in a
> timely fashion.  We should have discussed this last month; now I
> think we're stuck with a decision by default.

This, on the other hand, is more persuasive to me.  We now have a much
more public face than we did then.

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 09:28 AM, Bruce Momjian wrote:

On Fri, May 13, 2016 at 09:12:23AM -0700, Joshua Drake wrote:

There was no disrespect intended. I was trying to push forth an idea that
multi-company team collaboration is better for the community than single
company team collaboration. I will stand by that assertion.


Uh, we are already doing that.  EDB and NTT are working on FDWs and
sharding, PostgresPro and someone else is working on a transaction
manager, and EDB and 2nd Quadrant worked on parallelism.

What is the problem you are trying to solve?


Hey, if I am wrong that's awesome. The impression I have is the general 
workflow is this:


* Company(1) discusses feature with community
* Company(1) works on patch/feature for a period of time
* Company(1) delivers patch to community
* Standard operation continues (patch review, discussion, etc..)

This is not "bad" but it isn't as productive as something like this 
would be:


	* Company(1) + Company(2) get together and discuss using their 
respective resources to collaboratively develop X (multi-master for 
example).


* Company(1) + Company(2) discuss feature with community
* Company(1) + Company(2) work on patch/feature in the open, together
* Company(1) + Company(2) deliver patch to community
* Standard operation continues (patch review, discussion, etc...)

The difference being one of coopetition versions competition for the 
betterment of the community. If there are companies that are doing that 
already, that is awesome and I applaud it. I was just trying to further 
drive that idea home.


This on my end all sourced from this event:

https://www.commandprompt.com/blogs/joshua_drake/2016/04/you_are_my_fellow_community_member/

That event is childish and a poor representation of what our community 
stands for: Excellent, Correctness, Inclusion, Collaboration.


Sincerely,

JD



--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Dave Page


> On 13 May 2016, at 17:24, Magnus Hagander  wrote:
> 
>> On Fri, May 13, 2016 at 5:29 PM, Dave Page  wrote:
>> On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:
>> >
>> > Well, one potential issues is that there may be projects which have
>> > already coded in 9.6 checks for feature support.
>> 
>> I suspect that won't be an issue (I never heard of it being for 7.5,
>> which was released as 8.0 - but is smattered all over pgAdmin 3 for
>> example) - largely because in such apps we're almost always checking
>> for a version greater than or less than x.y.
>> 
>> I imagine the bigger issue will be apps that have been written
>> assuming the first part of the version number is only a single digit.
> 
> If they are, they are already broken by design. But more to the point, unless 
> you're arguing for *never* changing to 10.0, that's not really something that 
> should decide when we do it, because they will break.
> 
> We have provided multiple ways to check this. For example, we've had 
> PQserverVersion() since forever which returns an integer that you can just 
> compare. We have never claimed that it would be single digit in any of the 
> fields (first, second *or* third). I honestly don't care at all if those 
> applications break.
> 
> (We would, however, have a problem to go above 100 in all fields *except* the 
> first one, since the integer uses a two-digit representation for each)

Oh, I don't care about such code. Just opining that it's a likely issue 
somewhere - and if so, it should be up to users to fix their apps.

Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
David Fetter  writes:
> On Fri, May 13, 2016 at 04:34:34PM +0100, Thom Brown wrote:
>> On 13 May 2016 at 16:29, Dave Page  wrote:
>>> I imagine the bigger issue will be apps that have been written
>>> assuming the first part of the version number is only a single
>>> digit.

>> Is that likely?  That would be remarkably myopic, but I guess
>> possible.

> You might be astonished at the ubiquity of myopia out in the world.
> That's not an argument against 10.0, by the way.

Yeah, I do not think this is a very relevant argument.  We certainly
will go to 10.0 at some point; if we tried not to, come 2020 we'd be
shipping 9.10.x which would be just as likely to break badly-written
version parsing code.  So such code will have to be fixed eventually,
and whether we break it this year or next year seems like not our
problem.

I think you could, though, make an argument that breaking such code after
beta1 is a bit unfair.  People expect to be able to do compatibility
testing with a new major version starting with beta1.

More generally, rebranding after beta1 sends a very public signal that
we're a bunch of losers who couldn't make up our minds in a timely
fashion.  We should have discussed this last month; now I think we're
stuck with a decision by default.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Andreas Joseph Krogh
På fredag 13. mai 2016 kl. 18:22:00, skrev Magnus Hagander >:
On Fri, May 13, 2016 at 5:42 PM, Andreas Joseph Krogh > wrote: På fredag 13. mai 2016 kl. 17:05:23, skrev 
Robert Haas >:
Hi,

 There is a long-running thread on pgsql-hackers on whether 9.6 should
 instead be called 10.0.  Initially, opinions were mixed, but consensus
 seems now to have emerged that 10.0 is a good choice, with the major
 hesitation being that we've already released 9.6beta1, and therefore
 we might not want to change at this point.  That doesn't seem like an
 insuperable barrier to me, but I think it's now time for the
 discussion on this topic to move here, because:

 1. Some people who have strong opinions may not have followed the
 discussion on pgsql-advocacy, and

 2. If we're going to rebrand this as 10.0, the work will have to get done 
here.

 The major arguments advanced in favor of 10.0 are:

 - There are a lot of exciting features in this release.

 - Even if you aren't super-excited by the features in this release,
 PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
 sense to bump the version based on the amount of accumulated change
 between then and now.

 Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
 actually be involved in making the change?
 
>From a non-hacker...
 
>From a DBA/application-developer perspective  while there are many exiting
features in 9.6 I'd expect more from 10.0, like some of these features:
- Built in "Drop-in replacement" Multi-master replication
- Built-in per-database replication with sequences and DDL-changes
  (future versions of pglogical might solve this)
- Full (and effective) parallelism "everywhere"
- Improved executor (like Robert Haas suggested), more use of LLVM or similar
- All of Postgres Pro's GIN-improvements for really fast FTS (with proper, 
index-backed, sorting etc.)
- Pluggable storage-engines
 
 
I'm willing to declare that the likelihood you getting all of these in one 
release is zero. And there will always be "one more feature left".



 
I don't think anyone expects all of them for a 10.0 release:-) I just listed 
some stuff which would, IMHO, validate a 10.0 release, some combined with 
others, others alone (like MMR).
 
-- Andreas Joseph Krogh




Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 09:12:23AM -0700, Joshua Drake wrote:
> There was no disrespect intended. I was trying to push forth an idea that
> multi-company team collaboration is better for the community than single
> company team collaboration. I will stand by that assertion.

Uh, we are already doing that.  EDB and NTT are working on FDWs and
sharding, PostgresPro and someone else is working on a transaction
manager, and EDB and 2nd Quadrant worked on parallelism.

What is the problem you are trying to solve?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Magnus Hagander
On Fri, May 13, 2016 at 5:29 PM, Dave Page  wrote:

> On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:
> >
> > Well, one potential issues is that there may be projects which have
> > already coded in 9.6 checks for feature support.
>
> I suspect that won't be an issue (I never heard of it being for 7.5,
> which was released as 8.0 - but is smattered all over pgAdmin 3 for
> example) - largely because in such apps we're almost always checking
> for a version greater than or less than x.y.
>
> I imagine the bigger issue will be apps that have been written
> assuming the first part of the version number is only a single digit.
>

If they are, they are already broken by design. But more to the point,
unless you're arguing for *never* changing to 10.0, that's not really
something that should decide when we do it, because they will break.

We have provided multiple ways to check this. For example, we've had
PQserverVersion() since forever which returns an integer that you can just
compare. We have never claimed that it would be single digit in any of the
fields (first, second *or* third). I honestly don't care at all if those
applications break.

(We would, however, have a problem to go above 100 in all fields *except*
the first one, since the integer uses a two-digit representation for each)


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: [HACKERS] 10.0

2016-05-13 Thread Magnus Hagander
On Fri, May 13, 2016 at 5:42 PM, Andreas Joseph Krogh 
wrote:

> På fredag 13. mai 2016 kl. 17:05:23, skrev Robert Haas <
> robertmh...@gmail.com>:
>
> Hi,
>
> There is a long-running thread on pgsql-hackers on whether 9.6 should
> instead be called 10.0.  Initially, opinions were mixed, but consensus
> seems now to have emerged that 10.0 is a good choice, with the major
> hesitation being that we've already released 9.6beta1, and therefore
> we might not want to change at this point.  That doesn't seem like an
> insuperable barrier to me, but I think it's now time for the
> discussion on this topic to move here, because:
>
> 1. Some people who have strong opinions may not have followed the
> discussion on pgsql-advocacy, and
>
> 2. If we're going to rebrand this as 10.0, the work will have to get done
> here.
>
> The major arguments advanced in favor of 10.0 are:
>
> - There are a lot of exciting features in this release.
>
> - Even if you aren't super-excited by the features in this release,
> PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
> sense to bump the version based on the amount of accumulated change
> between then and now.
>
> Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
> actually be involved in making the change?
>
>
> From a non-hacker...
>
> From a DBA/application-developer perspective  while there are many exiting
> features in 9.6 I'd expect more from 10.0, like some of these features:
> - Built in "Drop-in replacement" Multi-master replication
> - Built-in per-database replication with sequences and DDL-changes
>   (future versions of pglogical might solve this)
> - Full (and effective) parallelism "everywhere"
> - Improved executor (like Robert Haas suggested), more use of LLVM or
> similar
> - All of Postgres Pro's GIN-improvements for really fast FTS (with proper,
> index-backed, sorting etc.)
> - Pluggable storage-engines
>
>
I'm willing to declare that the likelihood you getting all of these in one
release is zero. And there will always be "one more feature left".

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: [HACKERS] 10.0

2016-05-13 Thread Stephen Frost
On Friday, May 13, 2016, Bruce Momjian  wrote:

> On Fri, May 13, 2016 at 12:05:34PM -0400, Stephen Frost wrote:
> > * Dave Page (dp...@pgadmin.org ) wrote:
> > > I imagine the bigger issue will be apps that have been written
> > > assuming the first part of the version number is only a single digit.
> >
> > Let's just go with 2016 instead then.
> >
> > At least then users would see how old the version they're running is (I
> > was just recently dealing with a 8.4 user...).
>
> We tried, that, "Postgres95".  ;-)
>

Even better, we're being retro!  It's in style! ;)

Stephen


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Stephen Frost  writes:
> On Friday, May 13, 2016, Bruce Momjian  wrote:
>> On Fri, May 13, 2016 at 12:05:34PM -0400, Stephen Frost wrote:
>>> Let's just go with 2016 instead then.

>> We tried, that, "Postgres95".  ;-)

> Even better, we're being retro!  It's in style! ;)

It would sure put a premium on not slipping releases past December.
If you don't rebrand, you look silly; if you do, you've lost the
ability to put out a new major release in the next year.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread David Fetter
On Fri, May 13, 2016 at 04:34:34PM +0100, Thom Brown wrote:
> On 13 May 2016 at 16:29, Dave Page  wrote:
> > On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:
> >>
> >> Well, one potential issues is that there may be projects which
> >> have already coded in 9.6 checks for feature support.
> >
> > I suspect that won't be an issue (I never heard of it being for
> > 7.5, which was released as 8.0 - but is smattered all over pgAdmin
> > 3 for example) - largely because in such apps we're almost always
> > checking for a version greater than or less than x.y.
> >
> > I imagine the bigger issue will be apps that have been written
> > assuming the first part of the version number is only a single
> > digit.
> 
> Is that likely?  That would be remarkably myopic, but I guess
> possible.

You might be astonished at the ubiquity of myopia out in the world.

That's not an argument against 10.0, by the way.  Myopia of that type
tends to come with software quality so low that your best bet is never
to start using it, and your second-best is to eliminate it from your
system at high priority.

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-05-13 Thread Joshua D. Drake

On 05/13/2016 07:42 AM, Robert Haas wrote:

On Sat, Apr 30, 2016 at 8:46 PM, Joshua Drake  wrote:

Oh, absolutely. I was just pointing out how a lot of companies are hoarding
talent internally for no productive purpose.


Wow, really?

I disagree both with the idea that this is happening and with your
characterization of it.  First, there are lots of people contributing
code to PostgreSQL right now.  To look at the just the last
CommitFest, we've got multiple people from all of Crunchy Data,
2ndQuadrant, EnterpriseDB,  Postgres Pro, and NTT; plus Julien Rouhaud
from Dalibo and Peter Geoghegan at Heroku and Michael Paquier at
VMware, among many others.


Singular point contribution is not the point of my argument. My point is 
that if three people from EDB and three people from Citus got together 
and worked on a project in full collaboration it would be more 
beneficial to the project.



I'm not sure anyone at CommandPrompt
submitted a patch, though.  :-)


This is true. We have been a little busy doing a lot of things for the 
community that -hackers are generally not good at.




Second, when people don't contribute as much as you think they should
to the PostgreSQL community, I don't think that necessarily means
their employer is doing something wrong.  Sometimes, it may be the


This is also true.


employee's choice to spend more time on consulting or support or
whatever else they are doing; maybe that's what they like to do.  At
other times, it may be the thing that has to be done to pay the bills,
and I think that's legitimate, too.  People have a right to earning a
living, and companies have to make money to keep paying their
employees.


As someone in this community that is responsible for paying the bills 
including employee salaries, I fully agree with this.




Let's respect people and companies for what they do contribute, rather
than labeling it as not good enough.



There was no disrespect intended. I was trying to push forth an idea 
that multi-company team collaboration is better for the community than 
single company team collaboration. I will stand by that assertion.


Sincerely,

JD


--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Stephen Frost
On Friday, May 13, 2016, Dave Page  wrote:

> On Fri, May 13, 2016 at 5:08 PM, Bruce Momjian  > wrote:
> > On Fri, May 13, 2016 at 12:05:34PM -0400, Stephen Frost wrote:
> >> * Dave Page (dp...@pgadmin.org ) wrote:
> >> > I imagine the bigger issue will be apps that have been written
> >> > assuming the first part of the version number is only a single digit.
> >>
> >> Let's just go with 2016 instead then.
> >>
> >> At least then users would see how old the version they're running is (I
> >> was just recently dealing with a 8.4 user...).
> >
> > We tried, that, "Postgres95".  ;-)
>
> Awesome: Postgres16 > Postgres95.
>
> That won't be confusing now will it? :-)
>

We'll just say you have to be using a special collation with 9.5.0 to get
the right sort order.. ;)

/me hides from Peter

Thanks!

Stephen


Re: [HACKERS] 10.0

2016-05-13 Thread Dave Page
On Fri, May 13, 2016 at 5:08 PM, Bruce Momjian  wrote:
> On Fri, May 13, 2016 at 12:05:34PM -0400, Stephen Frost wrote:
>> * Dave Page (dp...@pgadmin.org) wrote:
>> > I imagine the bigger issue will be apps that have been written
>> > assuming the first part of the version number is only a single digit.
>>
>> Let's just go with 2016 instead then.
>>
>> At least then users would see how old the version they're running is (I
>> was just recently dealing with a 8.4 user...).
>
> We tried, that, "Postgres95".  ;-)

Awesome: Postgres16 > Postgres95.

That won't be confusing now will it? :-)

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 12:05:34PM -0400, Stephen Frost wrote:
> * Dave Page (dp...@pgadmin.org) wrote:
> > I imagine the bigger issue will be apps that have been written
> > assuming the first part of the version number is only a single digit.
> 
> Let's just go with 2016 instead then.
> 
> At least then users would see how old the version they're running is (I
> was just recently dealing with a 8.4 user...).

We tried, that, "Postgres95".  ;-)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Stephen Frost
* Dave Page (dp...@pgadmin.org) wrote:
> I imagine the bigger issue will be apps that have been written
> assuming the first part of the version number is only a single digit.

Let's just go with 2016 instead then.

At least then users would see how old the version they're running is (I
was just recently dealing with a 8.4 user...).

Thanks!

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] 10.0

2016-05-13 Thread Andreas Joseph Krogh
På fredag 13. mai 2016 kl. 17:05:23, skrev Robert Haas >:
Hi,

 There is a long-running thread on pgsql-hackers on whether 9.6 should
 instead be called 10.0.  Initially, opinions were mixed, but consensus
 seems now to have emerged that 10.0 is a good choice, with the major
 hesitation being that we've already released 9.6beta1, and therefore
 we might not want to change at this point.  That doesn't seem like an
 insuperable barrier to me, but I think it's now time for the
 discussion on this topic to move here, because:

 1. Some people who have strong opinions may not have followed the
 discussion on pgsql-advocacy, and

 2. If we're going to rebrand this as 10.0, the work will have to get done 
here.

 The major arguments advanced in favor of 10.0 are:

 - There are a lot of exciting features in this release.

 - Even if you aren't super-excited by the features in this release,
 PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
 sense to bump the version based on the amount of accumulated change
 between then and now.

 Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
 actually be involved in making the change?
 
>From a non-hacker...
 
>From a DBA/application-developer perspective  while there are many exiting
features in 9.6 I'd expect more from 10.0, like some of these features:
- Built in "Drop-in replacement" Multi-master replication
- Built-in per-database replication with sequences and DDL-changes
  (future versions of pglogical might solve this)
- Full (and effective) parallelism "everywhere"
- Improved executor (like Robert Haas suggested), more use of LLVM or similar
- All of Postgres Pro's GIN-improvements for really fast FTS (with proper, 
index-backed, sorting etc.)
- Pluggable storage-engines
 
Thanks.
 
-- Andreas Joseph Krogh


 


Re: [HACKERS] 10.0

2016-05-13 Thread Larry Rosenman

On 2016-05-13 10:34, Thom Brown wrote:

On 13 May 2016 at 16:29, Dave Page  wrote:

On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:


Well, one potential issues is that there may be projects which have
already coded in 9.6 checks for feature support.


I suspect that won't be an issue (I never heard of it being for 7.5,
which was released as 8.0 - but is smattered all over pgAdmin 3 for
example) - largely because in such apps we're almost always checking
for a version greater than or less than x.y.

I imagine the bigger issue will be apps that have been written
assuming the first part of the version number is only a single digit.


Is that likely?  That would be remarkably myopic, but I guess possible.

Thom
We (FreeBSD) had lots of that kind of fallout when 9->10.  Autoconf, and 
other tools

thought we were a.out and not ELF.


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@lerctr.org
US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Tom Lane
Robert Haas  writes:
> There is a long-running thread on pgsql-hackers on whether 9.6 should
> instead be called 10.0.

First I've seen it mentioned here.

I think you are just about exactly one week too late to bring this up.
Once we've shipped a beta, rebranding is way too confusing.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Thom Brown
On 13 May 2016 at 16:29, Dave Page  wrote:
> On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:
>>
>> Well, one potential issues is that there may be projects which have
>> already coded in 9.6 checks for feature support.
>
> I suspect that won't be an issue (I never heard of it being for 7.5,
> which was released as 8.0 - but is smattered all over pgAdmin 3 for
> example) - largely because in such apps we're almost always checking
> for a version greater than or less than x.y.
>
> I imagine the bigger issue will be apps that have been written
> assuming the first part of the version number is only a single digit.

Is that likely?  That would be remarkably myopic, but I guess possible.

Thom


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Dave Page
On Fri, May 13, 2016 at 4:23 PM, Thom Brown  wrote:
>
> Well, one potential issues is that there may be projects which have
> already coded in 9.6 checks for feature support.

I suspect that won't be an issue (I never heard of it being for 7.5,
which was released as 8.0 - but is smattered all over pgAdmin 3 for
example) - largely because in such apps we're almost always checking
for a version greater than or less than x.y.

I imagine the bigger issue will be apps that have been written
assuming the first part of the version number is only a single digit.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Thom Brown
On 13 May 2016 at 16:19, Bruce Momjian  wrote:
> On Fri, May 13, 2016 at 11:05:23AM -0400, Robert Haas wrote:
>> Hi,
>>
>> There is a long-running thread on pgsql-hackers on whether 9.6 should
>> instead be called 10.0.  Initially, opinions were mixed, but consensus
>> seems now to have emerged that 10.0 is a good choice, with the major
>> hesitation being that we've already released 9.6beta1, and therefore
>> we might not want to change at this point.  That doesn't seem like an
>> insuperable barrier to me, but I think it's now time for the
>> discussion on this topic to move here, because:
>>
>> 1. Some people who have strong opinions may not have followed the
>> discussion on pgsql-advocacy, and
>>
>> 2. If we're going to rebrand this as 10.0, the work will have to get done 
>> here.
>>
>> The major arguments advanced in favor of 10.0 are:
>>
>> - There are a lot of exciting features in this release.
>>
>> - Even if you aren't super-excited by the features in this release,
>> PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
>
> I think you meant "a world away from 9.0".
>
> Actually, I don't see the distance from 9.0 as a valid argument as 9.5
> was probably also a world away from 9.0.
>
> I prefer calling 9.7 as 10.0 because there will be near-zero-downtime
> major upgrades with pg_logical (?), and parallelism will cover more
> cases.  Built-in logical replication in 9.7 would be big too, and
> another reason to do 9.7 as 10.0.
>
> On the other hand, the _start_ of parallelism in 9.6 could be enough of
> a reason to call it 10.0, with the idea that the 10-series is
> increasingly parallel-aware.  You could argue that parallelism is a much
> bigger deal than near-zero-downtime upgrades.
>
> I think the fundamental issue is whether we want to lead the 10.0 branch
> with parallelism, or wait for an administrative change like
> near-zero-downtime major upgrades and built-in logical replication.
>
>> sense to bump the version based on the amount of accumulated change
>> between then and now.
>>
>> Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
>> actually be involved in making the change?
>
> Someone mentioned how Postgres 8.5 became 9.0, but then someone else
> said the change was made during alpha releases, not beta.  Can someone
> dig up the details?

We had 8.5 alpha 3, then 9.0 alpha 4:

REL8_5_ALPHA1
REL8_5_ALPHA2
REL8_5_ALPHA3
REL9_0_ALPHA4
REL9_0_ALPHA5
REL9_0_BETA1
REL9_0_BETA2
REL9_0_BETA3
REL9_0_BETA4
REL9_0_RC1

Thom


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Thom Brown
On 13 May 2016 at 16:05, Robert Haas  wrote:
>
> Hi,
>
> There is a long-running thread on pgsql-hackers on whether 9.6 should
> instead be called 10.0.  Initially, opinions were mixed, but consensus
> seems now to have emerged that 10.0 is a good choice, with the major
> hesitation being that we've already released 9.6beta1, and therefore
> we might not want to change at this point.  That doesn't seem like an
> insuperable barrier to me, but I think it's now time for the
> discussion on this topic to move here, because:
>
> 1. Some people who have strong opinions may not have followed the
> discussion on pgsql-advocacy, and
>
> 2. If we're going to rebrand this as 10.0, the work will have to get done 
> here.
>
> The major arguments advanced in favor of 10.0 are:
>
> - There are a lot of exciting features in this release.

True dat.

> - Even if you aren't super-excited by the features in this release,
> PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes
> sense to bump the version based on the amount of accumulated change
> between then and now.

Well, a .6 would be the first:

6.5
7.4
8.4

So a .6 would be the very first.  I think the argument of "accumulated
change" is persuasive.

> Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
> actually be involved in making the change?

Well, one potential issues is that there may be projects which have
already coded in 9.6 checks for feature support.  I don't know if
there would be any problems from the repo side of things for
beta-testers.

Thom


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 10.0

2016-05-13 Thread Bruce Momjian
On Fri, May 13, 2016 at 11:05:23AM -0400, Robert Haas wrote:
> Hi,
> 
> There is a long-running thread on pgsql-hackers on whether 9.6 should
> instead be called 10.0.  Initially, opinions were mixed, but consensus
> seems now to have emerged that 10.0 is a good choice, with the major
> hesitation being that we've already released 9.6beta1, and therefore
> we might not want to change at this point.  That doesn't seem like an
> insuperable barrier to me, but I think it's now time for the
> discussion on this topic to move here, because:
> 
> 1. Some people who have strong opinions may not have followed the
> discussion on pgsql-advocacy, and
> 
> 2. If we're going to rebrand this as 10.0, the work will have to get done 
> here.
> 
> The major arguments advanced in favor of 10.0 are:
> 
> - There are a lot of exciting features in this release.
> 
> - Even if you aren't super-excited by the features in this release,
> PostgreSQL 9.6/10.0 is a world away from 10.0, and therefore it makes

I think you meant "a world away from 9.0".

Actually, I don't see the distance from 9.0 as a valid argument as 9.5
was probably also a world away from 9.0.

I prefer calling 9.7 as 10.0 because there will be near-zero-downtime
major upgrades with pg_logical (?), and parallelism will cover more
cases.  Built-in logical replication in 9.7 would be big too, and
another reason to do 9.7 as 10.0.

On the other hand, the _start_ of parallelism in 9.6 could be enough of
a reason to call it 10.0, with the idea that the 10-series is
increasingly parallel-aware.  You could argue that parallelism is a much
bigger deal than near-zero-downtime upgrades.

I think the fundamental issue is whether we want to lead the 10.0 branch
with parallelism, or wait for an administrative change like
near-zero-downtime major upgrades and built-in logical replication.

> sense to bump the version based on the amount of accumulated change
> between then and now.
> 
> Thoughts?  Is it crazy to go from 9.6beta1 to 10.0beta2?  What would
> actually be involved in making the change?

Someone mentioned how Postgres 8.5 became 9.0, but then someone else
said the change was made during alpha releases, not beta.  Can someone
dig up the details?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


  1   2   >