Re: [HACKERS] PSA: don't be in a hurry to update to XCode 9.0
Tom, Did you ever find a solution to this without updating ? Dave Cramer da...@postgresintl.com www.postgresintl.com On 21 September 2017 at 13:01, Dave Cramer <p...@fastcrypt.com> wrote: > Too late I just stumbled over this as well! > > Dave Cramer > > da...@postgresintl.com > www.postgresintl.com > > On 20 September 2017 at 14:34, Tom Lane <t...@sss.pgh.pa.us> wrote: > >> It seems to install some libraries that depend on >> /usr/lib/system/libsystem_darwin.dylib, which doesn't exist. >> Googling suggests it will be there in macOS 10.13, >> but on Sierra or earlier, you're outta luck. >> >> It looks like the only directly-affected PG dependency is >> libxml; if you leave out --with-libxml you can still build. >> >> I found this out the hard way on longfin's host, so I've >> temporarily removed --with-libxml from that animal's >> configuration to restore it to service. I trust I'll be >> able to re-enable that after 10.13 comes out. >> >> regards, tom lane >> >> >> -- >> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-hackers >> > >
Re: [HACKERS] PSA: don't be in a hurry to update to XCode 9.0
Too late I just stumbled over this as well! Dave Cramer da...@postgresintl.com www.postgresintl.com On 20 September 2017 at 14:34, Tom Lane <t...@sss.pgh.pa.us> wrote: > It seems to install some libraries that depend on > /usr/lib/system/libsystem_darwin.dylib, which doesn't exist. > Googling suggests it will be there in macOS 10.13, > but on Sierra or earlier, you're outta luck. > > It looks like the only directly-affected PG dependency is > libxml; if you leave out --with-libxml you can still build. > > I found this out the hard way on longfin's host, so I've > temporarily removed --with-libxml from that animal's > configuration to restore it to service. I trust I'll be > able to re-enable that after 10.13 comes out. > > regards, tom lane > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] SCRAM in the PG 10 release notes
On 14 September 2017 at 02:21, Alvaro Hernandez <a...@ongres.com> wrote: > > > On 14/09/17 08:57, Heikki Linnakangas wrote: > >> On 09/12/2017 04:09 AM, Noah Misch wrote: >> >>> On Wed, May 10, 2017 at 10:50:51PM -0400, Bruce Momjian wrote: >>> >>>> On Mon, May 1, 2017 at 08:12:51AM -0400, Robert Haas wrote: >>>> >>>>> On Tue, Apr 25, 2017 at 10:16 PM, Bruce Momjian <br...@momjian.us> >>>>> wrote: >>>>> >>>>>> Well, we could add "MD5 users are encouraged to switch to >>>>>> SCRAM-SHA-256". Now whether we want to list this as something on the >>>>>> SCRAM-SHA-256 description, or mention it as an incompatibility, or >>>>>> under Migration. I am not clear that MD5 is in such terrible shape >>>>>> that >>>>>> this is warranted. >>>>>> >>>>> >>>>> I think it's warranted. The continuing use of MD5 has been a headache >>>>> for some EnterpriseDB customers who have compliance requirements which >>>>> they must meet. It's not that they themselves necessarily know or >>>>> care whether MD5 is secure, although in some cases they do; it's that >>>>> if they use it, they will be breaking laws or regulations to which >>>>> their business or agency is subject. I imagine customers of other >>>>> PostgreSQL companies have similar issues. But leaving that aside, the >>>>> advantage of SCRAM isn't merely that it uses a better algorithm to >>>>> hash the password. It has other advantages also, like not being >>>>> vulnerable to replay attacks. If you're doing password >>>>> authentication, you should really be using SCRAM, and encouraging >>>>> people to move to SCRAM after upgrading is a good idea. >>>>> >>>>> That having been said, SCRAM is a wire protocol break. You will not >>>>> be able to upgrade to SCRAM unless and until the drivers you use to >>>>> connect to the database add support for it. The only such driver >>>>> that's part of libpq; other drivers that have reimplemented the >>>>> PostgreSQL wire protocol will have to be updated with SCRAM support >>>>> before it will be possible to use SCRAM with those drivers. I think >>>>> this should be mentioned in the release notes, too. I also think it >>>>> would be great if somebody would put together a wiki page listing all >>>>> the popular drivers and (1) whether they use libpq or reimplement the >>>>> wire protocol, and (2) if the latter, the status of any efforts to >>>>> implement SCRAM, and (3) if those efforts have been completed, the >>>>> version from which they support SCRAM. Then, I think we should reach >>>>> out to all of the maintainers of those driver authors who aren't >>>>> moving to support SCRAM and encourage them to do so. >>>>> >>>> >>>> I have added this as an open item because we will have to wait to see >>>> where we are with driver support as the release gets closer. >>>> >>> >>> With the release near, I'm promoting this to the regular open issues >>> section. >>> >> >> Thanks. >> >> I updated the list of drivers on the wiki (https://wiki.postgresql.org/w >> iki/List_of_drivers), adding a column for whether the driver supports >> SCRAM authentication. Currently, the only non-libpq driver that has >> implemented SCRAM is the JDBC driver. I submitted a patch for the Go >> driver, but it hasn't been committed yet. >> > > On the JDBC driver, strictly speaking, code has not been released yet. > It is scheduled for v 42.2.0, and maybe the wiki should also mention from > what version of the driver it is supported (I guess for all cases, unless > their versioning would be synced with PostgreSQL's). We won't by syncing our version numbers with Postgres Dave Cramer da...@postgresintl.com www.postgresintl.com
[HACKERS] JDBC 42.1.4 released
*Notable changes* - Statements with non-zero fetchSize no longer require server-side named handle. This might cause issues when using old PostgreSQL versions (pre-8.4)+fetchSize+interleaved ResultSet processing combo. see issue 869 <https://github.com/pgjdbc/pgjdbc/issues/869> Thanks, Dave Cramer
[HACKERS] question about replication docs
The following makes an explicit reference to the simple query protocol being the only protocol allowed in walsender mode. It is my understanding this is true for logical replication as well ?? 51.3. Streaming Replication Protocol To initiate streaming replication, the frontend sends the replication parameter in the startup message. A Boolean value of true tells the backend to go into walsender mode, wherein a small set of replication commands can be issued instead of SQL statements. Only the simple query protocol can be used in walsender mode. Replication commands are logged in the server log when log_replication_commands is enabled. Passing database as the value instructs walsender to connect to the database specified in the dbname parameter, which will allow the connection to be used for logical replication from that database. Dave Cramer
Re: [HACKERS] Statement-level rollback
You have to turn it on using the autosave parameter. it's not on by default, and apparently not documented Dave Cramer da...@postgresintl.com www.postgresintl.com On 7 March 2017 at 17:15, legrand legrand <legrand_legr...@hotmail.com> wrote: > Thanks ! > > that's a very good new ! > > > I'm still receiving the famous > > "current transaction is aborted" error > when usingversion 42.0.0 with > > jdbc:postgresql://localhost:5432/postgres?autosave=always > > > But I will see that with pgjdbc team ;o) > Regards > PAscal > > -- > View this message in context: RE: Statement-level rollback > <http://www.postgresql-archive.org/Statement-level-rollback-tp5946725p5948053.html> > > Sent from the PostgreSQL - hackers mailing list archive > <http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html> at > Nabble.com. >
Re: [HACKERS] Statement-level rollback
On 7 March 2017 at 16:18, Michael Banck <michael.ba...@credativ.de> wrote: > On Tue, Mar 07, 2017 at 01:49:29PM -0700, legrand legrand wrote: > > JDBC has nothing and developers has to play with savepoint as described > > http://blog.endpoint.com/2015/02/postgres-onerrorrollback-explained.html > > JDBC has it since 9.4.1210 (2016-09-07), unless I am mistaken: > > https://github.com/pgjdbc/pgjdbc/commit/adc08d57d2a9726309ea80d574b1db > 835396c1c8 I thought he meant we have to play with savepoints. Yes, we do it for you now Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [JDBC] [HACKERS] PGSERVICEFILE as a connection string parameter
+Vladimir On 27 February 2017 at 11:36, Andres Freund <and...@anarazel.de> wrote: > On 2017-02-27 16:23:46 +0100, Magnus Hagander wrote: > > On Mon, Feb 27, 2017 at 7:03 AM, Andres Freund <and...@anarazel.de> > wrote: > > > On 2017-02-27 14:43:49 +0900, Michael Paquier wrote: > > > > I bumped into a case where it would have been rather useful to > specify > > > > a service file path in a connection string with a service name. In my > > > > case, I have finished by setting up PGSERVICEFILE, but now like > > > > PGPASSFILE I think that being able to define the service file > > > > available as well as a connection parameter would be useful as well. > > > > > > > > I am not planning to work on that immediately (there is one day left > > > > for the last CF of PG10!), but I was wondering if people would be > > > > interested in something like that. > > > > > > Hm - I'm not sure that's a good idea. service files are a libpq > feature, > > > but connection strings are a bit more universal than just libpq... > > > > > > > That same argument applies to PGPASSFILE, does it not? > > It does. I'm not really convinced it's a good idea to have that as a > full blown parameter, but as you say: > > > Properly implementing PGSERVICEFILE is more complicated though -- as it > > requires LDAP support to go there the whole way for example. > > > > > But it might not hurt to encourage other drivers (such as jdbc) to > > support at least the basic format of pgpass. > > Probably makes sense to bring in some of the external driver authors > (jdbc, npgsql CCed). > > Currently PGPASS is in the users home directory Many JDBC applications are in much larger apps such as tomcat, etal this concept is a bit foreign to JDBC. That being said I don't think it's difficult to implement. Just somewhat harder to specify for us. psql is rather limited being a command line app which is *usually* evoked directly from the command line. Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] [JDBC] SEGFAULT in HEAD with replication
Not sure you can get the exec. We are working on producing the BT from
Travis-CI or I will build and run the test locally and get the trace
Dave Cramer
On 19 January 2017 at 15:31, Craig Ringer <craig.rin...@2ndquadrant.com>
wrote:
>
>
> On 20 Jan. 2017 04:13, "Robert Haas" <robertmh...@gmail.com> wrote:
>
> On Thu, Jan 19, 2017 at 12:09 PM, Dave Cramer <davecra...@gmail.com>
> wrote:
> > I would have expected more, but this is what I have
> >
> > bt full
> > #0 InitPredicateLocks () at predicate.c:1250
> > i =
> > info = {num_partitions = 1, ssize = 140731424825288, dsize = 1,
> > max_dsize = 0, ffactor = 140731424836952, keysize =
> > 140356326474085,
> > entrysize = 140728909791233, hash = 0x7ffe96960d58,
> > match = 0x16da2d1, keycopy = 0x7ffe96960d58, alloc = 0x1703af0,
> > hcxt = 0x16da2d0, hctl = 0x0}
> > max_table_size = 117899280
> > requestSize =
> > found = 0 '\000'
>
> I would say that's not a valid stack trace. There hasn't been a
> change made to that file since October of last year, and the crash is
> apparently recent; also, line 1250 in that file doesn't look like
> something that can crash. I would guess that you're using an
> executable which doesn't match the core dump, or perhaps that you
> don't have complete debug symbols.
>
>
> You really need the same compiler flags, configure opts and preferably
> much the same compiler. Similar or same C library etc.
>
> Can't we get the executables from Travis CI or from whoever produced the
> core? Or get them to obtain a bt ?
>
Re: [HACKERS] [JDBC] SEGFAULT in HEAD with replication
I would have expected more, but this is what I have
bt full
#0 InitPredicateLocks () at predicate.c:1250
i =
info = {num_partitions = 1, ssize = 140731424825288, dsize = 1,
max_dsize = 0, ffactor = 140731424836952, keysize =
140356326474085,
entrysize = 140728909791233, hash = 0x7ffe96960d58,
match = 0x16da2d1, keycopy = 0x7ffe96960d58, alloc = 0x1703af0,
hcxt = 0x16da2d0, hctl = 0x0}
max_table_size = 117899280
requestSize =
found = 0 '\000'
Dave Cramer
On 19 January 2017 at 12:05, Dave Cramer <davecra...@gmail.com> wrote:
> I'll try to get the stack trace from the core dump, have to build master
> first
>
> Dave Cramer
>
> On 19 January 2017 at 12:01, Jorge Solórzano <jor...@gmail.com> wrote:
>
>> Robert, the logs I get from postgres (at least the section that matters)
>> is here:
>> If you need something else just ask...
>>
>> 2017-01-19 08:54:57.319 CST [31734] DEBUG: CommitTransaction(1) name:
>>> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.321 CST [31734] DEBUG: parse S_1: DROP TABLE
>>> rollbacktest CASCADE
>>> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest
>>> CASCADE
>>> 2017-01-19 08:54:57.321 CST [31734] DEBUG: StartTransaction(1) name:
>>> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest
>>> CASCADE
>>> 2017-01-19 08:54:57.321 CST [31734] LOG: duration: 0.289 ms
>>> 2017-01-19 08:54:57.321 CST [31734] DEBUG: bind to S_1
>>> 2017-01-19 08:54:57.321 CST [31734] LOG: duration: 0.073 ms
>>> 2017-01-19 08:54:57.321 CST [31734] LOG: execute S_1: DROP TABLE
>>> rollbacktest CASCADE
>>> 2017-01-19 08:54:57.321 CST [31734] ERROR: table "rollbacktest" does
>>> not exist
>>> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest
>>> CASCADE
>>> 2017-01-19 08:54:57.322 CST [31734] DEBUG: parse S_2: CREATE TABLE
>>> rollbacktest (a int, str text)
>>> 2017-01-19 08:54:57.322 CST [31734] STATEMENT: CREATE TABLE
>>> rollbacktest (a int, str text)
>>> 2017-01-19 08:54:57.322 CST [31734] DEBUG: StartTransaction(1) name:
>>> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.322 CST [31734] STATEMENT: CREATE TABLE
>>> rollbacktest (a int, str text)
>>> 2017-01-19 08:54:57.322 CST [31734] LOG: duration: 0.279 ms
>>> 2017-01-19 08:54:57.323 CST [31734] DEBUG: bind to S_2
>>> 2017-01-19 08:54:57.323 CST [31734] LOG: duration: 0.163 ms
>>> 2017-01-19 08:54:57.323 CST [31734] LOG: execute S_2: CREATE TABLE
>>> rollbacktest (a int, str text)
>>> 2017-01-19 08:54:57.324 CST [31734] DEBUG: building index
>>> "pg_toast_163960_index" on table "pg_toast_163960"
>>> 2017-01-19 08:54:57.324 CST [31734] STATEMENT: CREATE TABLE
>>> rollbacktest (a int, str text)
>>> 2017-01-19 08:54:57.327 CST [31734] LOG: duration: 4.232 ms
>>> 2017-01-19 08:54:57.327 CST [31734] DEBUG: CommitTransaction(1) name:
>>> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 270571/1/5
>>> 2017-01-19 08:54:57.329 CST [31734] DEBUG: parse S_3:
>>> 2017-01-19 08:54:57.329 CST [31734] STATEMENT:
>>> 2017-01-19 08:54:57.329 CST [31734] DEBUG: StartTransaction(1) name:
>>> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.329 CST [31734] STATEMENT:
>>> 2017-01-19 08:54:57.329 CST [31734] LOG: duration: 0.148 ms
>>> 2017-01-19 08:54:57.329 CST [31734] DEBUG: bind to S_3
>>> 2017-01-19 08:54:57.329 CST [31734] LOG: duration: 0.088 ms
>>> 2017-01-19 08:54:57.329 CST [31734] DEBUG: CommitTransaction(1) name:
>>> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.330 CST [31734] DEBUG: parse S_4: BEGIN
>>> 2017-01-19 08:54:57.330 CST [31734] STATEMENT: BEGIN
>>> 2017-01-19 08:54:57.330 CST [31734] DEBUG: StartTransaction(1) name:
>>> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0
>>> 2017-01-19 08:54:57.330 CST [31734] STATEMENT: BEGIN
>>> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.189 ms
>>> 2017-01-19 08:54:57.331 CST [31734] DEBUG: bind to S_4
>>> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.105 ms
>>> 2017-01-19 08:54:57.331 CST [31734] LOG: execute S_4: BEGIN
>>> 2017-01-19 08:54:5
Re: [HACKERS] [JDBC] SEGFAULT in HEAD with replication
I'll try to get the stack trace from the core dump, have to build master first Dave Cramer On 19 January 2017 at 12:01, Jorge Solórzano <jor...@gmail.com> wrote: > Robert, the logs I get from postgres (at least the section that matters) > is here: > If you need something else just ask... > > 2017-01-19 08:54:57.319 CST [31734] DEBUG: CommitTransaction(1) name: >> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.321 CST [31734] DEBUG: parse S_1: DROP TABLE >> rollbacktest CASCADE >> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest >> CASCADE >> 2017-01-19 08:54:57.321 CST [31734] DEBUG: StartTransaction(1) name: >> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest >> CASCADE >> 2017-01-19 08:54:57.321 CST [31734] LOG: duration: 0.289 ms >> 2017-01-19 08:54:57.321 CST [31734] DEBUG: bind to S_1 >> 2017-01-19 08:54:57.321 CST [31734] LOG: duration: 0.073 ms >> 2017-01-19 08:54:57.321 CST [31734] LOG: execute S_1: DROP TABLE >> rollbacktest CASCADE >> 2017-01-19 08:54:57.321 CST [31734] ERROR: table "rollbacktest" does not >> exist >> 2017-01-19 08:54:57.321 CST [31734] STATEMENT: DROP TABLE rollbacktest >> CASCADE >> 2017-01-19 08:54:57.322 CST [31734] DEBUG: parse S_2: CREATE TABLE >> rollbacktest (a int, str text) >> 2017-01-19 08:54:57.322 CST [31734] STATEMENT: CREATE TABLE rollbacktest >> (a int, str text) >> 2017-01-19 08:54:57.322 CST [31734] DEBUG: StartTransaction(1) name: >> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.322 CST [31734] STATEMENT: CREATE TABLE rollbacktest >> (a int, str text) >> 2017-01-19 08:54:57.322 CST [31734] LOG: duration: 0.279 ms >> 2017-01-19 08:54:57.323 CST [31734] DEBUG: bind to S_2 >> 2017-01-19 08:54:57.323 CST [31734] LOG: duration: 0.163 ms >> 2017-01-19 08:54:57.323 CST [31734] LOG: execute S_2: CREATE TABLE >> rollbacktest (a int, str text) >> 2017-01-19 08:54:57.324 CST [31734] DEBUG: building index >> "pg_toast_163960_index" on table "pg_toast_163960" >> 2017-01-19 08:54:57.324 CST [31734] STATEMENT: CREATE TABLE rollbacktest >> (a int, str text) >> 2017-01-19 08:54:57.327 CST [31734] LOG: duration: 4.232 ms >> 2017-01-19 08:54:57.327 CST [31734] DEBUG: CommitTransaction(1) name: >> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 270571/1/5 >> 2017-01-19 08:54:57.329 CST [31734] DEBUG: parse S_3: >> 2017-01-19 08:54:57.329 CST [31734] STATEMENT: >> 2017-01-19 08:54:57.329 CST [31734] DEBUG: StartTransaction(1) name: >> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.329 CST [31734] STATEMENT: >> 2017-01-19 08:54:57.329 CST [31734] LOG: duration: 0.148 ms >> 2017-01-19 08:54:57.329 CST [31734] DEBUG: bind to S_3 >> 2017-01-19 08:54:57.329 CST [31734] LOG: duration: 0.088 ms >> 2017-01-19 08:54:57.329 CST [31734] DEBUG: CommitTransaction(1) name: >> unnamed; blockState: STARTED; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.330 CST [31734] DEBUG: parse S_4: BEGIN >> 2017-01-19 08:54:57.330 CST [31734] STATEMENT: BEGIN >> 2017-01-19 08:54:57.330 CST [31734] DEBUG: StartTransaction(1) name: >> unnamed; blockState: DEFAULT; state: INPROGR, xid/subid/cid: 0/1/0 >> 2017-01-19 08:54:57.330 CST [31734] STATEMENT: BEGIN >> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.189 ms >> 2017-01-19 08:54:57.331 CST [31734] DEBUG: bind to S_4 >> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.105 ms >> 2017-01-19 08:54:57.331 CST [31734] LOG: execute S_4: BEGIN >> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.154 ms >> 2017-01-19 08:54:57.331 CST [31734] DEBUG: parse S_5: insert into >> rollbacktest(a, str) values (0, 'test') >> 2017-01-19 08:54:57.331 CST [31734] STATEMENT: insert into >> rollbacktest(a, str) values (0, 'test') >> 2017-01-19 08:54:57.331 CST [31734] LOG: duration: 0.277 ms >> 2017-01-19 08:54:57.332 CST [31734] DEBUG: bind to S_5 >> 2017-01-19 08:54:57.332 CST [31734] LOG: duration: 0.168 ms >> 2017-01-19 08:54:57.332 CST [31734] LOG: execute S_5: insert into >> rollbacktest(a, str) values (0, 'test') >> 2017-01-19 08:54:57.332 CST [31734] LOG: duration: 0.233 ms >> 2017-01-19 08:54:57.333 CST [31734] DEBUG: parse S_6: select * from >> rollbacktest >> 2017-01-19 08:54:57.333 CST [31734] STATEMENT: select * from rollbacktest >> 2017-01-19 08:54:57.333 CST
[HACKERS] SEGFAULT in HEAD with replication
The travis job https://travis-ci.org/pgjdbc/pgjdbc/jobs/192517342 is seeing a segfault when we are testing against HEAD with REPLICATION turned on. Logs can be found here https://drive.google.com/drive/folders/0B-Heg5ZYCWbreEE4Uk5LdnJ5eWM?usp=sharing Regards, Dave Cramer
Re: [HACKERS] Retiring from the Core Team
On 11 January 2017 at 16:29, Josh Berkus <j...@agliodbs.com> wrote: > Hackers: > > You will have noticed that I haven't been very active for the past year. > My new work on Linux containers and Kubernetes has been even more > absorbing than I anticipated, and I just haven't had a lot of time for > PostgreSQL work. > > For that reason, as of today, I am stepping down from the PostgreSQL > Core Team. > I joined the PostgreSQL Core Team in 2003. I decided to take on project > advocacy, with the goal of making PostgreSQL one of the top three > databases in the world. Thanks to the many contributions by both > advocacy volunteers and developers -- as well as the efforts by > companies like EnterpriseDB and Heroku -- we've achieved that goal. > Along the way, we proved that community ownership of an OSS project can > compete with, and ultimately outlast, venture-funded startups. > Thanks for your hard work. PostgreSQL has come a long way since then! Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] building HEAD on macos fails with #error no source of random numbers configured
That will teach me to copy and paste a config from somewhere ... Thanks Dave Cramer On 9 December 2016 at 10:43, Tom Lane <t...@sss.pgh.pa.us> wrote: > Dave Cramer <davecra...@gmail.com> writes: > > Looking at src/port/pg_strong_random.c this would be a bug in autoconf > > It looks more like self-inflicted damage from here: > > > ./configure --prefix=/usr/local/pgsql/10 --enable-debug --with-python > > --with-openssl --with-libraries=/usr/local/opt/openssl/lib > > --with-includes=/usr/local/opt/openssl/include/ --no-create > --no-recursion > > Why are you using either --no-create or --no-recursion? The former > *definitely* breaks things: > > $ ./configure --help | grep create > -n, --no-create do not create output files > > Presumably the proximate cause of that error message is that configure > hasn't updated pg_config.h from some ancient version thereof, as a > consequence of this switch. > > I'm not sure what --no-recursion does, but I would say that we'd > consider that unsupported as well. > > regards, tom lane >
[HACKERS] building HEAD on macos fails with #error no source of random numbers configured
Looking at src/port/pg_strong_random.c this would be a bug in autoconf #else /* The autoconf script should not have allowed this */ #error no source of random numbers configured My configure line is: ./configure --prefix=/usr/local/pgsql/10 --enable-debug --with-python --with-openssl --with-libraries=/usr/local/opt/openssl/lib --with-includes=/usr/local/opt/openssl/include/ --no-create --no-recursion I am using openssl-1.0.2j Regards, Dave Cramer
Re: [HACKERS] RFC Changing the version number for JDBC
On 27 November 2016 at 11:29, Tom Lane <t...@sss.pgh.pa.us> wrote: > Dave Cramer <davecra...@gmail.com> writes: > > We are proposing changing the JDBC version from > > 9.4. to 42.x.x > > > We have two issues we are trying to address here. > > > 1) we do not want to be tied to the server release schedule. This has > been > > somewhat addressed already but has left us with the second issue. > > > 2) Avoid confusion as to which version to use with which server version. > > Currently the naming scheme has 9.4 in it which leads people to believe > it > > is for server version 9.4 > > To clarify --- are you planning to advance the "42" part fairly often, > or is it intended to stay static? If the latter, I think this design > is shortsighted. Given current project policies, server version 42 > should come out in 2049, plus or minus a bit, and you'd be right back > with the is-this-meant-to-match-the-server-version problem. > > Admittedly, many of us won't be around in 2049, but it's not out of > the realm of possibility that the project would still be kicking. > > If you advance the major version part every year or so, it'd be OK > since you could expect to stay well ahead of the server's major > version number forever. > Ya we could easily stay ahead of the server. Thanks, Dave Cramer
[HACKERS] RFC Changing the version number for JDBC
We are proposing changing the JDBC version from 9.4. to 42.x.x We have two issues we are trying to address here. 1) we do not want to be tied to the server release schedule. This has been somewhat addressed already but has left us with the second issue. 2) Avoid confusion as to which version to use with which server version. Currently the naming scheme has 9.4 in it which leads people to believe it is for server version 9.4 The driver is version agnostic for the most point so there is no reason to tie it to a specific server version. I've already talked to the package managers and they see no problems. Please speak up now if you foresee any issues with this idea. FYI, 42 was more or less chosen at random. But it is large enough to avoid any future conflicts with the server, and greater than 9 to avoid issues with maven requesting things like > 9 Dave Cramer
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
On 10 October 2016 at 13:42, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > > 2016-10-10 12:31 GMT+02:00 Heikki Linnakangas <hlinn...@iki.fi>: > >> On 10/01/2016 02:45 AM, Jim Nasby wrote: >> >>> On 9/29/16 1:51 PM, Heikki Linnakangas wrote: >>> >>>> Now, back to multi-dimensional arrays. I can see that the Sequence >>>> representation is problematic, with arrays, because if you have a python >>>> list of lists, like [[1, 2]], it's not immediately clear if that's a >>>> one-dimensional array of tuples, or two-dimensional array of integers. >>>> Then again, we do have the type definitions available. So is it really >>>> ambiguous? >>>> >>> >>> [[1,2]] is a list of lists... >>> In [4]: b=[[1,2]] >>> >>> In [5]: type(b) >>> Out[5]: list >>> >>> In [6]: type(b[0]) >>> Out[6]: list >>> >>> If you want a list of tuples... >>> In [7]: c=[(1,2)] >>> >>> In [8]: type(c) >>> Out[8]: list >>> >>> In [9]: type(c[0]) >>> Out[9]: tuple >>> >> >> Hmm, so we would start to treat lists and tuples differently? A Python >> list would be converted into an array, and a Python tuple would be >> converted into a composite type. That does make a lot of sense. The only >> problem is that it's not backwards-compatible. A PL/python function that >> returns an SQL array of rows, and does that by returning Python list of >> lists, it would start failing. >> > > is not possible do decision in last moment - on PL/Postgres interface? > There the expected type should be known. > > Regards > > Pavel > > >> >> I think we should bite the bullet and do that anyway. As long as it's >> clearly documented, and the error message you get contains a clear hint on >> how to fix it, I don't think it would be too painful to adjust existing >> application. >> >> We could continue to accept a Python list for a plain composite type, >> this would only affect arrays of composite types. >> >> I don't use PL/python much myself, so I don't feel qualified to make the >> call, though. Any 3rd opinions? > > Can't you determine the correct output based on the function output definition ? For instance if the function output was an array type then we would return the list as an array if the function output was a set of then we return tuples ? Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
On 27 September 2016 at 14:58, Heikki Linnakangas <hlinn...@iki.fi> wrote:
> On 09/27/2016 02:04 PM, Dave Cramer wrote:
>
>> On 26 September 2016 at 14:52, Dave Cramer <p...@fastcrypt.com> wrote:
>>
>>> This crashes with arrays with non-default lower bounds:
>>>>
>>>> postgres=# SELECT * FROM test_type_conversion_array_int
>>>> 4('[2:4]={1,2,3}');
>>>> INFO: ([1, 2, ], )
>>>> server closed the connection unexpectedly
>>>> This probably means the server terminated abnormally
>>>> before or while processing the request.
>>>>
>>>> Attached patch fixes this bug, and adds a test for it.
>>>>
>>>
> I spent some more time massaging this:
>
> * Changed the loops from iterative to recursive style. I think this indeed
> is slightly easier to understand.
>
> * Fixed another segfault, with too deeply nested lists:
>
> CREATE or replace FUNCTION test_type_conversion_mdarray_toodeep() RETURNS
> int[] AS $$
> return [[1]]
> $$ LANGUAGE plpythonu;
>
> * Also, in PLySequence_ToArray(), we must check that the 'len' of the
> array doesn't overflow.
>
> * Fixed reference leak in the loop in PLySequence_ToArray() to count the
> number of dimensions.
>
> I'd like to see some updates to the docs for this. The manual doesn't
>>>> currently say anything about multi-dimensional arrays in pl/python, but
>>>> it
>>>> should've mentioned that they're not supported. Now that it is
>>>> supported,
>>>> should mention that, and explain briefly that a multi-dimensional array
>>>> is
>>>> mapped to a python list of lists.
>>>>
>>>> If the code passes I'll fix the docs
>>>
>>
> Please do, thanks!
>
>
see attached
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
0002-WIP-Multi-dimensional-arrays-in-PL-python.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
On 26 September 2016 at 14:52, Dave Cramer <p...@fastcrypt.com> wrote:
>
>
>
>>
>> This crashes with arrays with non-default lower bounds:
>>
>> postgres=# SELECT * FROM test_type_conversion_array_int
>> 4('[2:4]={1,2,3}');
>> INFO: ([1, 2, ], )
>> server closed the connection unexpectedly
>> This probably means the server terminated abnormally
>> before or while processing the request.
>>
>> Attached patch fixes this bug, and adds a test for it.
>
>>
>> I'd like to see some updates to the docs for this. The manual doesn't
>> currently say anything about multi-dimensional arrays in pl/python, but it
>> should've mentioned that they're not supported. Now that it is supported,
>> should mention that, and explain briefly that a multi-dimensional array is
>> mapped to a python list of lists.
>>
>> If the code passes I'll fix the docs
>
>> It seems we don't have any mention in the docs about arrays with
>> non-default lower-bounds ATM. That's not this patch's fault, but it would
>> be good to point out that the lower bounds are discarded when an array is
>> passed to python.
>>
>> I find the loop in PLyList_FromArray() quite difficult to understand. Are
>> the comments there mixing up the "inner" and "outer" dimensions? I wonder
>> if that would be easier to read, if it was written in a recursive-style,
>> rather than iterative with stacks for the dimensions.
>>
>> Yes, it is fairly convoluted.
>
>
> Dave Cramer
>
> da...@postgresintl.com
> www.postgresintl.com
>
>
>
PL-Python-adding-support-for-multi-dimensional-arrays-20160926.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
>
> This crashes with arrays with non-default lower bounds:
>
> postgres=# SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
> INFO: ([1, 2, ], )
> server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
>
> Attached patch fixes this bug, and adds a test for it.
>
> I'd like to see some updates to the docs for this. The manual doesn't
> currently say anything about multi-dimensional arrays in pl/python, but it
> should've mentioned that they're not supported. Now that it is supported,
> should mention that, and explain briefly that a multi-dimensional array is
> mapped to a python list of lists.
>
> If the code passes I'll fix the docs
> It seems we don't have any mention in the docs about arrays with
> non-default lower-bounds ATM. That's not this patch's fault, but it would
> be good to point out that the lower bounds are discarded when an array is
> passed to python.
>
> I find the loop in PLyList_FromArray() quite difficult to understand. Are
> the comments there mixing up the "inner" and "outer" dimensions? I wonder
> if that would be easier to read, if it was written in a recursive-style,
> rather than iterative with stacks for the dimensions.
>
> Yes, it is fairly convoluted.
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
On 18 September 2016 at 09:27, Dave Cramer <p...@fastcrypt.com> wrote:
>
> On 10 August 2016 at 01:53, Pavel Stehule <pavel.steh...@gmail.com> wrote:
>
>> Hi
>>
>> 2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishche...@pivotal.io>:
>>
>>> On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
>>> agrishche...@pivotal.io> wrote:
>>>
>>>> Hi
>>>>
>>>> Current implementation of PL/Python does not allow the use of
>>>> multi-dimensional arrays, for both input and output parameters. This forces
>>>> end users to introduce workarounds like casting arrays to text before
>>>> passing them to the functions and parsing them after, which is an
>>>> error-prone approach
>>>>
>>>> This patch adds support for multi-dimensional arrays as both input and
>>>> output parameters for PL/Python functions. The number of dimensions
>>>> supported is limited by Postgres MAXDIM macrovariable, by default equal to
>>>> 6. Both input and output multi-dimensional arrays should have fixed
>>>> dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
>>>> represent MxNxK cube, etc.
>>>>
>>>> This patch does not support multi-dimensional arrays of composite
>>>> types, as composite types in Python might be represented as iterators and
>>>> there is no obvious way to find out when the nested array stops and
>>>> composite type structure starts. For example, if we have a composite type
>>>> of (int, text), we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'],
>>>> [4,'d'] ] ]", and it is hard to find out that the first two lists are
>>>> lists, and the third one represents structure. Things are getting even more
>>>> complex when you have arrays as members of composite type. This is why I
>>>> think this limitation is reasonable.
>>>>
>>>> Given the function:
>>>>
>>>> CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
>>>> int4[] AS $$
>>>> plpy.info(x, type(x))
>>>> return x
>>>> $$ LANGUAGE plpythonu;
>>>>
>>>> Before patch:
>>>>
>>>> # SELECT * FROM test_type_conversion_array_int
>>>> 4(ARRAY[[1,2,3],[4,5,6]]);
>>>> ERROR: cannot convert multidimensional array to Python list
>>>> DETAIL: PL/Python only supports one-dimensional arrays.
>>>> CONTEXT: PL/Python function "test_type_conversion_array_int4"
>>>>
>>>>
>>>> After patch:
>>>>
>>>> # SELECT * FROM test_type_conversion_array_int
>>>> 4(ARRAY[[1,2,3],[4,5,6]]);
>>>> INFO: ([[1, 2, 3], [4, 5, 6]], )
>>>> test_type_conversion_array_int4
>>>> -
>>>> {{1,2,3},{4,5,6}}
>>>> (1 row)
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Alexey Grishchenko
>>>>
>>>
>>> Also this patch incorporates the fix for https://www.postgresql.org
>>> /message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-2NR9jUfv3q
>>> wHA%40mail.gmail.com, as they touch the same piece of code - array
>>> manipulation in PL/Python
>>>
>>>
>> I am sending review of this patch:
>>
>> 1. The implemented functionality is clearly benefit - passing MD arrays,
>> pretty faster passing bigger arrays
>> 2. I was able to use this patch cleanly without any errors or warnings
>> 3. There is no any error or warning
>> 4. All tests passed - I tested Python 2.7 and Python 3.5
>> 5. The code is well commented and clean
>> 6. For this new functionality the documentation is not necessary
>>
>> 7. I invite more regress tests for both directions (Python <-> Postgres)
>> for more than two dimensions
>>
>> My only one objection is not enough regress tests - after fixing this
>> patch will be ready for commiters.
>>
>> Good work, Alexey
>>
>> Thank you
>>
>> Regards
>>
>> Pavel
>>
>>
>>> --
>>> Best regards,
>>> Alexey Grishchenko
>>>
>>
>>
>
> Pavel,
>
> I will pick this up.
>
>
> Pavel,
Please see attached patch which provides more test cases
I just realized this patch contains the original patch as well. What is the
protocol for sending in subsequent patches ?
>
> Dave Cramer
>
> da...@postgresintl.com
> www.postgresintl.com
>
>
0002-PL-Python-adding-support-for-multi-dimensional-arrays.patch
Description: Binary data
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Fix for PL/Python slow input arrays traversal issue
Yes, this should be closed as it is contained in https://commitfest.postgresql.org/10/697/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PL/Python adding support for multi-dimensional arrays
On 10 August 2016 at 01:53, Pavel Stehule <pavel.steh...@gmail.com> wrote:
> Hi
>
> 2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishche...@pivotal.io>:
>
>> On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
>> agrishche...@pivotal.io> wrote:
>>
>>> Hi
>>>
>>> Current implementation of PL/Python does not allow the use of
>>> multi-dimensional arrays, for both input and output parameters. This forces
>>> end users to introduce workarounds like casting arrays to text before
>>> passing them to the functions and parsing them after, which is an
>>> error-prone approach
>>>
>>> This patch adds support for multi-dimensional arrays as both input and
>>> output parameters for PL/Python functions. The number of dimensions
>>> supported is limited by Postgres MAXDIM macrovariable, by default equal to
>>> 6. Both input and output multi-dimensional arrays should have fixed
>>> dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
>>> represent MxNxK cube, etc.
>>>
>>> This patch does not support multi-dimensional arrays of composite types,
>>> as composite types in Python might be represented as iterators and there is
>>> no obvious way to find out when the nested array stops and composite type
>>> structure starts. For example, if we have a composite type of (int, text),
>>> we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
>>> it is hard to find out that the first two lists are lists, and the third
>>> one represents structure. Things are getting even more complex when you
>>> have arrays as members of composite type. This is why I think this
>>> limitation is reasonable.
>>>
>>> Given the function:
>>>
>>> CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
>>> int4[] AS $$
>>> plpy.info(x, type(x))
>>> return x
>>> $$ LANGUAGE plpythonu;
>>>
>>> Before patch:
>>>
>>> # SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
>>> ERROR: cannot convert multidimensional array to Python list
>>> DETAIL: PL/Python only supports one-dimensional arrays.
>>> CONTEXT: PL/Python function "test_type_conversion_array_int4"
>>>
>>>
>>> After patch:
>>>
>>> # SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
>>> INFO: ([[1, 2, 3], [4, 5, 6]], )
>>> test_type_conversion_array_int4
>>> -
>>> {{1,2,3},{4,5,6}}
>>> (1 row)
>>>
>>>
>>> --
>>> Best regards,
>>> Alexey Grishchenko
>>>
>>
>> Also this patch incorporates the fix for https://www.postgresql.org
>> /message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-
>> 2NR9jUfv3qwHA%40mail.gmail.com, as they touch the same piece of code -
>> array manipulation in PL/Python
>>
>>
> I am sending review of this patch:
>
> 1. The implemented functionality is clearly benefit - passing MD arrays,
> pretty faster passing bigger arrays
> 2. I was able to use this patch cleanly without any errors or warnings
> 3. There is no any error or warning
> 4. All tests passed - I tested Python 2.7 and Python 3.5
> 5. The code is well commented and clean
> 6. For this new functionality the documentation is not necessary
>
> 7. I invite more regress tests for both directions (Python <-> Postgres)
> for more than two dimensions
>
> My only one objection is not enough regress tests - after fixing this
> patch will be ready for commiters.
>
> Good work, Alexey
>
> Thank you
>
> Regards
>
> Pavel
>
>
>> --
>> Best regards,
>> Alexey Grishchenko
>>
>
>
Pavel,
I will pick this up.
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
Re: [HACKERS] Fix for PL/Python slow input arrays traversal issue
Pavel, I will pick these up. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Send numeric version to clients
On 29 August 2016 at 15:42, Tom Lane <t...@sss.pgh.pa.us> wrote: > Kevin Grittner <kgri...@gmail.com> writes: > > Regarding Java, for anything above the driver itself the > > JDBC API says the DatabaseMetaData class must implement these > > methods: > > ... > > That *should* make it just a problem for the driver itself. That > > seems simple enough until you check what those methods have been > > returning so far. > > Seems like we just make getDatabaseMinorVersion() return zero for > any major >= 10. That might not have been the best thing to do in > a green field, but given the precedent ... > > regards, tom lane > > > seems to me that it should report 10 for the major and whatever comes after the . for the minor ? Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] Slowness of extended protocol
> > > I'm not going to respond to the part about dealing with prepared > statements errors, since I think we've already covered that and there's > nothing new being said. I don't find automatic savepointing acceptable, and > a significant change of the PostgreSQL protocol to support this doesn't > seem reasonable (but you can try proposing). > As mentioned before. JDBC is not the only postgres driver to do this the ODBC driver does this as well. This is a requested feature by users. We didn't just decide to do it on our own. One thing to keep in mind is that both JDBC and ODBC are not exclusively PostgreSQL drivers and as such we sometimes have to jump through hoops to provide the semantics requested by the API. Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] Slowness of extended protocol
On 11 August 2016 at 10:18, Shay Rojansky <r...@roji.org> wrote: > > > On Thu, Aug 11, 2016 at 1:22 PM, Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> wrote: > > 2) The driver can use safepoints and autorollback to the good "right >> before failure" state in case of a known failure. Here's the >> implementation: https://github.com/pgjdbc/pgjdbc/pull/477 >> > As far as I can remember, performance overheads are close to zero (no >> extra roundtrips to create a safepoint) >> > > What? Do you mean you do implicit savepoints and autorollback too? How > does the driver decide when to do a savepoint? Is it on every single > command? If not, commands can get lost when an error is raised and you > automatically roll back? If you do a savepoint on every single command, > that surely would impact performance even without extra roundtrips...? > > You seem to have a very "unique" idea of what a database driver should do > under-the-hood for its users. At the very least I can say that your concept > is very far from almost any database driver I've seen up to now (PostgreSQL > JDBC, psycopg, Npgsql, libpq...). I'm not aware of other drivers that > implicitly prepare statements, and definitely of no drivers that internally > create savepoints and roll the back without explicit user APIs. At the very > least you should be aware (and also clearly admit!) that you're doing > something very different - not necessarily wrong - and not attempt to > impose your ideas on everyone as if it's the only true way to write a db > driver. > A number of other drivers default to this behaviour, including at least MS-SQL and Oracle. psqlODBC also supports this behaviour with statement rollback mode. And obviously PostgreSQL JDBC which Vladimir is referring to. Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] No longer possible to query catalogs for index capabilities?
On 8 August 2016 at 03:49, Vladimir Sitnikov <sitnikov.vladi...@gmail.com> wrote: > > > Tom Lane <t...@sss.pgh.pa.us>: > >> FWIW, this thread started on 25-July, less than two weeks ago. > > > Technically speaking, there was a pgsql-jdbc thread started on May 14: > https://www.postgresql.org/message-id/nh72v6%24582%241%40ger.gmane.org > > 9.6beta1 was released on May 12 > > The fact that it wasn't raised >> till more than 6 months after we committed the pg_am changes > > > This means that nobody was testing compatibility of "postgresql's master > branch with existing third-party clients". > Testing against well-known clients makes sense to catch bugs early. > > I've added "build postgresql from master branch" test to the pgjdbc's > regression suite a week ago, so I hope it would highlight issues early > (even before the official postgresql beta is released). > > However, pgjdbc tests are executed only for pgjdbc commits, so if there's > no pgjdbc changes, then there is no logic to trigger "try newer postgres > with current pgjdbc". > > Ideally, postgresql's regression suite should validate well-known clients > as well. > I've no idea how long would it take to add something to postgresql's > buildfarm, so I just went ahead and created Travis test configuration at > https://github.com/vlsi/postgres > Do you think those Travis changes can be merged to the upstream? > > I mean the following: > 1) Activate TravisCI integration for https://github.com/postgres/postgres > mirror. > 2) Add relevant Travis CI file so it checks postgresql's regression suite, > and the other > 3) Add build badge to the readme (link to travis ci build) for simplified > navigation to test results. > > Here's the commit: https://github.com/vlsi/postgres/commit/ > 4841f8bc00b7c6717d91f51c98979ce84b4f7df3 > Here's how test results look like: https://travis-ci.org/vlsi/postgres > > Nice work +! Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] regression test for extended query protocol
On 3 August 2016 at 10:53, Tom Lane <t...@sss.pgh.pa.us> wrote: > Tatsuo Ishii <is...@postgresql.org> writes: > >>> In my understanding, we don't have any regression test for protocol > >>> level prepared query (we do have SQL level prepared query tests, > >>> though). Shouldn't we add those tests to the regression test suites? > > >> I thought that ECPG was covering a portion of that. Right? > > > In my understanding, ECPG calls libpq, thus the test cases are limited > > to the cases which are only possible with libpq (or it may be even > > limited to the cases where ECPG deal with.) > > I do not think it's very practical for the core tests to try to cover > any behavior that's not reachable via libpq. Even testing stuff that's > not reached through ecpg would require a whole slew of new infrastructure > with no use except testing. > > I think realistically a better approach to this would be to get some > buildfarm members running the JDBC regression tests (I assume there > are some ...) > > regards, tom lane > > We currently run tests every time a PR is created on github, but I don't think there are any animals running the JDBC test suite We can add tests, what exactly do we want to test. Then setting up an animal to run the tests would be fairly straight forward. Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] One process per session lack of sharing
On 18 July 2016 at 06:04, <amatv...@bitec.ru> wrote: > Hi > > > There's https://github.com/jnr/jnr-ffi that enables to call C > > functions without resorting to writing JNI wrappers. > I have not said that you are wrong. > It's the dark side of "like seprate process" > They can cheaply call sql from jvm. > And they can't cheaply call jvm from sql. > This https://github.com/davecramer/plj-new is a very old project that did work at one time which attempted to do RPC calls to the jvm to address exactly this problem. However "cheaply" calling jvm from sql or vice-versa is not really possible. I do like the idea of the background worker and shared memory though. Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] SPI_exec ERROR in pl/r of R 3.2.4 on PostgreSQL on Windows 7
Does anyone else have a Windows 7 installation we can test this on ?
This
https://github.com/postgres-plr/plr/files/191013/plr-8.3.0.16-pg9.5-win32.zip
is actually a 64 bit version built on windows 10. I've had one confirmation
that it works.
Dave
Dave Cramer
da...@postgresintl.com
www.postgresintl.com
On 30 April 2016 at 12:39, Andre Mikulec <andre_miku...@hotmail.com> wrote:
> Joe,
>
> "
> Who did the compiling? Did you compile everything yourself, or use
> binary installers for some of it? If so, which ones?
> "
>
> This is really a continuation of the experience I had with Dave Cramer in
> here.
>
> Postgresql 9.5 support #1
> https://github.com/postgres-plr/plr/issues/1
>
>
> To try to figure out the problem, ( and perhaps? eliminate Microsoft from
> the problem),
> I compiled a PostgreSQL [debug] version myself.
>
> C:\Users\AnonymousUser\Desktop\PostgreSQL.9.5.1\App\PgSQL>chcp 1252 > nul
> && "%PGSQL%\bin\psql.exe"
> psql (9.5.1)
> Type "help" for help.
>
> postgres=# select version();
> version
>
>
> PostgreSQL 9.5.1 on i686-pc-mingw32, compiled by gcc.exe
> (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 5.3.0, 64-bit
> (1 row)
>
> I also built a non-debug plr.dll/plr myself too.
> I modified ( mostly simplified )
> https://github.com/jconway/plr/blob/master/Makefile
> in the Makefile, I eliminated ( by much trial and error ) the OS
> non_window stuff, the pkg-config stuff, and the PGXS stuff .
>
> Then I did,
> AnonymousUser@ANONYMOUSX /c/Users/AnonymousUser/postgresql-9.4.1/contrib
> $ make -C plr clean
>
> AnonymousUser@ANONYMOUSX /c/Users/AnonymousUser/postgresql-9.4.1/contrib
> $ make -C plr all
>
> So now I have my own plr.dll.
>
> Then, I followed the instructions ( INSTALL.txt ) found in here.
> http://www.joeconway.com/plr/plr-8.3.0.16-pg9.4-win64.zip
>
> However, I used my own plr.dll/plr
> Seems, that in the destination, I had to copy plr.dll to plr, but that
> seems to work fine.
>
> Later, after I finish following "create extension plr;" found in
> http://www.joeconway.com/plr/doc/plr-install.html
>
> I do
>
> postgres=# select plr_version();
> plr_version
> -
> 08.03.00.16
> (1 row)
>
> postgres=# select plr_environ();
>
> (PGDATA,C:/Users/AnonymousUser/Desktop/PostgreSQL.9.5.1/Data/data)
> (PGDATABASE,postgres)
>
>
> (PGLOCALEDIR,"C:\\Users\\AnonymousUser\\Desktop\\PostgreSQL.9.5.1\\App\\PgSQL\\share\\")
>
> (PGLOG,"C:\\Users\\AnonymousUser\\Desktop\\PostgreSQL.9.5.1\\Data\\log.txt")
> (PGSQL,"C:\\Users\\AnonymousUser\\Desktop\\PostgreSQL.9.5.1\\App\\PgSQL")
>
> (PGSYSCONFDIR,C:/Users/AnonymousUser/Desktop/PostgreSQL.9.5.1/App/PgSQL/etc)
> (PGUSER,postgres)
>
> (R_ARCH,/x64)
> (R_HOME,C:/Users/AnonymousUser/Desktop/R-3.2.4)
> (R_KEEP_PKG_SOURCE,yes)
> (R_LIBS_USER,"C:\\Users\\AnonymousUser\\Documents/R/win-library/3.2")
> (R_USER,"C:\\Users\\AnonymousUser\\Documents")
>
> NOTE: The directory structure is from Postgre 9.4 Portable, I just use
> ONLY the directory structure.
> The one and ONLY file I use is the pgsql.cmd batch startup file ( I did my
> 'environment' and 'user friendly modifications.' )
>
> postgres=#
>
> I do this, I get no results, and no error.
>
> postgres=# SELECT * FROM pg_catalog.pg_class WHERE relname = 'plr_modules'
> AND relnamespace = 2200;
> relname | relnamespace | reltype | reloftype | relowner | relam |
> relfilenode | reltablespace | relpages
>
> -+--+-+---+--+---+-+---+--
> (0 rows)
>
> But, then this ( R language code ) strangely works.
>
> postgres=# select r_version(); # THIS IS THE 'R LANGUAGE' ( IF THE
> EXTENSION 'works' )
> r_version
> -
> (platform,x86_64-w64-mingw32)
> (arch,x86_64)
> (os,mingw32)
> (system,"x86_64, mingw32")
> (status,"")
> (major,3)
> (minor,2.4)
> (year,2016)
> (month,03)
> (day,10)
> ("svn rev",70301)
> (language,R)
> (version.string,"R version 3.2.4 (2016-03-10)")
> (nickname,"Very Secure Dishes")
> (14 rows)
>
> This does not work.
> postgres=# select upper(typname) || 'OID' as typename, oid from
> pg_catalog.pg_type where typtype = 'b' order by typname;
> ERROR: could not open file "bas
Re: [HACKERS] Add jsonb_compact(...) for whitespace-free jsonb to text
On 26 April 2016 at 12:49, Stephen Frost <sfr...@snowman.net> wrote: > * Ryan Pedela (rped...@datalanche.com) wrote: > > On Sun, Apr 24, 2016 at 4:02 PM, Sehrope Sarkuni <sehr...@jackdb.com> > wrote: > > > The default text representation of jsonb adds whitespace in between > > > key/value pairs (after the colon ":") and after successive properties > > > (after the comma ","): > > [...] > > > > It'd be nice to have a stable text representation of a jsonb value with > > > minimal whitespace. The latter would also save a few bytes per record > in > > > text output formats, on the wire, and in backups (ex: COPY ... TO > STDOUT). > > > > +1 > > > > I cannot comment on the patch itself, but I welcome jsonb_compact() or > some > > way to get JSON with no inserted whitespace. > > As I mentioned to Sehrope on IRC, at least for my 2c, if you want a > compact JSON format to reduce the amount of traffic over the wire or to > do things with on the client side, we should probably come up with a > binary format, rather than just hack out the whitespace. It's not like > representing numbers using ASCII characters is terribly efficient > either. > > +1 Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] Proposal: RETURNING primary_key()
On 3 April 2016 at 22:20, Stephen Frost <sfr...@snowman.net> wrote: > * Craig Ringer (cr...@2ndquadrant.com) wrote: > > On 4 April 2016 at 10:13, Dave Cramer <p...@fastcrypt.com> wrote: > > > Async notification is the easier part, I wasn't aware that the ssl > library > > > had this problem though > > > > AFAIK the issue is that even if there are bytes available on the > underlying > > socket, the SSL lib doesn't know if that means there are bytes readable > > from the wrapped SSL socket. The traffic on the underlying socket could > be > > renegotiation messages or whatever. > > > > We really need non-blocking reads. > > That would certainly be a good way to address this, but I'm guessing > it's non-trivial to implement. > > AFAICT, the ng driver still has to generate traffic as well. Dave Cramer da...@postgresintl.com www.postgresintl.com > Thanks! > > Stephen >
Re: [HACKERS] Proposal: RETURNING primary_key()
On 3 April 2016 at 21:56, Stephen Frost <sfr...@snowman.net> wrote: > Dave, > > * Dave Cramer (p...@fastcrypt.com) wrote: > > On 3 April 2016 at 15:35, Stephen Frost <sfr...@snowman.net> wrote: > > > Not generally much of a JDBC user myself, but the inability to avoid > > > polling for LISTEN notifications is a pretty big annoyance, which I > just > > > ran into with a client. I understand that -ng has a way to avoid that, > > > even for SSL connections. > > > > Yes, it is a custom api. Easy enough to add. Is this something of > interest ? > > I'd say that there is definite interest in this and there's a lot of > conversation about it on the interwebs (stackoverflow, etc). > > My understanding is that the problem is actually with the SSL library > that the JDBC driver uses and that it basically lies about if there are > bytes available for reading (claiming that there never is by always > returning zero). The -ng driver, as I understand it, uses a newer SSL > library which better supports asking if there are bytes available to > read. > > Hmmm. that complicates things... Async notification is the easier part, I wasn't aware that the ssl library had this problem though Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] Proposal: RETURNING primary_key()
On 3 April 2016 at 12:18, Igal @ Lucee.org <i...@lucee.org> wrote: > On 4/3/2016 8:21 AM, Dave Cramer wrote: > > > On 9 March 2016 at 20:49, Craig Ringer <cr...@2ndquadrant.com> wrote: > >> On 3/8/2016 5:12 PM, Craig Ringer wrote: >> >>> >>> Are there good reasons to use pgjdbc over pgjdbc-ng then? >>> >>> >> Maturity, support for older versions (-ng just punts on support for >> anything except new releases) and older JDBC specs, completeness of support >> for some extensions. TBH I haven't done a ton with -ng yet. >> >> > I'd like to turn this question around. Are there good reasons to use -ng > over pgjdbc ? > > As to your question, you may be interested to know that pgjdbc is more > performant than ng. > > That's good to know, but unfortunately pgjdbc is unusable for us until > https://github.com/pgjdbc/pgjdbc/issues/488 is fixed. > > Also, as I mentioned in the ticket, I can't imagine RETURNING * being > performant if, for example, I INSERT a large chunk of data like an image > data or an uploaded file. > > > Thanks for the reminder! So I"m guessing the reason to use ng is to avoid returning * ? Dave Cramer da...@postgresintl.com www.postgresintl.com > Igal > >
Re: [HACKERS] Proposal: RETURNING primary_key()
On 3 April 2016 at 15:35, Stephen Frost <sfr...@snowman.net> wrote: > * Dave Cramer (p...@fastcrypt.com) wrote: > > On 9 March 2016 at 20:49, Craig Ringer <cr...@2ndquadrant.com> wrote: > > > > > On 10 March 2016 at 00:41, Igal @ Lucee.org <i...@lucee.org> wrote: > > > > > >> On 3/8/2016 5:12 PM, Craig Ringer wrote: > > >> > > >>> One of the worst problems (IMO) is in the driver architecture its > self. > > >>> It attempts to prevent blocking by guestimating the server's send > buffer > > >>> state and its recv buffer state, trying to stop them filling and > causing > > >>> the server to block on writes. It should just avoid blocking on its > own > > >>> send buffer, which it can control with confidence. Or use some of > Java's > > >>> rather good concurrency/threading features to simultaneously consume > data > > >>> from the receive buffer and write to the send buffer when needed, > like > > >>> pgjdbc-ng does. > > >>> > > >> > > >> Are there good reasons to use pgjdbc over pgjdbc-ng then? > > >> > > >> > > > Maturity, support for older versions (-ng just punts on support for > > > anything except new releases) and older JDBC specs, completeness of > support > > > for some extensions. TBH I haven't done a ton with -ng yet. > > > > I'd like to turn this question around. Are there good reasons to use -ng > > over pgjdbc ? > > Not generally much of a JDBC user myself, but the inability to avoid > polling for LISTEN notifications is a pretty big annoyance, which I just > ran into with a client. I understand that -ng has a way to avoid that, > even for SSL connections. > > Yes, it is a custom api. Easy enough to add. Is this something of interest ? Dave Cramer da...@postgresintl.com www.postgresintl.com
Re: [HACKERS] Proposal: RETURNING primary_key()
On 9 March 2016 at 20:49, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 10 March 2016 at 00:41, Igal @ Lucee.org <i...@lucee.org> wrote: > >> On 3/8/2016 5:12 PM, Craig Ringer wrote: >> >>> One of the worst problems (IMO) is in the driver architecture its self. >>> It attempts to prevent blocking by guestimating the server's send buffer >>> state and its recv buffer state, trying to stop them filling and causing >>> the server to block on writes. It should just avoid blocking on its own >>> send buffer, which it can control with confidence. Or use some of Java's >>> rather good concurrency/threading features to simultaneously consume data >>> from the receive buffer and write to the send buffer when needed, like >>> pgjdbc-ng does. >>> >> >> Are there good reasons to use pgjdbc over pgjdbc-ng then? >> >> > Maturity, support for older versions (-ng just punts on support for > anything except new releases) and older JDBC specs, completeness of support > for some extensions. TBH I haven't done a ton with -ng yet. > > I'd like to turn this question around. Are there good reasons to use -ng over pgjdbc ? As to your question, you may be interested to know that pgjdbc is more performant than ng. Dave Cramer da...@postgresintl.com www.postgresintl.com
[HACKERS] building on windows using VC 2008
Getting lots of POSTGRESQL_TRACE_... undefined. Any hints ? Dave Cramer
Re: [HACKERS] Packaging of postgresql-jdbc
On 16 February 2016 at 07:50, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 16 February 2016 at 20:15, Pavel Kajaba <pkaj...@redhat.com> wrote: > >> Hello pg-hackers, >> >> I need advice about postgresql-jdbc driver. >> >> Current version in Fedora is behind latest version of postgresql-jdbc >> (1200 vs 1207). >> >> We are trying to package latest version into Fedora, but there are >> dependencies, which are not useless in Fedora (waffle-jna) > > > Which *are* useless in Fedora. I know that was just an editing mistake. > It's a library used in PgJDBC for windows SSPI support. > > I don't really see the problem here. If your packaging policy prevents you > from incorporating it, patch it out. It's use is simple, self-contained and > already optional. > > >> and ones which we are not 100% open source (osgi-enterprise). We talked >> with upstream quite intensively but not been able to find any solution >> which would meet our requirements. >> > > ... which you should probably outline here, because otherwise nobody will > understand the problem. > > >> We think that it's not a good, when open-source project depending on >> packages, which licence is not 100% clear. >> > > Well, frankly, that's Java. So long as they're soft-dependencies I really > don't care. > > I've already explained the JDBC position here. There is an impedance mismatch between the java ecosystem and distros. We have moved to maven as have most other java projects. As Craig said, if you want to build it, patch it out, and create a ant/Makefile to make the jar. Dave Cramer da...@postgresintl.com www.postgresintl.com
[HACKERS] Fwd: [JDBC] Re: 9.4-1207 behaves differently with server side prepared statements compared to 9.2-1102
We have an interesting problem, and the reporter has been kind enough to provide logs for which we can't explain. I'd be interested to hear any plausible explanations for a prepared plan suddenly going from 2ms to 60ms for the same input values ? Dave Cramer da...@postgresintl.com www.postgresintl.com -- Forwarded message -- From: Thomas Kellerer <spam_ea...@gmx.net> Date: 12 January 2016 at 04:03 Subject: [JDBC] Re: 9.4-1207 behaves differently with server side prepared statements compared to 9.2-1102 To: pgsql-j...@postgresql.org > Is it possible to get server logs ? I have picked out one of the statements that suffered from this and sanitized the logfile. http://sql-workbench.net/pg_jdbc_94.log The complete statement is at the top of the file and in the messages themselves, I have replaced each occurrence of the statement with "select " The interesting thing (at least for me) is that the first few executions of the server side statement have pretty much the same runtime as the ones before the prepare. And then suddenly the runtime shoots through the rough going up from 1ms to 40ms or even 60ms Regards Thomas -- Sent via pgsql-jdbc mailing list (pgsql-j...@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-jdbc
Re: [HACKERS] Fwd: [JDBC] Re: 9.4-1207 behaves differently with server side prepared statements compared to 9.2-1102
Hi Marko, Interesting so why would it choose a worse plan at that point ? Why would it change at all if the current plan is working well ? Dave Cramer da...@postgresintl.com www.postgresintl.com On 12 January 2016 at 07:15, Marko Tiikkaja <ma...@joh.to> wrote: > On 12/01/16 13:00, Dave Cramer wrote: > >> We have an interesting problem, and the reporter has been kind enough to >> provide logs for which we can't explain. >> >> I'd be interested to hear any plausible explanations for a prepared plan >> suddenly going from 2ms to 60ms for the same input values ? >> > > This is a new feature in 9.2, where on the fifth (or sixth, not sure) > execution the planner might choose to use a generic plan. From the 9.2 > release notes (though I'm fairly certain this is documented somewhere in > the manual as well): > > In the past, a prepared statement always had a single "generic" plan that > was used for all parameter values, which was frequently much inferior to > the plans used for non-prepared statements containing explicit constant > values. Now, the planner attempts to generate custom plans for specific > parameter values. A generic plan will only be used after custom plans have > repeatedly proven to provide no benefit. This change should eliminate the > performance penalties formerly seen from use of prepared statements > (including non-dynamic statements in PL/pgSQL). > > > .m >
Re: [HACKERS] JDBC driver debug out?
Add DriverManager.setLogWriter(new PrintWriter(System.out)); below the setLogLevel and it will I'll fix this though Dave Cramer da...@postgresintl.com www.postgresintl.com On 23 October 2015 at 22:35, Tatsuo Ishii <is...@postgresql.org> wrote: > Unfortunately it doesn't work (no debug trace). > > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp > > > This should work better > > > > > https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc41-SNAPSHOT/ > > > > Dave Cramer > > > > da...@postgresintl.com > > www.postgresintl.com > > > > On 23 October 2015 at 21:32, Dave Cramer <p...@fastcrypt.com> wrote: > > > >> No, I need to provide you with a 41 version. > >> > >> I just happened to have java 1.8 on my machine. > >> > >> Dave Cramer > >> > >> da...@postgresintl.com > >> www.postgresintl.com > >> > >> On 23 October 2015 at 21:31, Tatsuo Ishii <is...@postgresql.org> wrote: > >> > >>> Dave, > >>> > >>> Thanks for the quick response. Unfortunately now I'm getting error > >>> with the JDBC driver. > >>> > >>> warning: > >>> > /usr/local/pgsql/share/postgresql-9.4-1204-jdbc42-20151023.230759-1.jar(org/postgresql/Driver.class): > >>> major version 52 is newer than 51, the highest major version supported > by > >>> this compiler. > >>> It is recommended that the compiler be upgraded. > >>> 1 warning > >>> [snip] > >>> Exception in thread "main" java.lang.UnsupportedClassVersionError: > >>> org/postgresql/Driver : Unsupported major.minor version 52.0 > >>> at java.lang.ClassLoader.defineClass1(Native Method) > >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > >>> at > >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > >>> at java.security.AccessController.doPrivileged(Native Method) > >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > >>> at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > >>> at java.lang.Class.forName0(Native Method) > >>> at java.lang.Class.forName(Class.java:191) > >>> at begin_select_sleep.dbAccess(begin_select_sleep.java:39) > >>> at begin_select_sleep.main(begin_select_sleep.java:21) > >>> > >>> $ java -version > >>> java version "1.7.0_79" > >>> OpenJDK Runtime Environment (IcedTea 2.5.6) > (7u79-2.5.6-0ubuntu1.14.04.1) > >>> OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode) > >>> > >>> Probably I should upgrade Java... > >>> > >>> Best regards, > >>> -- > >>> Tatsuo Ishii > >>> SRA OSS, Inc. Japan > >>> English: http://www.sraoss.co.jp/index_en.php > >>> Japanese:http://www.sraoss.co.jp > >>> > >>> > Tatsuo, > >>> > > >>> > Can you confirm it is fixed in this snapshot > >>> > > >>> > https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc42-SNAPSHOT/ > >>> > > >>> > Dave Cramer > >>> > > >>> > da...@postgresintl.com > >>> > www.postgresintl.com > >>> > > >>> > On 23 October 2015 at 19:00, Dave Cramer <p...@fastcrypt.com> wrote: > >>> > > >>> >> Tatsuo, > >>> >> > >>> >> posting to jdbc list > >>> >> > >>> >> Dave Cramer > >>> >> > >>> >> da...@postgresintl.com > >>> >> www.postgresintl.com > >>> >> > >>> >> On 23 October 2015 at 18:28, Tats
Re: [HACKERS] JDBC driver debug out?
No, I need to provide you with a 41 version. I just happened to have java 1.8 on my machine. Dave Cramer da...@postgresintl.com www.postgresintl.com On 23 October 2015 at 21:31, Tatsuo Ishii <is...@postgresql.org> wrote: > Dave, > > Thanks for the quick response. Unfortunately now I'm getting error > with the JDBC driver. > > warning: > /usr/local/pgsql/share/postgresql-9.4-1204-jdbc42-20151023.230759-1.jar(org/postgresql/Driver.class): > major version 52 is newer than 51, the highest major version supported by > this compiler. > It is recommended that the compiler be upgraded. > 1 warning > [snip] > Exception in thread "main" java.lang.UnsupportedClassVersionError: > org/postgresql/Driver : Unsupported major.minor version 52.0 > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:191) > at begin_select_sleep.dbAccess(begin_select_sleep.java:39) > at begin_select_sleep.main(begin_select_sleep.java:21) > > $ java -version > java version "1.7.0_79" > OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1) > OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode) > > Probably I should upgrade Java... > > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp > > > Tatsuo, > > > > Can you confirm it is fixed in this snapshot > > > https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc42-SNAPSHOT/ > > > > Dave Cramer > > > > da...@postgresintl.com > > www.postgresintl.com > > > > On 23 October 2015 at 19:00, Dave Cramer <p...@fastcrypt.com> wrote: > > > >> Tatsuo, > >> > >> posting to jdbc list > >> > >> Dave Cramer > >> > >> da...@postgresintl.com > >> www.postgresintl.com > >> > >> On 23 October 2015 at 18:28, Tatsuo Ishii <is...@postgresql.org> wrote: > >> > >>> It seems > >>> org.postgresql.Driver.setLogLevel(org.postgresql.Driver.DEBUG) does > >>> not work anymore in the newer JDBC driver. > >>> > >>> As far as I know, postgresql-9.2-1003.jdbc4.jar or > >>> postgresql-9.3-1104.jdbc41.jar work fine and produce following output > >>> for example: > >>> > >>> 16:36:36.459 (1) PostgreSQL 9.3 JDBC4.1 (build 1104) > >>> 16:36:36.464 (1) Trying to establish a protocol version 3 connection to > >>> localhost:11000 > >>> 16:36:36.481 (1) Receive Buffer Size is 530904 > >>> 16:36:36.481 (1) Send Buffer Size is 1313280 > >>> 16:36:36.481 (1) FE=> StartupPacket(user=t-ishii, database=test, > >>> client_encoding=UTF8, DateStyle=ISO, TimeZone=Asia/Tokyo, > >>> extra_float_digits=2) > >>> 16:36:36.487 (1)<=BE AuthenticationOk > >>> > >>> However postgresql-9.4-1204.jdbc41.jar shows nothing. > >>> > >>> Best regards, > >>> -- > >>> Tatsuo Ishii > >>> SRA OSS, Inc. Japan > >>> English: http://www.sraoss.co.jp/index_en.php > >>> Japanese:http://www.sraoss.co.jp > >>> > >>> > >>> -- > >>> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > >>> To make changes to your subscription: > >>> http://www.postgresql.org/mailpref/pgsql-hackers > >>> > >> > >> >
Re: [HACKERS] JDBC driver debug out?
This should work better https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc41-SNAPSHOT/ Dave Cramer da...@postgresintl.com www.postgresintl.com On 23 October 2015 at 21:32, Dave Cramer <p...@fastcrypt.com> wrote: > No, I need to provide you with a 41 version. > > I just happened to have java 1.8 on my machine. > > Dave Cramer > > da...@postgresintl.com > www.postgresintl.com > > On 23 October 2015 at 21:31, Tatsuo Ishii <is...@postgresql.org> wrote: > >> Dave, >> >> Thanks for the quick response. Unfortunately now I'm getting error >> with the JDBC driver. >> >> warning: >> /usr/local/pgsql/share/postgresql-9.4-1204-jdbc42-20151023.230759-1.jar(org/postgresql/Driver.class): >> major version 52 is newer than 51, the highest major version supported by >> this compiler. >> It is recommended that the compiler be upgraded. >> 1 warning >> [snip] >> Exception in thread "main" java.lang.UnsupportedClassVersionError: >> org/postgresql/Driver : Unsupported major.minor version 52.0 >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:191) >> at begin_select_sleep.dbAccess(begin_select_sleep.java:39) >> at begin_select_sleep.main(begin_select_sleep.java:21) >> >> $ java -version >> java version "1.7.0_79" >> OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1) >> OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode) >> >> Probably I should upgrade Java... >> >> Best regards, >> -- >> Tatsuo Ishii >> SRA OSS, Inc. Japan >> English: http://www.sraoss.co.jp/index_en.php >> Japanese:http://www.sraoss.co.jp >> >> > Tatsuo, >> > >> > Can you confirm it is fixed in this snapshot >> > >> https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc42-SNAPSHOT/ >> > >> > Dave Cramer >> > >> > da...@postgresintl.com >> > www.postgresintl.com >> > >> > On 23 October 2015 at 19:00, Dave Cramer <p...@fastcrypt.com> wrote: >> > >> >> Tatsuo, >> >> >> >> posting to jdbc list >> >> >> >> Dave Cramer >> >> >> >> da...@postgresintl.com >> >> www.postgresintl.com >> >> >> >> On 23 October 2015 at 18:28, Tatsuo Ishii <is...@postgresql.org> >> wrote: >> >> >> >>> It seems >> >>> org.postgresql.Driver.setLogLevel(org.postgresql.Driver.DEBUG) does >> >>> not work anymore in the newer JDBC driver. >> >>> >> >>> As far as I know, postgresql-9.2-1003.jdbc4.jar or >> >>> postgresql-9.3-1104.jdbc41.jar work fine and produce following output >> >>> for example: >> >>> >> >>> 16:36:36.459 (1) PostgreSQL 9.3 JDBC4.1 (build 1104) >> >>> 16:36:36.464 (1) Trying to establish a protocol version 3 connection >> to >> >>> localhost:11000 >> >>> 16:36:36.481 (1) Receive Buffer Size is 530904 >> >>> 16:36:36.481 (1) Send Buffer Size is 1313280 >> >>> 16:36:36.481 (1) FE=> StartupPacket(user=t-ishii, database=test, >> >>> client_encoding=UTF8, DateStyle=ISO, TimeZone=Asia/Tokyo, >> >>> extra_float_digits=2) >> >>> 16:36:36.487 (1)<=BE AuthenticationOk >> >>> >> >>> However postgresql-9.4-1204.jdbc41.jar shows nothing. >> >>> >> >>> Best regards, >> >>> -- >> >>> Tatsuo Ishii >> >>> SRA OSS, Inc. Japan >> >>> English: http://www.sraoss.co.jp/index_en.php >> >>> Japanese:http://www.sraoss.co.jp >> >>> >> >>> >> >>> -- >> >>> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> >>> To make changes to your subscription: >> >>> http://www.postgresql.org/mailpref/pgsql-hackers >> >>> >> >> >> >> >> > >
Re: [HACKERS] JDBC driver debug out?
Tatsuo, Can you confirm it is fixed in this snapshot https://oss.sonatype.org/content/repositories/snapshots/org/postgresql/postgresql/9.4-1204-jdbc42-SNAPSHOT/ Dave Cramer da...@postgresintl.com www.postgresintl.com On 23 October 2015 at 19:00, Dave Cramer <p...@fastcrypt.com> wrote: > Tatsuo, > > posting to jdbc list > > Dave Cramer > > da...@postgresintl.com > www.postgresintl.com > > On 23 October 2015 at 18:28, Tatsuo Ishii <is...@postgresql.org> wrote: > >> It seems >> org.postgresql.Driver.setLogLevel(org.postgresql.Driver.DEBUG) does >> not work anymore in the newer JDBC driver. >> >> As far as I know, postgresql-9.2-1003.jdbc4.jar or >> postgresql-9.3-1104.jdbc41.jar work fine and produce following output >> for example: >> >> 16:36:36.459 (1) PostgreSQL 9.3 JDBC4.1 (build 1104) >> 16:36:36.464 (1) Trying to establish a protocol version 3 connection to >> localhost:11000 >> 16:36:36.481 (1) Receive Buffer Size is 530904 >> 16:36:36.481 (1) Send Buffer Size is 1313280 >> 16:36:36.481 (1) FE=> StartupPacket(user=t-ishii, database=test, >> client_encoding=UTF8, DateStyle=ISO, TimeZone=Asia/Tokyo, >> extra_float_digits=2) >> 16:36:36.487 (1)<=BE AuthenticationOk >> >> However postgresql-9.4-1204.jdbc41.jar shows nothing. >> >> Best regards, >> -- >> Tatsuo Ishii >> SRA OSS, Inc. Japan >> English: http://www.sraoss.co.jp/index_en.php >> Japanese:http://www.sraoss.co.jp >> >> >> -- >> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-hackers >> > >
Re: [HACKERS] JDBC driver debug out?
Tatsuo, posting to jdbc list Dave Cramer da...@postgresintl.com www.postgresintl.com On 23 October 2015 at 18:28, Tatsuo Ishii <is...@postgresql.org> wrote: > It seems > org.postgresql.Driver.setLogLevel(org.postgresql.Driver.DEBUG) does > not work anymore in the newer JDBC driver. > > As far as I know, postgresql-9.2-1003.jdbc4.jar or > postgresql-9.3-1104.jdbc41.jar work fine and produce following output > for example: > > 16:36:36.459 (1) PostgreSQL 9.3 JDBC4.1 (build 1104) > 16:36:36.464 (1) Trying to establish a protocol version 3 connection to > localhost:11000 > 16:36:36.481 (1) Receive Buffer Size is 530904 > 16:36:36.481 (1) Send Buffer Size is 1313280 > 16:36:36.481 (1) FE=> StartupPacket(user=t-ishii, database=test, > client_encoding=UTF8, DateStyle=ISO, TimeZone=Asia/Tokyo, > extra_float_digits=2) > 16:36:36.487 (1)<=BE AuthenticationOk > > However postgresql-9.4-1204.jdbc41.jar shows nothing. > > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 19 May 2015 at 19:18, Jan de Visser j...@de-visser.net wrote: On May 19, 2015 09:31:32 PM Greg Sabino Mullane wrote: Jan de Visser wrote: Well, one could argue that it *is* their problem, as they should be using the standard Postgres way for placeholders, which is $1, $2, $3... Shirley you are joking: Many products use JDBC as an abstraction layer facilitating (mostly) seamless switching between databases. I know the product I worked on did. Are you advocating that every single statement should use SELECT * FROM foo WHERE bar = $1 on pg and SELECT * FROM foo WHERE bar = ? on every other database? I'm not joking, and don't call me Shirley. If you are running into situations where you have question mark operators in your queries, you have already lost the query abstraction battle. There will be no seamless switching if you are using jsonb, hstore, ltree, etc. My statement was more about pointing out that Postgres already offers a complete placeholder system, which drivers are free to implement if they want. I must have misunderstood you strikeShirley/strike Greg, because to me it parsed as if you were suggesting (paraphrasing) ah forget about those pesky standardized drivers and their pesky syntax requirements. Just use ours like a big boy. I understand that once you start using '?' as (part of) operator names in your queries you're not portable anymore. I just thought that your proposed solution was to throw all portability out the window. But I was probably (hopefully?) wrong. jan Using anything other than ? in JDBC is a non-starter you might as well just stop supporting java entirely. Back to the issue at hand. Does anyone have a recommendation for a replacement operator besides ? When I first noticed this one thought was to create duplicate operators specifically for the use of the JDBC driver. I had dismissed this at the time, now I'm not so sure Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 18 May 2015 at 18:49, David G. Johnston david.g.johns...@gmail.com wrote: On Mon, May 18, 2015 at 3:31 PM, Bruno Harbulot br...@distributedmatter.net wrote: On Sun, May 17, 2015 at 5:15 PM, Greg Sabino Mullane g...@turnstep.com wrote: In that case my vote is new operators. This has been a sore point for the JDBC driver Um, no, new operators is a bad idea. Question marks are used by hstore, json, geometry, and who knows what else. I think the onus is solely on JDBC to solve this problem. DBD::Pg solved it in 2008 with the pg_placeholder_dollaronly solution, and earlier this year by allowing backslashes before the question mark (because other parts of the stack were not able to smoothly implement pg_placeholder_dollaronly.) I recommend all drivers implement \? as a semi-standard workaround. See also: http://blog.endpoint.com/2015/01/dbdpg-escaping-placeholders-with.html I'm not sure the onus is solely on JDBC. Using question marks in operators clearly has required a number of connectors to implement their own workarounds, in different ways. This also seems to affect some libraries and frameworks that depend on those connectors (and for which the workarounds may even be more convoluted). My main point was that this is not specific to JDBC. Considering that even PostgreSQL's own ECPG is affected, the issue goes probably deeper than it seems. I'm just not convinced that passing the problem onto connectors, libraries and ultimately application developers is the right thing to do here. In the discussion on the OpenJDK JDBC list two years ago ( http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/50.html ), Lance Andersen said There is nothing in the SQL standard that would support the use of an '?' as anything but a parameter marker.. It might be worth finding out whether this is indeed the case according to the SQL specifications (I'm afraid I'm not familiar with these specifications to do it myself). CREATE OPERATOR is a PostgreSQL extension. There are no provisions for user-defined operators in the SQL standard. http://www.postgresql.org/docs/9.4/interactive/sql-createoperator.html And by extension if indeed the standard does require the use of ? for parameters we are in violation there because the backend protocol deals with $# placeholders and not ? I too do not know enough here. Note that it would not be enough to change the existing operators - any use of ? would have to be forbidden including those created by users. The first step on this path would be for someone to propose a patch adding alternative operators for every existing operator that uses ?. If this idea is to move forward at all that patch would have to be accepted. Such a patch is likely to see considerable bike-shedding. We then at least provide an official way to avoid ? operators that shops can make use of at their discretion. Removing the existing operators or forbidding custom operators is a separate discussion. David J. It would seem that choosing ? for operators was ill advised; I'm not convinced that deprecating them is a bad idea. If we start now, in 5 years they should be all but gone Agreed a patch would be the first place to start Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 19 May 2015 at 10:23, Kevin Grittner kgri...@ymail.com wrote:
David G. Johnston david.g.johns...@gmail.com wrote:
On Mon, May 18, 2015 at 3:31 PM, Bruno Harbulot
br...@distributedmatter.netwrote:
In the discussion on the OpenJDK JDBC list two years ago
(
http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/50.html
),
Lance Andersen said There is nothing in the SQL standard that
would support the use of an '?' as anything but a parameter
marker..
CREATE OPERATOR is a PostgreSQL extension. There are no
provisions for user-defined operators in the SQL standard.
Exactly. The standard specifies the characters to use for the
predicates that it defines, and provides no mechanism for adding
additional predicates; but who in the world would want to exclude
all extensions to the standard?
And by extension if indeed the standard does require the use of
? for parameters we are in violation there because the backend
protocol deals with $# placeholders and not ?
We're talking about a different specification that has question
marks as parameter placeholders. That's in the Java Database
Connector (JDBC) specification. (It is apparently also specified
in other documents, although I'm not familiar enough with those to
comment.) Note that it would create all sorts of pain if both the
SQL statements and a connector issuing them used the same
convention for substituting parameters; it is a *good* thing that
plpgsql and SQL function definitions use a different convention
than JDBC!
The JDBC spec provides for escapes using curly braces (including
product-specific escapes); it seems like a big mistake for us to
have chosen a completely different mechanism for escaping the
question mark character in a SQL statement. Perhaps the least
painful path would be to add support for {?} as the escape for a
question mark, and a connection option to supplement that with
support for the legacy \? escape. I would bet a lot of money that
even with an if test for that option, the curly brace escape
would be faster than what's there now (when the option was not
set). Some operators would look a little funny in Java string
literals, but that's not so bad.
Perhaps reviewing https://github.com/pgjdbc/pgjdbc/pull/187 might help
understand why we chose ??
Dave Cramer
dave.cramer(at)credativ(dot)ca
http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On 19 May 2015 at 13:15, Mike Blackwell mike.blackw...@rrd.com wrote: A Google search suggests Oracle 9.x supports a unary '?' operator (fuzzy match), so the use of '?' in an operator name is not without precedent. Interesting argument. There is considerable precedent where we take the position that just because xyz supports it we don't. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 19 May 2015 at 16:36, Kevin Grittner kgri...@ymail.com wrote: Gavin Flower gavinflo...@archidevsys.co.nz wrote: I prefer the $1 approach, others can't use that, and there are situations where I could not either. So, how about defaulting to the '?' approach, but have a method to explicitly set the mode - to switch to using '$'? Are you suggesting that we implement something other than what is described in these documents for prepared statement parameters?: http://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html http://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf If so, I strongly oppose that. If we are not going to deprecate use of the question mark character for operators, we need some nonstandard hack to our JDBC implementation, but an alternative syntax for specifying PreparedStatement and CallableStatement parameters seems entirely the wrong way to go. The issue here is what to do about the difficulties in using JDBC prepared statements in combination with the PostgreSQL extension of operator names containing question marks. Using a double question mark is not horrible as a solution. Actually the issue is what to do about a number of connectors which use a fairly standard '?' as a placeholder. Notably absent from the discussion is ODBC upon which JDBC was modelled and probably predates any use of ? as an operator It may not be what we would have arrived at had the discussion taken place on the pgsql-jdbc list rather than underneath a github pull request, but we can only move forward from where we are. possibly, however all of the current JDBC maintainers opined and reached an agreement on this. Out of curiosity, how long has the ?? solution been implemented in a driver jar file available as a public download? At least since February of this year What are the guidelines for what discussion belongs on the pgsql-jdbc list and what discussion belongs on github? Is someone interested in participating in the discussions leading to decisions about our JDBC connector expected to follow both? Currently pull requests are the easiest to deal with so most discussion is on github. I guess updating the JDBC web page would be in order. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 19 May 2015 at 15:02, Tom Lane t...@sss.pgh.pa.us wrote: Greg Sabino Mullane g...@turnstep.com writes: Dave Cramer opined: It would seem that choosing ? for operators was ill advised; I'm not convinced that deprecating them is a bad idea. If we start now, in 5 years they should be all but gone Ha ha ha ha ha! That's a good one. We still have clients on Postgres 7! Five years is way too short to replace something that major. Yeah, that's a big problem for this line of thought. Even if we had consensus today, the first release that would actually contain alternative operators would be 9.6, more than a year out (since 9.5 is past feature freeze now). It would take several years after that before there would be any prospect of removing the old ones, and several years more before PG versions containing the old operators were out of support. Now there are different ways you could look at this. From the perspective of a particular end user, you could imagine instituting a shop policy of not using the operators containing '?' as soon as you had a release where there were alternatives. So in that context you might have a fix available as soon as 9.6 came out. But from the perspective of a driver author who has to support queries written by other people, the problem would not be gone for at least ten years more. Changing the driver's behavior sounds like a more practical solution. The current JDBC driver doesn't really support anything beyond 8.4 except for CRUD operations. We are also are no longer supporting JVM's older than 1.6 in the current driver. People who insist on staying on old code get what they get. I don't see a problem with saying after a certain date we just don't support it in the current code. After all I have heard rumblings about deprecating V2 protocol ? FWIW, I was content to leave this alone. JDBC has a workable solution. However I've not seen a good argument for continuing to use the ? operator as it's conflicts with many clients and is apparently not in the SQL standard. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
Not sure what the point of this is: as you indicated the ship has sailed so
to speak
Dave Cramer
dave.cramer(at)credativ(dot)ca
http://www.credativ.ca
On 15 May 2015 at 15:14, Bruno Harbulot br...@distributedmatter.net wrote:
Hello,
I've been trying to use the new JSONB format using JDBC, and ran into
trouble with the question mark operators (?, ?| and ?).
I realise there has already been a discussion about this (actually, it was
about hstore, not jsonb, but that's more or less the same problem):
- http://www.postgresql.org/message-id/51114165.4070...@abshere.net
-
http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/48.html
From what I gather, the JDBC team seems to think that using ? in operators
is not in line with the SQL standards, but the outcome on the PostgreSQL
list team suggested that a fix could be implemented in the PostgreSQL JDBC
driver anyway.
I think this problem might actually affect a number of other places,
unfortunately. I must admit I don't know the SQL specifications very well
(a quick look at a draft seemed to suggest the question mark was indeed a
reserved character, but this is probably out of context), and this isn't
about finding out who is right or who is wrong, but from a practical point
of view, this also seemed to affect other kinds of clients, for example:
- Perl:
http://blog.endpoint.com/2015/01/dbdpg-escaping-placeholders-with.html
- JavaScript: https://github.com/tgriesser/knex/issues/519
Of course, there can be workarounds in some cases, but even if they work,
they can be quite awkward, especially if they differ from one language to
another (in particular if you want to be able to re-use the same query from
multiple languages).
As far, as I can tell, question mark operators are also incompatible with
PostgreSQL's ECPG when using dynamic SQL.
http://www.postgresql.org/docs/current/static/ecpg-dynamic.html
(I'm pasting an example at the end of this message, tried with a
PostgreSQL 9.4 server.)
I realise it's a bit late to raise this concern, considering that these
operators have been around for a few versions now (at least as far as
hstore), but wouldn't it be better to provide official alternative
notations altogether, something that is less likely to conflict with most
client implementations? Perhaps a function or a notation similar to what
'CAST(x AS y)' is to 'x::y' would be suitable if other symbols aren't
better (although I think a short operator would still be preferable).
Best wishes,
Bruno.
ECPG test output:
** Using query: SELECT ('{key1:123,key2:Hello}'::jsonb -
?::text)::text
Result should be 123 for 'key1': 123
Result should be empty for 'key3':
** Using query: SELECT ('{key1:123,key2:Hello}'::jsonb ?
?::text)::text
SQL error: syntax error at or near $1 on line 52
SQL error: invalid statement name mystmt3 on line 55
Result should be true for 'key1':
SQL error: invalid statement name mystmt3 on line 59
Result should be false for 'key3':
SQL error: invalid statement name mystmt3 on line 62
ECPG test code:
#include stdio.h
#include stdlib.h
int main()
{
EXEC SQL BEGIN DECLARE SECTION;
char* target = unix:postgresql://localhost/mydatabase;
char result1[2048];
int result1_ind;
char *key1_str = key1;
char *key3_str = key3;
char *stmt2 = SELECT ('{\key1\:123,\key2\:\Hello\}'::jsonb
- ?::text)::text;
char *stmt3 = SELECT ('{\key1\:123,\key2\:\Hello\}'::jsonb
? ?::text)::text;
EXEC SQL END DECLARE SECTION;
EXEC SQL WHENEVER SQLWARNING SQLPRINT;
EXEC SQL WHENEVER SQLERROR SQLPRINT;
EXEC SQL CONNECT TO :target AS testdb;
printf(\n\n** Using query: %s\n\n, stmt2);
EXEC SQL PREPARE mystmt2 FROM :stmt2;
result1[0] = 0;
EXEC SQL EXECUTE mystmt2 INTO :result1 :result1_ind USING :key1_str;
printf(Result should be 123 for 'key1': %s\n, result1);
result1[0] = 0;
EXEC SQL EXECUTE mystmt2 INTO :result1 :result1_ind USING :key3_str;
printf(Result should be empty for 'key3': %s\n, result1);
EXEC SQL DEALLOCATE PREPARE mystmt2;
printf(\n\n** Using query: %s\n\n, stmt3);
EXEC SQL PREPARE mystmt3 FROM :stmt3;
result1[0] = 0;
EXEC SQL EXECUTE mystmt3 INTO :result1_ind USING :key1_str;
printf(Result should be true for 'key1': %s\n, result1);
result1[0] = 0;
EXEC SQL EXECUTE mystmt3 INTO :result1_ind USING :key3_str;
printf(Result should be false for 'key3': %s\n, result1);
EXEC SQL DEALLOCATE PREPARE mystmt3;
EXEC SQL DISCONNECT ALL;
return 0;
}
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 15 May 2015 at 16:41, Robert Haas robertmh...@gmail.com wrote: On Fri, May 15, 2015 at 4:38 PM, Dave Cramer p...@fastcrypt.com wrote: I don't really want to take a violently strong position on this without understanding what's really going on here. Well our solution was to use ?? but that does mean we have to do some extra parsing which in a perfect world wouldn't be necessary. So what about strings quoted with '' or $$ or $something$ - how would you handle those? We parse for strings; the ?? just adds to the parsing load which we really try to avoid. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 15 May 2015 at 16:21, Robert Haas robertmh...@gmail.com wrote: On Fri, May 15, 2015 at 4:13 PM, Dave Cramer p...@fastcrypt.com wrote: Not sure what the point of this is: as you indicated the ship has sailed so to speak Well, if we were to agree this was a problem, we could introduce new, less-problematic operator names and then eventually deprecate the old ones. Personally, it wouldn't take a lot to convince me that if a certain set of operator names is problematic for important connectors, we should avoid using those and switch to other ones. I expect others on this mailing list to insist that if the connectors don't work, that's the connector drivers fault for coding their connectors wrong. And maybe that's the right answer, but on the other hand, maybe it's a little myopic. I think the discussion is worth having. In that case my vote is new operators. This has been a sore point for the JDBC driver Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 15 May 2015 at 16:35, Robert Haas robertmh...@gmail.com wrote:
On Fri, May 15, 2015 at 4:23 PM, Dave Cramer p...@fastcrypt.com wrote:
Well, if we were to agree this was a problem, we could introduce new,
less-problematic operator names and then eventually deprecate the old
ones. Personally, it wouldn't take a lot to convince me that if a
certain set of operator names is problematic for important connectors,
we should avoid using those and switch to other ones. I expect others
on this mailing list to insist that if the connectors don't work,
that's the connector drivers fault for coding their connectors wrong.
And maybe that's the right answer, but on the other hand, maybe it's a
little myopic. I think the discussion is worth having.
In that case my vote is new operators. This has been a sore point for the
JDBC driver
I guess JDBC has the same problem as Perl and JavaScript here: ?
signals a bind variable. The next question is, why isn't there some
escaping mechanism for that, like writing ?? or \? or something?
I ask because, you know, suppose you write this:
INSERT INTO foo VALUES ('How many pickled peppers did Peter Piper pick?');
Or alternatively this:
INSERT INTO foo VALUES ($$If Peter piper picked a peck of pickled
peppers, where's the peck of pickled peppers Peter Piper picked?$$);
Those have also got question marks in them. Do they also get
interpreted as bind variables?
I don't really want to take a violently strong position on this
without understanding what's really going on here.
Well our solution was to use ?? but that does mean we have to do some
extra parsing which in a perfect world wouldn't be necessary.
Dave Cramer
dave.cramer(at)credativ(dot)ca
http://www.credativ.ca
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
On 15 May 2015 at 16:44, Dave Cramer p...@fastcrypt.com wrote: On 15 May 2015 at 16:41, Robert Haas robertmh...@gmail.com wrote: On Fri, May 15, 2015 at 4:38 PM, Dave Cramer p...@fastcrypt.com wrote: I don't really want to take a violently strong position on this without understanding what's really going on here. Well our solution was to use ?? but that does mean we have to do some extra parsing which in a perfect world wouldn't be necessary. So what about strings quoted with '' or $$ or $something$ - how would you handle those? We parse for strings; the ?? just adds to the parsing load which we really try to avoid. The ?? is just harder to deal with because ? is part of the JDBC spec as a placeholder Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] orangutan seizes up during isolation-check
On 16 January 2015 at 01:33, Noah Misch n...@leadboat.com wrote: On Thu, Jan 15, 2015 at 09:24:01AM -0500, Robert Haas wrote: On Thu, Jan 15, 2015 at 1:04 AM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 14, 2015 at 04:48:53PM -0500, Peter Eisentraut wrote: What I'm seeing now is that the unaccent regression tests when run under make check-world abort with FATAL: postmaster became multithreaded during startup HINT: Set the LC_ALL environment variable to a valid locale. contrib/unaccent/Makefile sets NO_LOCALE=1, so that makes sense. I expect the patch over here will fix it: http://www.postgresql.org/message-id/20150109063015.ga2491...@tornado.leadboat.com I just hit this same problem; are you going to commit that patch soon? It's rather annoying to have make check-world fail. Sure, done. Dave, orangutan should now be able to pass with --enable-nls. Would you restore that option? I can, but is this for HEAD or all versions ?
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
Since we already do escape processing much like c that might not be so
complex. However I haven't looked at the code, so I could be way off base.
The question I would pose is how palatable is it to use ? In other words is
it worth pursuing ?
Dave Cramer
dave.cramer(at)credativ(dot)ca
http://www.credativ.ca
On Wed, Feb 13, 2013 at 12:10 PM, Seamus Abshere sea...@abshere.net wrote:
On 2/11/13 1:35 PM, Heikki Linnakangas wrote:
I agree it's pretty dumb that there's currently no such escape. I think
JDBC inherited that design mistake from ODBC. Fixing that would be a good
idea.
Lance Anderson, Oracle's JDBC spec lead, says [1] we can implement
something like:
SELECT * FROM tbl WHERE data {postgres qm} 'abc'
Thanks to Mark Rotteveel for driving the discussion. [2]
That said, it occurs to me that there's one fairly simple thing we could
also do in the backend. At the moment, unlike function and table names,
operators cannot be quoted. It would be easy to allow this in the grammar:
select 1 operator(+) 1;
I guess I see 2 simple options and 1 complex option:
a) [simple] operator(+) per Heikki
b) [simple, but not popular] alias ? to ~ per Seamus
c) [complex] {postgres blah} per Lance/Mark
How to decide?
Best,
Seamus
[1] http://mail.openjdk.java.net/**pipermail/jdbc-spec-discuss/**
2013-February/58.htmlhttp://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/58.html
[2] http://mail.openjdk.java.net/**pipermail/jdbc-spec-discuss/**
2013-February/date.html#51http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/date.html#51(threaded
view gets it out of order)
--
Seamus Abshere
sea...@abshere.net
https://github.com/**seamusabshere https://github.com/seamusabshere
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
On Fri, Feb 8, 2013 at 11:43 AM, Kris Jurka bo...@ejurka.com wrote: On Wed, 6 Feb 2013, Seamus Abshere wrote: I personally don't know of any way around the conflict except changing JDBC or hstore, and I don't think JDBC is gonna change. I think changing JDBC is the way to go. Currently JDBC supports escape sequences for cross database portability and it seems reasonable to support an escape sequence that allowed passing ? to the backend instead of interpreting it as a parameter. This will be more complicated than you might hope because the escape processing currently happens prior to bind parameter detection so I'm not sure what a good patch would really look like, but given the feedback provided here, it's worth investigating. Kris Jurka Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca
Re: [HACKERS] Alias hstore's ? to ~ so that it works with JDBC
That would seem to be the implication. JDBC wouldn't really know anything about hstore. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Fri, Feb 8, 2013 at 12:51 PM, Andrew Dunstan and...@dunslane.net wrote: On 02/08/2013 12:41 PM, Kris Jurka wrote: On Fri, 8 Feb 2013, Dave Cramer wrote: Would this be an postgresql specific escape sequence ? I don't think the spec allows for this does it ? Yes, this would be a postgresql jdbc driver specific escape. The spec doesn't have a concept of private escape sequences, but that doesn't seem like the end of the world. Clearly the user here is writing postgresql specific code to use hstore operators, so there's not a portability loss here. I assume, though, that you're not talking about something that's hstore-specific, but rather something that will allow the user to put a non-parameter question mark in the query string. As has been noted upthread, the hstore use is far from the only one that causes users to trip on this. cheers andrew
Re: [HACKERS] [JDBC] Regarding GSoc Application
On Tue, Apr 10, 2012 at 11:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: ... We have to invoke java and there are two basic ways to tie into the java runtime: one is to jump through SPI via the SQL executor. The other is JNI into the pl/java jvm which I think you were hinting was the better approach. Hm? SPI doesn't know anything about Java either. regards, tom lane Having pl/java as a dependancy here makes this a very complex feature to setup. The potential benefits are quite minimal since almost any decent ETL tool can handle multiple data sources Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
How will the user access this? Will it be a normal query through the existing API ? Will it be a private postgresql API ? How will they set it up ? It appears complicated as you have to setup PL/Java as well Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Mon, Apr 9, 2012 at 11:45 AM, Merlin Moncure mmonc...@gmail.com wrote: On Sun, Apr 8, 2012 at 8:56 AM, Dave Cramer p...@fastcrypt.com wrote: Hi Atri, Is there some JDBC API that supports this in newer versions of the API ? Didn't parse that question. My understanding is that the only JDBC features needed are what's already there, to make connections to databases and execute queries. The GSoC proposal is here: https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2012/atrisharma/1001 merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
On Mon, Apr 9, 2012 at 11:55 AM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Apr 9, 2012 at 10:47 AM, Dave Cramer p...@fastcrypt.com wrote: How will the user access this? Will it be a normal query through the existing API ? Will it be a private postgresql API ? How will they set it up ? It appears complicated as you have to setup PL/Java as well Yeah -- it will run through pl/java (at least, that's the idea). What pl/java brings to the table is well thought out integration of the JVM to postgres so that you can invoke java as functions from postgres. PL/java of course is a heavy dependency and non-trivial to set up and install. But to access the jdbc from postgres I think it's the easiest way forward. Straight JNI to the JVM from FDW might be a better/cleaner route but we haven't done the research to see exactly what's involved there. I suspect that invoking java from postgres is non trivial any way you slice it and that's not a wheel worth re-inventing. In other words, the basic idea is to do two things: a dblink-ish wrapper for JDBC via pl/java and a FDW wrapper through that via SPI. Better ideas and criticism are welcome of course. merlin So I'm confused, once they link a file to an FDW can't you just read it with an normal select ? What additional functionality will this provide ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
On Mon, Apr 9, 2012 at 12:45 PM, Andrew Dunstan and...@dunslane.net wrote: On 04/09/2012 12:14 PM, Dave Cramer wrote: So I'm confused, once they link a file to an FDW can't you just read it with an normal select ? What additional functionality will this provide ? I'm confused about what you're confused about. Surely this won't be linking files to an FDW, but foreign DBMS tables, in anything you can access via JDBC. All you'll need on the postgres side is the relevant JDBC driver, so you'd have instant access via standard select queries to anything you can get a JDBC driver to talk to. That seems to me something worth having. I imagine it would look rather like this: CREATE FOREIGN DATA WRAPPER foodb HANDLER pljava_jdbc_handler OPTIONS (driver 'jdbc.foodb.org'); CREATE SERVER myfoodb FOREIGN DATA WRAPPER foodb OPTIONS(host '1.2.3.4', user 'foouser', password 'foopw'); CREATE FOREIGN TABLE footbl (id int, data text) SERVER myfoodb; SELECT * from footbl; Well this is certainly more explanation than we have so far. Is this the intended use case ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
Hi Atri, Is there some JDBC API that supports this in newer versions of the API ? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Sat, Apr 7, 2012 at 7:07 AM, Atri Sharma atri.j...@gmail.com wrote: Hi All, I submitted a GSoc application yesterday. Please review it and let me know if anyone needs any clarifications. Atri -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] who's familiar with the GSOC application process
Someone has approached the JDBC list to do some work on the driver as a GSOC project. I need to know what is involved in mentoring and how to get the project approved Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsphere
Hi Oleg, On Sun, Jan 8, 2012 at 1:19 PM, Oleg Bartunov o...@sai.msu.su wrote: Dave, The situation with pgshpere is so, that I think we need new developer, since Janko keep silence :) I wrote him several time, since I wanted pgsphere now could benefit very much from our KNNGiST feature. This is number one development from my point of view. I and Teodor have no time to work on pgsphere, sorry. But, there are some astronomers I'm working with, who can take part in this. Sergey Karpov has done extensive benchmarks of q3c, rtree and pgsphere and found the latter still has some benefits in some workload, so we are interesting in development. Regards, Oleg So where do we go from here ? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Fri, 6 Jan 2012, Andrew Dunstan wrote: On 01/06/2012 12:32 PM, Dave Cramer wrote: I've been asked by someone to support pgshpere. It would appear that the two project owners are MIA. If anyone knows different can they let me know ? Does anyone have any objection to me taking over the project? One of the owners is Teodor, who is a core committer ... I hope he's not MIA. cheers andrew Regards, Oleg _ Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru), Sternberg Astronomical Institute, Moscow University, Russia Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(495)939-16-83, +007(495)939-23-83 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsphere
Well I've sent Teodor a personal email asking him if he was interested and so far no response, so I interpret that as he no longer has interest in the project. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca On Fri, Jan 6, 2012 at 12:40 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/06/2012 12:32 PM, Dave Cramer wrote: I've been asked by someone to support pgshpere. It would appear that the two project owners are MIA. If anyone knows different can they let me know ? Does anyone have any objection to me taking over the project? One of the owners is Teodor, who is a core committer ... I hope he's not MIA. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgsphere
I've been asked by someone to support pgshpere. It would appear that the two project owners are MIA. If anyone knows different can they let me know ? Does anyone have any objection to me taking over the project? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgsphere
I've been asked by someone to support pgshpere. It would appear that the two project owners are MIA. If anyone knows different can they let me know ? Does anyone have any objection to me taking over the project? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Generating a query that never returns
I have a need to test timeouts in JDBC, is there a query that is guaranteed not to return ? Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] error building head on OS X 10.7.1
Get the following error configure:3274: ccache gcc -V 5 llvm-gcc-4.2: argument to `-V' is missing should be ccache gcc -v 5 Dave Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] error building head on OS X 10.7.1
On Wed, Sep 7, 2011 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Cramer p...@fastcrypt.com writes: Get the following error configure:3274: ccache gcc -V 5 llvm-gcc-4.2: argument to `-V' is missing should be ccache gcc -v 5 That's not an error, that's normal behavior. Mind you, I have no idea why autoconf chooses to do this when it's already tried -v, but this is not the source of whatever problem you're having. regards, tom lane Well the problem is that buildfarm can't build HEAD on OS X 10.7.1 Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] error building head on OS X 10.7.1
On Wed, Sep 7, 2011 at 11:32 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Dave Cramer p...@fastcrypt.com writes:
Well the problem is that buildfarm can't build HEAD on OS X 10.7.1
HEAD builds fine on my 10.7.1 laptop. If you're referring to orangutan,
it's not failing on that, it's failing here:
configure:3301: checking for C compiler default output file name
configure:3323: ccache gcc /opt/local/include conftest.c 5
ld: in /opt/local/include, can't map file, errno=22 for architecture x86_64
collect2: ld returned 1 exit status
which is probably because you have a malformed value of CFLAGS:
'config_env' = {
'CFLAGS' = '/opt/local/include',
regards, tom lane
Thanks Tom, that was it.
Dave Cramer
dave.cramer(at)credativ(dot)ca
http://www.credativ.ca
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?
On Thu, Aug 25, 2011 at 5:51 PM, Jim Nasby j...@nasby.net wrote: On Aug 23, 2011, at 1:57 PM, Peter Eisentraut wrote: Ultimately my ulterior motive is to make sure the information schema is actually useful and ensure that it maintains that status. The information schema only exposes privileges defined in the SQL standard on objects defined in the SQL standard. So psql or any tool that wants to provide information on PostgreSQL-specific features can't use that. But perhaps the JDBC interfaces only expose certain standard information anyway, so it could be useful. Depends on the details, though. Related to this is the newsysviews project, which was intended to present more human-friendly info ala info_schema, but to also provide PostgreSQL-specific information (which info_schema can't). I'll have to have a look at this when I get a moment, sounds promising. Will be even better if we get all clients using it. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Why doesn't psql use the information schema to get ACL description ?
Recently a bug was reported on the JDBC list regarding table privileges. It turns out that we have not been parsing the ACL tables correctly. This is partially due to having to track the catalogs through each version. So this breaks down into two questions 1) is it possible to get all of the ACL data from the information_schema ? 2) If so why doesn't psql use these tables ? Ultimately my ulterior motive is to make sure the information schema is actually useful and ensure that it maintains that status. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Support for cert auth in JDBC
Marc, Please just send a cvs context diff from HEAD to the JDBC list. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca 2011/5/17 Marc-André Laverdière marc-an...@atc.tcs.com: Hello developers, My project had a requirement to use certificate authentication to the PG server. Our application uses Hibernate. We did just that and my boss has OKed a source release. Now, the current version of the code has dependencies on our internal libraries, so I'll need to spend a bit of time making this 'standard' Java code. Would you please tell me how you'd prefer for me to proceed to do that? Do I need write access to your CVS repo, or should I just send the code and test case by email? Is there a specific version of the JDBC code you want me to work from, should I just pick whatever is HEAD? Any package you'd like me to choose? Any specific crypto/ssl requirements to consider? Any specific dependencies to use instead of others? (e.g. I like SLF4J, but that's not everyone's choice...) -- Marc-André Laverdière Software Security Scientist Innovation Labs, Tata Consultancy Services Hyderabad, India -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] [HACKERS] JDBC connections to 9.1
On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Bernd Helmle maili...@oopsware.de writes: If i am reading it correct, it reads UTF8 from the backend, while expecting UNICODE only. Not sure what change has caused this, though. I am --- when I redid the GUC assign_hook logic a few weeks ago, I changed the client_encoding code so that it shows the normalized (official) name of the encoding, not whatever random string the client sent over. For instance, previous versions: regression=# set client_encoding = 'UnIcOdE'; SET regression=# show client_encoding ; client_encoding - UnIcOdE (1 row) versus HEAD: regression=# set client_encoding = 'UnIcOdE'; SET regression=# show client_encoding ; client_encoding - UTF8 (1 row) I wasn't aware that JDBC would fail on that. It's pretty annoying that it does, but maybe we should grin and bear it, ie revert the change to canonicalize the GUC's value? regards, tom lane Older drivers will fail for sure. We can fix newer drivers, but if we leave it we will see a slew of bug reports. The reason the driver does this is to guarantee the client encoding is unicode. Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] [HACKERS] JDBC connections to 9.1
On Mon, Apr 18, 2011 at 11:14 AM, Mike Fowler m...@mlfowler.com wrote: On 18/04/11 15:57, Tom Lane wrote: Bernd Helmlemaili...@oopsware.de writes: If i am reading it correct, it reads UTF8 from the backend, while expecting UNICODE only. Not sure what change has caused this, though. I am --- when I redid the GUC assign_hook logic a few weeks ago, I changed the client_encoding code so that it shows the normalized (official) name of the encoding, not whatever random string the client sent over. For instance, previous versions: regression=# set client_encoding = 'UnIcOdE'; SET regression=# show client_encoding ; client_encoding - UnIcOdE (1 row) versus HEAD: regression=# set client_encoding = 'UnIcOdE'; SET regression=# show client_encoding ; client_encoding - UTF8 (1 row) I wasn't aware that JDBC would fail on that. It's pretty annoying that it does, but maybe we should grin and bear it, ie revert the change to canonicalize the GUC's value? regards, tom lane Am I right in thinking that would be that change committed on the 7th (http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ? I've just run the JDBC test build on my machine and it fails dismally with this very message repeated over and over again. What concerns me most is that (assuming my dates are right) the JDBC driver has been broken for 11 days and no one noticed. This would lead me to believe that there is no JDBC build server. What would it take to set one up? If someone can point me to a test machine I'd happily assist in setting one up. As for the breakage itself I'm OK with a new driver version for a new database version and from my experience people expect that. I recall a number of people asking me if an 8.4 driver would be OK to use against 9 before the 9 version was stable. Regards, One would need a machine which supports java 1.4, 1.5, and 1.6 since the driver builds all 3 versions. There's actually a 4th between 1.4 and 1.5 but I don't recall the specifics Dave -- Mike Fowler Registered Linux user: 379787 -- Sent via pgsql-jdbc mailing list (pgsql-j...@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-jdbc -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [JDBC] [HACKERS] JDBC connections to 9.1
On Mon, Apr 18, 2011 at 11:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Cramer p...@fastcrypt.com writes: On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wasn't aware that JDBC would fail on that. It's pretty annoying that it does, but maybe we should grin and bear it, ie revert the change to canonicalize the GUC's value? Older drivers will fail for sure. We can fix newer drivers, but if we leave it we will see a slew of bug reports. Yeah. I'm thinking what we should do here is revert the change, with a note in the source about why, and also change the JDBC driver to send and expect UTF8 not UNICODE (which as Kevin says is more correct anyway). Then in a few releases' time we can un-revert the server change. Well initially my concern was that people would have a challenge in the case where they had to re-certify their application if we made this change, however I realize they will have to do this anyway since upgrading to 9.1 is what necessitates it. So I'm less concerned with bug reports since people can just upgrade both Dave Cramer dave.cramer(at)credativ(dot)ca http://www.credativ.ca -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] create tablespace fails silently, or succeeds improperly
as seen below create tablespace does not throw an error or appear to do anything other than register the tablespace. postg...@db01:~ less /opt/pg/data/jnj_indexes/PG_VERSION 8.4 postg...@db01:~ /opt/pg91/bin/psql -p 5433 psql (9.0.1) Type help for help. postgres=# select version(); version --- PostgreSQL 9.0.1 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-11), 64-bit (1 row) postgres=# create TABLESPACE jnj_indexes location '/opt/pg/data/jnj_indexes'; CREATE TABLESPACE Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] create tablespace fails silently, or succeeds improperly
On Mon, Oct 18, 2010 at 11:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Cramer p...@fastcrypt.com writes: as seen below create tablespace does not throw an error or appear to do anything other than register the tablespace. I suspect this behavior is partially intentional, because tablespace creation now involves an extra level of subdirectory. However, it's not clear to me why CREATE TABLESPACE is still changing the permissions on the parent directory. Bruce, exactly what is the rationale here? OK, it appears there are a few loose ends here then as the documentation http://www.postgresql.org/docs/9.0/interactive/sql-createtablespace.html says the directory needs to be empty. Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] create tablespace fails silently, or succeeds improperly
On Mon, Oct 18, 2010 at 2:20 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: Hi Dave, Excerpts from Dave Cramer's message of lun oct 18 12:23:40 -0300 2010: as seen below create tablespace does not throw an error or appear to do anything other than register the tablespace. postg...@db01:~ less /opt/pg/data/jnj_indexes/PG_VERSION 8.4 postg...@db01:~ /opt/pg91/bin/psql -p 5433 psql (9.0.1) Type help for help. postgres=# select version(); version --- PostgreSQL 9.0.1 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-11), 64-bit (1 row) postgres=# create TABLESPACE jnj_indexes location '/opt/pg/data/jnj_indexes'; CREATE TABLESPACE IIRC the reason this works is that the tablespace code now creates a version-specific subdirectory inside the specified directory. This was done to help binary upgrades. Right, the directory is catalog-version named, which was done to allow for pg_upgrade to work for alpha/beta upgrades (pretty cool). The case above happened because 8.4 still has data in that tablespace. pg_upgrade does supply a script to delete old data files, but it was not used in the case above. right that's because I did not use pg_upgrade. I was manually running create tablespace. Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] create tablespace fails silently, or succeeds improperly
On Mon, Oct 18, 2010 at 2:39 PM, Bruce Momjian br...@momjian.us wrote: Dave Cramer wrote: On Mon, Oct 18, 2010 at 2:20 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: Hi Dave, Excerpts from Dave Cramer's message of lun oct 18 12:23:40 -0300 2010: as seen below create tablespace does not throw an error or appear to do anything other than register the tablespace. postg...@db01:~ less /opt/pg/data/jnj_indexes/PG_VERSION 8.4 postg...@db01:~ /opt/pg91/bin/psql -p 5433 psql (9.0.1) Type help for help. postgres=# select version(); ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? version --- ?PostgreSQL 9.0.1 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-11), 64-bit (1 row) postgres=# create TABLESPACE jnj_indexes location '/opt/pg/data/jnj_indexes'; CREATE TABLESPACE IIRC the reason this works is that the tablespace code now creates a version-specific subdirectory inside the specified directory. ?This was done to help binary upgrades. Right, the directory is catalog-version named, which was done to allow for pg_upgrade to work for alpha/beta upgrades (pretty cool). ?The case above happened because 8.4 still has data in that tablespace. pg_upgrade does supply a script to delete old data files, but it was not used in the case above. right that's because I did not use pg_upgrade. I was manually running create tablespace. OK, so you were sharing the tablespace with old and new clusters. You are right that in the past that would not have been possible because PG_VERSION would have conflicted, but it is now possible with all new releases because of the catalog-version-named subdirectory. That seems like I a feature, I guess. Sounds unintended. As it turns out I was expecting it to fail and was surprised when it succeeded. Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgstatindex still throws ERROR: value 3220078592 is out of range for type integer
On Mon, Jun 7, 2010 at 8:30 PM, Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp wrote: Dave Cramer p...@fastcrypt.com wrote: I noted on line 169 that max_avail is still an int ? Where else would it be having problems ? It should not a problem because the local variable only stores byte size in a page. It will be at most only BLCKSZ (=8192). I wonder why you had ERROR: value ... is out of range for type integer message because we don't use any integer data types for sizes in pgstatindex. The error should have been thrown by SQL typin functions rather than C routines. Takahiro, Yes, this was the problem. I patched the C code but did not change the functions. Thanks, Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgstatindex still throws ERROR: value 3220078592 is out of range for type integer
I noted on line 169 that max_avail is still an int ? Where else would it be having problems ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenSSL key renegotiation with patched openssl
Tom Lane wrote: Dave Cramer p...@fastcrypt.com writes: Recently openssl has been patched to not renegotiate keys. http://www.links.org/?p=780 After a certain amount of data has gone through a postgresql connection the server will attempt to switch session keys. What is the workaround (if any ) to avoid this in postgresql ? Install the updated openssl library. Why are you bugging us about an openssl patch? regards, tom lane After applying the updated openssl library slony dies, presumably because the server requests a new session key Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenSSL key renegotiation with patched openssl
On Fri, Nov 27, 2009 at 4:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: Tom Lane wrote: The discussion I saw suggested that you need such a patch at both ends. and likely requires a restart of both postgresql and slony afterwards... Actually, after looking through the available info about this: https://svn.resiprocate.org/rep/ietf-drafts/ekr/draft-rescorla-tls-renegotiate.txt I think my comment above is wrong. It is useful to patch the *server*-side library to reject a renegotiation request. Applying that patch on the client side, however, is useless and simply breaks things. regards, tom lane I've looked at the available patches for openssl, and so far can only see that ssl3_renegotiate returns 0 if a renegotiation is requested, which would cause pg to throw an error. Is there another patch that fixes this ? I would have expected openssl to simply ignore this request if renegotiation is removed from the library ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] OpenSSL key renegotiation with patched openssl
Recently openssl has been patched to not renegotiate keys. http://www.links.org/?p=780 After a certain amount of data has gone through a postgresql connection the server will attempt to switch session keys. What is the workaround (if any ) to avoid this in postgresql ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] OpenSSL key renegotiation with patched openssl
Tom Lane wrote: Dave Cramer p...@fastcrypt.com writes: Recently openssl has been patched to not renegotiate keys. http://www.links.org/?p=780 After a certain amount of data has gone through a postgresql connection the server will attempt to switch session keys. What is the workaround (if any ) to avoid this in postgresql ? Install the updated openssl library. Why are you bugging us about an openssl patch? regards, tom lane After applying the updated openssl library slony dies, presumably because the server requests a new session key Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pg_standby could not open wal file after selecting new timeline
I am trying to move a db from one machine to another. pg_standby applies all the logs fine, then I trigger it and this happens ??? 2008-11-05 11:43:45 EST [14853] LOG: restored log file 000316ED007E from archive 2008-11-05 11:43:45 EST [14853] LOG: selected new timeline ID: 4 2008-11-05 11:43:45 EST [14853] LOG: archive recovery complete 2008-11-05 11:43:48 EST [14853] FATAL: could not open file pg_xlog/ 000416ED007F (log file 5869, segment 127): Invalid argument 2008-11-05 11:43:48 EST [14846] LOG: startup process (PID 14853) exited with exit code 1 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_standby could not open wal file after selecting new timeline
Tom, On 5-Nov-08, at 12:21 PM, Tom Lane wrote: nvalid argument?? On the platforms I have handy, the only documented reason for open(2) to fail with EINVAL is illegal value of the flags argument, which should be impossible. What platform is this and what wal_sync_method are you using? Red Hat Enterprise Linux Server release 5.2 (Tikanga) wal_sync method is open_sync Thing is the server is running off of a ramdisk for the move (very temporarily) Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_standby could not open wal file after selecting new timeline
On 5-Nov-08, at 1:00 PM, Tom Lane wrote: I wrote: Huh, is it possible that Linux rejects O_SYNC for a file on ramdisk? I found this in the Fedora 9 manpage for open(2): O_DIRECT support was added under Linux in kernel version 2.4.10. Older Linux kernels simply ignore this flag. Some filesystems may not imple- ment the flag and open() will fail with EINVAL if it is used. so it may not be ramdisk per se that's the issue, but the filesystem you're using on it. We set O_DIRECT along with O_SYNC whenever O_DIRECT is defined. I wonder whether there's a need to make that decision more configurable. fsync=off works fine if that helps regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pg_ctl less than useful error message on windows when privileges wrong for postgres
I have a client who mistakenly gave the postgres user on a windows machine admin privileges. This mistake results in the service being unable to start up due to postgres refusing to start with admin privileges. The error message from pg_ctl start -D bindir is PG_CTL..could not locate matching postgres executable Is it not possible to provide a more useful error message ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_ctl less than useful error message on windows when privileges wrong for postgres
On 21-Oct-08, at 5:24 PM, Tom Lane wrote: Dave Cramer [EMAIL PROTECTED] writes: I have a client who mistakenly gave the postgres user on a windows machine admin privileges. This mistake results in the service being unable to start up due to postgres refusing to start with admin privileges. The error message from pg_ctl start -D bindir is PG_CTL..could not locate matching postgres executable It's fairly hard to see how that mistake leads to that symptom. Can you poke a bit more into exactly what is happening? find_my_exec returns -1 for a number of errors resulting in an error message much like above (wording is slightly different). when I executed postgres.exe directly it complained of the user having admin privs I removed admin privs from the pg user and pg_ctl worked fine. I know it's not very specific, but fairly damning. Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] log shipping pg_standby
After terminating recovery mode by creating a trigger file postgres starts up and complains about missing WAL files in pg_xlog; The last two which were replayed. Is this normal ? Dave -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] can't stop autovacuum by HUP'ing the server
On Mon, Aug 25, 2008 at 1:07 PM, Dave Cramer [EMAIL PROTECTED] wrote: On 25-Aug-08, at 10:43 AM, Alvaro Herrera wrote: Dave Cramer wrote: Well, I go the extra mile and kill any remaing autovac procs Here are the logs 2008-08-25 04:00:01 EDT [32276] LOG: autovacuum launcher shutting down 2008-08-25 04:00:01 EDT [20526] LOG: autovacuum launcher started What did you SIGHUP, the launcher or postmaster? You need the latter. The launcher should exit automatically at that time. No, I am HUP'ing the postmaster then subsequently killing any autovacuums still around, which may be the problem. I may be killing the launcher prematurely. I'll try again tonight. Ok, here are the logs from last night 2008-08-26 04:00:02 EDT [25407] LOG: received SIGHUP, reloading configuration files 2008-08-26 04:00:02 EDT [22649] LOG: autovacuum launcher shutting down 2008-08-26 04:00:02 EDT [30438] LOG: autovacuum launcher started you can see the SIGHUP, the launcher being shut down, and starting right back up again ??? is this expected behaviour ? Dave
Re: [HACKERS] can't stop autovacuum by HUP'ing the server
On Tue, Aug 26, 2008 at 9:37 AM, Alvaro Herrera [EMAIL PROTECTED]wrote: Dave Cramer wrote: Ok, here are the logs from last night 2008-08-26 04:00:02 EDT [25407] LOG: received SIGHUP, reloading configuration files 2008-08-26 04:00:02 EDT [22649] LOG: autovacuum launcher shutting down 2008-08-26 04:00:02 EDT [30438] LOG: autovacuum launcher started you can see the SIGHUP, the launcher being shut down, and starting right back up again ??? is this expected behaviour ? Certainly not, and that's not what I see here either. I assume process 25407 is (was) the postmaster, yes? If you show autovacuum, is it on? Yes that was the postmaster, and I did check to see if autovacuum was on, and it was not. Dave -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
