[SQL] Invalid message format Exception
Hi, Because there was no response for this question already posted in pgsql-j...@postgresql.org mailing list, I'm posting it here. I'm using PostgreSQL 8.2 and my production server is based on CentOS release 5.2 (Final). JDBC Jar: postgresql-8.2-508.jdbc4.jar I noticed from my server log that some INSERT statements are failing with "invalid message format" PSQLException. Once this exception is thrown, it is not committed to the database. What could be the reason for this? EXCEPTION org.postgresql.util.PSQLException: ERROR: invalid message format STACKTRACE org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorI mpl.java:1592) org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.ja va:1327) org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:193) org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.j ava:452) org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2St atement.java:337) org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2State ment.java:283) I also read about this on: http://archives.postgresql.org/pgsql-jdbc/2008-04/msg00085.php "JDBC driver is doing something wrong when calculating a message length or message content for the frontend/backend protocol" But how do I resolve this issue? http://archives.postgresql.org/pgsql-jdbc/2004-03/msg00142.php you can't represent a \0 byte in a text/varchar constant Even though I don't have much idea on encoding, I have a little doubt on whether encoding is a problem. Here is my database encoding: mydb=# \l List of databases Name | Owner | Encoding --+--+--- mydb | zoniac | SQL_ASCII Any pointers in right direction are appreciated. -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] Invalid message format Exception
I'm sure most will urge you to move to UTF-8 encoding asap. Have you tracked down the "offending" insert statement? Perhaps it's a trigger trying to generate a log message? On 05/12/2010 04:34 AM, Gnanakumar wrote: > Hi, > > Because there was no response for this question already posted in > pgsql-j...@postgresql.org mailing list, I'm posting it here. > > I'm using PostgreSQL 8.2 and my production server is based on CentOS release > 5.2 (Final). > > JDBC Jar: postgresql-8.2-508.jdbc4.jar > > I noticed from my server log that some INSERT statements are failing with > "invalid message format" PSQLException. Once this exception is thrown, it > is not committed to the database. What could be the reason for this? > > EXCEPTION > org.postgresql.util.PSQLException: ERROR: invalid message format > > STACKTRACE > org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorI > mpl.java:1592) > org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.ja > va:1327) > org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:193) > org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.j > ava:452) > org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2St > atement.java:337) > org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2State > ment.java:283) > > I also read about this on: > http://archives.postgresql.org/pgsql-jdbc/2008-04/msg00085.php > "JDBC driver is doing something wrong when calculating a message > length or message content for the frontend/backend protocol" > But how do I resolve this issue? > http://archives.postgresql.org/pgsql-jdbc/2004-03/msg00142.php > you can't represent a \0 byte in a text/varchar constant > > Even though I don't have much idea on encoding, I have a little doubt on > whether encoding is a problem. > > Here is my database encoding: > mydb=# \l > List of databases > Name | Owner | Encoding > --+--+--- > mydb | zoniac | SQL_ASCII > > Any pointers in right direction are appreciated. > > > -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] best paging strategies for large datasets?
On 5/12/2010 1:41 AM, Louis-David Mitterrand wrote: > Hi, > > I have a large dataset (page 1 at http://www.cruisefish.net/stat.md) and > am in the process of developping a pager to let users leaf through it > (30K rows). > That's not that big of a record set. > Ideally I'd like to know when requesting any 'page' of data where I am > within the dataset: how many pages are available each way, etc. > > Of course that can be done by doing a count(*) query before requesting a > limit/offset subset. But the main query is already quite slow, so I'd > like to minimize them. > What do you mean by quite slow?? On a 30K record table count() and query should speed should be a problem.. > But I am intrigued by window functions, especially the row_number() and > ntile(int) ones. > > Adding "row_number() over (order by)" to my query will > return the total number of rows in the first row, letting my deduce the > number of pages remaining, etc. row_number() apparently adds very little > cost to the main query. > That will get a sequential number, but you still don't know how many records are in the table, limit and offset block that value. I don't see how this helps? Limit and Offset with Total Record count tell us where we are in the record set and which page we are on. RecordCount/Limit = Number of pages CurrentPage = (offset%RecordCount)/Limit to complicate things further what if the site allows user to change the number of records displayed per page. The pager logic needs to figure out how many records need to be return per page, and what the next and previous iterations are. Without the total count records I don't see how that is even possible. I have written pagers in ASP and PHP All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored. CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately. Thank you. -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] best paging strategies for large datasets?
oops typos On 5/12/2010 1:41 AM, Louis-David Mitterrand wrote: > Hi, > > I have a large dataset (page 1 at http://www.cruisefish.net/stat.md) and > am in the process of developping a pager to let users leaf through it > (30K rows). > That's not that big of a record set. > Ideally I'd like to know when requesting any 'page' of data where I am > within the dataset: how many pages are available each way, etc. > > Of course that can be done by doing a count(*) query before requesting a > limit/offset subset. But the main query is already quite slow, so I'd > like to minimize them. > What do you mean by quite slow?? On a 30K record table count() and query speed should not be a problem.. > But I am intrigued by window functions, especially the row_number() and > ntile(int) ones. > > Adding "row_number() over (order by)" to my query will > return the total number of rows in the first row, letting my deduce the > number of pages remaining, etc. row_number() apparently adds very little > cost to the main query. > That will get a sequential number, but you still don't know how many records are in the table, limit and offset block that value. I don't see how this helps? Limit and Offset with Total Record count tell us where we are in the record set and which page we are on. RecordCount/Limit = Number of pages CurrentPage = (offset%RecordCount)/Limit to complicate things further what if the site allows user to change the number of records displayed per page. The pager logic needs to figure out how many records need to be return per page, and what the next and previous iterations are. Without the total record count I don't see how that is even possible. I have written pagers in ASP and PHP All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored. CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately. Thank you. -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
[SQL] Greetings folks, dumb question maybe?
Hello, I'm a little new at this so please bear with me.
I am trying to create a function that loads 100M test records into a
database, however I am having a hard time building the function that
does so.
I'm trying to do this in PGAdmin III for Ubuntu. Is there something
that I have wrong with this? I know that this works in MySQL (and yes I
know that MySQL bends the SQL Standards), but I am not sure what I am
doing wrong exactly. I am coming up with the error that says there's an
error in my syntax near the v INTEGER := 0 line. I get the same error
in psql as I do in the PGAdmin III.
I have the following so far:
DECLARE
v INTEGER := 0;
BEGIN
while v < 1
DO
INSERT INTO unpart_tbl_test VALUES
(v, 'test string data', adddate('1995-01-01', (rand(v)*36520) mod 3652));
v := v + 1;
END WHILE;
END;
Any insight would be greatly appreciated.
- J
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] Greetings folks, dumb question maybe?
On 05/12/2010 01:32 PM, Josh wrote:
Hello, I'm a little new at this so please bear with me.
I am trying to create a function that loads 100M test records into a
database, however I am having a hard time building the function that
does so.
I'm trying to do this in PGAdmin III for Ubuntu. Is there something
that I have wrong with this? I know that this works in MySQL (and yes
I know that MySQL bends the SQL Standards), but I am not sure what I
am doing wrong exactly. I am coming up with the error that says
there's an error in my syntax near the v INTEGER := 0 line. I get the
same error in psql as I do in the PGAdmin III.
I have the following so far:
DECLARE
v INTEGER := 0;
BEGIN
while v < 1
DO
INSERT INTO unpart_tbl_test VALUES
(v, 'test string data', adddate('1995-01-01', (rand(v)*36520) mod 3652));
v := v + 1;
END WHILE;
END;
Any insight would be greatly appreciated.
- J
after some digging I had to first create a language plpgsql, then I
changed the function to be as follows:
CREATE FUNCTION no_part_tbl() RETURNS void AS '
DECLARE
v INTEGER := 0;
BEGIN
WHILE v < 1 LOOP
INSERT INTO no_part_tbl VALUES
(v, "testing no parts", adddate("1995-01-01",
(rand(v)*36520 % 3652));
v := v + 1;
END LOOP;
END;
' LANGUAGE 'plpgsql';
And it seems to accepted the function finally. SOrry for the waste of
bandwidth and anyones time. I'm not used to this syntax, so it will
take me a bit to get on boad with it.
- J
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] Greetings folks, dumb question maybe?
On Wed, May 12, 2010 at 10:32 AM, Josh wrote: > I am trying to create a function that loads 100M test records into a > database, however I am having a hard time building the function that does > so. Here is one idea a function, but is could be wrapped by one: p2082849b=> INSERT INTO Testtable ( n, c, d ) p2082849b-> WITH RECURSIVE T(n,c,d) as ( SELECT 1, 'A', CAST( '2009-12-31' AS DATE ) + INTERVAL '1 DAY' p2082849b(>UNION ALL SELECT n + 1, c, d + INTERVAL '1 day' p2082849b(>FROM T p2082849b(> WHERE n <= 1) p2082849b-> SELECT n,c,d FROM T; INSERT 0 10001 -- Regards, Richard Broersma Jr. Visit the Los Angeles PostgreSQL Users Group (LAPUG) http://pugs.postgresql.org/lapug -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
Re: [SQL] Greetings folks, dumb question maybe?
I've been using the generate_series function and finding it very useful. It generates an integer, but I cast to 'day' interval. Might be useful. doug -Original Message- From: pgsql-sql-ow...@postgresql.org [mailto:pgsql-sql-ow...@postgresql.org] On Behalf Of Richard Broersma Sent: Wednesday, May 12, 2010 4:05 PM To: Josh Cc: pgsql-sql@postgresql.org Subject: Re: [SQL] Greetings folks, dumb question maybe? On Wed, May 12, 2010 at 10:32 AM, Josh wrote: > I am trying to create a function that loads 100M test records into a > database, however I am having a hard time building the function that does > so. Here is one idea a function, but is could be wrapped by one: p2082849b=> INSERT INTO Testtable ( n, c, d ) p2082849b-> WITH RECURSIVE T(n,c,d) as ( SELECT 1, 'A', CAST( '2009-12-31' AS DATE ) + INTERVAL '1 DAY' p2082849b(>UNION ALL SELECT n + 1, c, d + INTERVAL '1 day' p2082849b(>FROM T p2082849b(> WHERE n <= 1) p2082849b-> SELECT n,c,d FROM T; INSERT 0 10001 -- Regards, Richard Broersma Jr. Visit the Los Angeles PostgreSQL Users Group (LAPUG) http://pugs.postgresql.org/lapug -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql -- Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-sql
