Thank you for your responses and sorry about the delay. Unfortunately, it is difficult for us to extract that part of the code from our java application as it's a complex transactional system with lots of dependencies. On the other hand we have checked and verified that this transaction is isolated in the code, i.e. this is the only transaction that is being committed to DB in this scope so that would mean that the code you were using to test is similar enough. However, we replaced the usage of NOW with timestamp prepared in java code in the SQL statement which caused the crash and the problem doesn't seem to appear any more. Since this is not the only place in our code where we use NOW in statements, it is odd that the problem doesn't appear anywhere else, but we'll continue testing and maybe it reappears - then we could be sure that this is what's causing the problem. Seeing your environment I can notice only that you were using newer version of postgreSQL (9.1.1 compared to 8.4.7 which we are using), so maybe that could also be the cause of the problem or the reason why you can't reproduce the error. In any case, I'll let you know if we experience any more problems during our tests. Regards,
silvio -----Original Message----- From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] Sent: 18. studeni 2011 5:57 To: Silvio Svečnjak Cc: pgpool-general@pgfoundry.org Subject: Re: [Pgpool-general] FW: Child process was terminated by segmentation fault > Thank you for your effort. The problem occurs almost always, but not every > time, when the following SQL query is issued: > > UPDATE ahs_transaction SET end_timestamp = NOW(), response_code = $1 WHERE id > = $2 I did some tests with code something like below: PreparedStatement pstmt = null; for (int i = 0; i < N; i++) { connection.setAutoCommit(false); pstmt = connection.prepareStatement("UPDATE ahs_transaction SET end_timestamp = NOW(), response_code = ? WHERE id = ?"); pstmt.setInt(1, i); pstmt.setInt(2, i); pstmt.executeUpdate(); pstmt.close(); connection.commit(); But had not seen crash. > Where ahs_transaction table is defined as follows: > > CREATE TABLE ahs_transaction > ( > ID BIGSERIAL NOT NULL PRIMARY KEY, > start_timestamp TIMESTAMP NOT NULL DEFAULT NOW(), > end_timestamp TIMESTAMP, > request_ID INTEGER NOT NULL REFERENCES ahs_request(ID), > response_code SMALLINT > ); Because the table references another table, I removed the foreign key when I did the test: CREATE TABLE ahs_transaction ( ID BIGSERIAL NOT NULL PRIMARY KEY, start_timestamp TIMESTAMP NOT NULL DEFAULT NOW(), end_timestamp TIMESTAMP, request_ID INTEGER NOT NULL, response_code SMALLINT ); In summary, I could not reproduce your problem here. More details about my env: OS: Linux kernel 2.6.35 x86_64 jdbc driver: postgresql-9.1-901.jdbc4.jar PostgreSQL 9.1.1 java-1.6.0-openjdk-1.6.0.0-4.b22vl6.x86_64 It would be helpfull if you give me standalone sample Java program and table (and data if neccessary) which can reproduce the proble. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp > On this transaction our java application which is issuing the query receives > an SQL exception (I/O error). > I hope this helps. If I can help you with anything else, let me know. > Regards, > > silvio > > > > > -----Original Message----- > From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] > Sent: 17. studeni 2011 0:33 > To: Silvio Svečnjak > Cc: pgpool-general@pgfoundry.org > Subject: Re: [Pgpool-general] FW: Child process was terminated by > segmentation fault > > Thanks. This backtrace is much better than mine (probably the best at > this point). I'm going to debug using the info. In the mean time I > might need additional info such as variable contents from the core. If > it occurs, I will let you know. So please keep the core file > executable file of pgpool and sources. > > Also if you know how to reproduce the problem, please let me know. > With the info, I could reproduce the problem here and make it very > easy(thus quickly) to find the cause of the bug. Because core is just > the result of the bug, sometimes it is very hard to find the cause of > the bug. Knowing how to reporduce the problem is the best way to find > the cause. > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese: http://www.sraoss.co.jp > >> 2 files I sent you were the backtrace from the core dump file ($ gdb >> /usr/local/bin/pgpool core-file + (gdb)bt). I have repeated the error and >> took the backtrace again (file is enclosed), but the backtrace is the same. >> How to get additional info in the backtrace? >> Regards, >> >> silvio >> >> >> >> -----Original Message----- >> From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] >> Sent: 16. studeni 2011 12:41 >> To: Silvio Svečnjak >> Cc: pgpool-general@pgfoundry.org >> Subject: Re: [Pgpool-general] FW: Child process was terminated by >> segmentation fault >> >> Can you show me backtrace? >> >> $ gdb pgpool core-file >> (gdb)bt >> >> I expect to get more usefull info because the core was generated by >> your pgpool binary. >> -- >> Tatsuo Ishii >> SRA OSS, Inc. Japan >> English: http://www.sraoss.co.jp/index_en.php >> Japanese: http://www.sraoss.co.jp >> >>> Hello! >>> >>> Could you please clarify what would be more useful info for further >>> analysis? Thanks, >>> >>> silvio >>> >>> >>> -----Original Message----- >>> From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] >>> Sent: 16. studeni 2011 2:03 >>> To: Silvio Svečnjak >>> Cc: pgpool-general@pgfoundry.org >>> Subject: Re: [Pgpool-general] FW: Child process was terminated by >>> segmentation fault >>> >>> Silvio, >>> >>> Thank you for sending cores. I got backtrace this: >>> >>> gdb .libs/pgpool ~/core-pgpool-11-0-0-20473-1321347122 >>> GNU gdb (GDB) 7.2 >>> Copyright (C) 2010 Free Software Foundation, Inc. >>> License GPLv3+: GNU GPL version 3 or later >>> <http://gnu.org/licenses/gpl.html> >>> This is free software: you are free to change and redistribute it. >>> There is NO WARRANTY, to the extent permitted by law. Type "show copying" >>> and "show warranty" for details. >>> This GDB was configured as "x86_64-vine-linux". >>> For bug reporting instructions, please see: >>> <http://www.gnu.org/software/gdb/bugs/>... >>> Reading symbols from >>> /home/t-ishii/work/pgfoundry/pgpool-II-3.1/.libs/pgpool...done. >>> >>> warning: exec file is newer than core file. >>> [New Thread 20473] >>> Reading symbols from /lib64/ld-linux-x86-64.so.2...done. >>> Loaded symbols for /lib64/ld-linux-x86-64.so.2 >>> >>> warning: no loadable sections found in added symbol-file system-supplied >>> DSO at 0x7fff50f70000 >>> Core was generated by `pgpool: vcms'. >>> Program terminated with signal 11, Segmentation fault. >>> #0 0x00000000004626ef in core_yylex_init (ptr_yy_globals=0x100) at >>> scan.c:3444 >>> 3444 memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); >>> (gdb) bt >>> #0 0x00000000004626ef in core_yylex_init (ptr_yy_globals=0x100) at >>> scan.c:3444 >>> #1 0x00000000000000b2 in ?? () >>> #2 0x0000000000000050 in ?? () >>> #3 0x000000001c1978b0 in ?? () >>> #4 0x0000000000000000 in ?? () >>> (gdb) >>> >>> It seems the backtrace does not show too much usefull info. I think >>> this is because the stack frame was trashed badly by a overrun of >>> memory copy. I'm looking into the problem anyway but it would be great >>> if you or someone provide me more usefull info... >>> -- >>> Tatsuo Ishii >>> SRA OSS, Inc. Japan >>> English: http://www.sraoss.co.jp/index_en.php >>> Japanese: http://www.sraoss.co.jp >>> >>>> Thanks for your quick response. Dump/trace files are enclosed - I hope >>>> they contain the information you need - if not please let me know what >>>> else could help you. >>>> Regards, >>>> >>>> silvio >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] >>>> Sent: 14. studeni 2011 16:05 >>>> To: Silvio Svečnjak >>>> Cc: pgpool-general@pgfoundry.org >>>> Subject: Re: [Pgpool-general] FW: Child process was terminated by >>>> segmentation fault >>>> >>>>> We're using pgpool 3.1 with PostgreSQL 8.4.7 on RHEL 5.3 x64 where pgpool >>>>> is configured for replication only on 2 backends. Pgpool is used by a >>>>> real time java application which occasionally (without any visible >>>>> pattern) catches an exception (org.postgresql.util.PSQLException: An I/O >>>>> error occured while sending to the backend). >>>>> >>>>> >>>>> >>>>> We've determined that the exception is caused by pgpool as at the same >>>>> time pgpool records the following error: >>>>> >>>>> >>>>> >>>>> 2011-11-14 14:14:45 ERROR: pid 10783: Child process 10812 was terminated >>>>> by segmentation fault >>>>> >>>>> 2011-11-14 14:14:45 DEBUG: pid 10783: child 10812 exits with status 11 by >>>>> signal 11 >>>>> >>>>> >>>>> >>>>> Bigger chunk of log file is enclosed as well as the pgpool configuration >>>>> file. >>>>> >>>>> >>>>> >>>>> After some research on the mailing lists, I found that some patches have >>>>> been released for version 3.1 which may resolve the issue. So I >>>>> downloaded and deployed the latest stable release >>>>> pgpool2-V3_1_STABLE-8b47d55, but it didn't solve the problem. >>>>> >>>>> >>>>> >>>>> I would appreciate very much if someone could provide any help with this? >>>>> Thank you in advance, >>>> >>>> Can you take a core and show us the backtrace? It will be very helpfull. >>>> -- >>>> Tatsuo Ishii >>>> SRA OSS, Inc. Japan >>>> English: http://www.sraoss.co.jp/index_en.php >>>> Japanese: http://www.sraoss.co.jp _______________________________________________ Pgpool-general mailing list Pgpool-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgpool-general
