Hi Pierre, The hs_err log file gives some info. Please send it in. But, we need the core file to analyze the failure. Can you please upload this core file in Google drive and allow me to download this file by sending an email with download link?
Does anyone know a better way to get this file across? Selva -----Original Message----- From: Pierre Smits [mailto:[email protected]] Sent: Wednesday, March 30, 2016 12:23 AM To: [email protected] Subject: Re: problem_with_server_read Hi Selva, The file at /proc/sys/kernel/core_pattern shows: |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e This morning, after a clean restart of both Trafodion and Apache OFBiz. the same happened again (hanging OFBiz). This seems to be related to a specific query: SELECT JOB_ID, JOB_NAME, RUN_TIME, POOL_ID, STATUS_ID, PARENT_JOB_ID, PREVIOUS_JOB_ID, SERVICE_NAME, LOADER_NAME, MAX_RETRY, CURRENT_RETRY_COUNT, AUTH_USER_LOGIN_ID, RUN_AS_USER, RUNTIME_DATA_ID, RECURRENCE_INFO_ID, TEMP_EXPR_ID, CURRENT_RECURRENCE_COUNT, MAX_RECURRENCE_COUNT, RUN_BY_INSTANCE_ID, START_DATE_TIME, FINISH_DATE_TIME, CANCEL_DATE_TIME, JOB_RESULT, LAST_UPDATED_STAMP, LAST_UPDATED_TX_STAMP, CREATED_STAMP, CREATED_TX_STAMP FROM OFBIZ.JOB_SANDBOX WHERE (((RUN_TIME <= ? AND START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL) AND (POOL_ID IS NULL OR POOL_ID = ?))) ORDER BY RUN_TIME ASC (The message id: problem_with_server_read) In sqlci I executed a somewhat similar query: select * from ofbiz.job_sandbox This resulted in: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f71e7a1f907, pid=50530, tid=140127074763584 # # JRE version: Java(TM) SE Runtime Environment (8.0_40-b25) (build 1.8.0_40-b25) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libexecutor.so+0x214907] ExHbaseAccessTcb::createSQRowFromHbaseFormat(long*)+0x3c7 # # Core dump written. Default location: /home/trafodion/core or core.50530 # # An error report file with more information is saved as: # /home/trafodion/hs_err_pid50530.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp The file core.50530 is huge. But the /home/trafodion/hs_err_pid50530.log is better sized to send via email (if you want me too). Best regards, Pierre Smits ORRTIZ.COM <http://www.orrtiz.com> OFBiz based solutions & services OFBiz Extensions Marketplace http://oem.ofbizci.net/oci-2/ On Tue, Mar 29, 2016 at 8:41 PM, Selva Govindarajan < [email protected]> wrote: > Hi Pierre, > > Are you seeing any core dump of the mxosrvr? The core is dumped at > the location based on /proc/sys/kernel/core_pattern. > > Selva > > -----Original Message----- > From: Pierre Smits [mailto:[email protected]] > Sent: Tuesday, March 29, 2016 11:35 AM > To: [email protected] > Subject: Re: problem_with_server_read > > This happened also (approx at the same time): > > *** glibc detected *** mxosrvr: corrupted double-linked list: > 0x0000000002f1f000 *** > > ======= Backtrace: ========= > > /lib64/libc.so.6[0x3e9f275e66] > > /lib64/libc.so.6[0x3e9f279d0f] > > /lib64/libc.so.6(__libc_malloc+0x71)[0x3e9f27a6b1] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x907445)[0x7fe > 07382b445] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x2dbeb4)[0x7fe > 0731ffeb4] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x2dcd46)[0x7fe > 073200d46] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x2dcf76)[0x7fe > 073200f76] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x8dda98)[0x7fe > 073801a98] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x497b9f)[0x7fe > 0733bbb9f] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x49c44a)[0x7fe > 0733c044a] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x49df4a)[0x7fe > 0733c1f4a] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x3ebeb8)[0x7fe > 07330feb8] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x4a843a)[0x7fe > 0733cc43a] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x4a93e6)[0x7fe > 0733cd3e6] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0xa5c96f)[0x7fe > 07398096f] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0xa5ca9c)[0x7fe > 073980a9c] > > > /usr/jdk64/jdk1.8.0_40/jre/lib/amd64/server/libjvm.so(+0x910ee8)[0x7fe > 073834ee8] > > /lib64/libpthread.so.0[0x3e9f6079d1] > > /lib64/libc.so.6(clone+0x6d)[0x3e9f2e88fd] > > ======= Memory map: ======== > > 00400000-0051a000 r-xp 00000000 fd:02 4587565 > /home/trafodion/apache-trafodion-1.3.0-incubating-bin/export/bin64/mxo > srvr > > 00719000-0071d000 rw-p 00119000 fd:02 4587565 > /home/trafodion/apache-trafodion-1.3.0-incubating-bin/export/bin64/mxo > srvr > > 0071d000-00741000 rw-p 00000000 00:00 0 > > 00fc0000-03e32000 rw-p 00000000 00:00 0 [heap] > > 10000000-14000000 rw-s 00000000 00:04 720913 /SYSV10008cfd (deleted) > > e0000000-e4080000 rw-p 00000000 00:00 0 > > e4080000-f5580000 ---p 00000000 00:00 0 > > f5580000-100000000 rw-p 00000000 00:00 0 > > 100000000-1002a0000 rw-p 00000000 00:00 0 > > 1002a0000-140000000 ---p 00000000 00:00 0 > > 3be7600000-3be7652000 r-xp 00000000 fd:00 2757384 > /usr/lib64/libcurl.so.4.1.1 > > 3be7652000-3be7851000 ---p 00052000 fd:00 2757384 > /usr/lib64/libcurl.so.4.1.1 > > 3be7851000-3be7854000 rw-p 00051000 fd:00 2757384 > /usr/lib64/libcurl.so.4.1.1 > > 3be8600000-3be860e000 r-xp 00000000 fd:00 2359348 > /lib64/liblber-2.4.so.2.10.2 > > 3be860e000-3be880d000 ---p 0000e000 fd:00 2359348 > /lib64/liblber-2.4.so.2.10.2 > > 3be880d000-3be880e000 r--p 0000d000 fd:00 2359348 > /lib64/liblber-2.4.so.2.10.2 > > 3be880e000-3be880f000 rw-p 0000e000 fd:00 2359348 > /lib64/liblber-2.4.so.2.10.2 > > 3be9600000-3be9602000 r-xp 00000000 fd:00 2359333 /lib64/libfreebl3.so > > 3be9602000-3be9801000 ---p 00002000 fd:00 2359333 /lib64/libfreebl3.so > > 3be9801000-3be9802000 r--p 00001000 fd:00 2359333 /lib64/libfreebl3.so > > 3be9802000-3be9803000 rw-p 00002000 fd:00 2359333 /lib64/libfreebl3.so > > 3be9a00000-3be9a07000 r-xp 00000000 fd:00 2359334 > /lib64/libcrypt-2.12.so > > 3be9a07000-3be9c07000 ---p 00007000 fd:00 2359334 > /lib64/libcrypt-2.12.so > > 3be9c07000-3be9c08000 r--p 00007000 fd:00 2359334 > /lib64/libcrypt-2.12.so > > 3be9c08000-3be9c09000 rw-p 00008000 fd:00 2359334 > /lib64/libcrypt-2.12.so > > 3be9c09000-3be9c37000 rw-p 00000000 00:00 0 > > 3beda00000-3beda39000 r-xp 00000000 fd:00 2359335 /lib64/libnspr4.so > > 3beda39000-3bedc39000 ---p 00039000 fd:00 2359335 /lib64/libnspr4.so > > 3bedc39000-3bedc3a000 r--p 00039000 fd:00 2359335 /lib64/libnspr4.so > > 3bedc3a000-3bedc3c000 rw-p 0003a000 fd:00 2359335 /lib64/libnspr4.so > > 3bedc3c000-3bedc3e000 rw-p 00000000 00:00 0 > > 3bede00000-3bede25000 r-xp 00000000 fd:00 2758097 > /usr/lib64/libnssutil3.so > > 3bede25000-3bee025000 ---p 00025000 fd:00 2758097 > /usr/lib64/libnssutil3.so > > 3bee025000-3bee02b000 r--p 00025000 fd:00 2758097 > /usr/lib64/libnssutil3.so > > 3bee02b000-3bee02c000 rw-p 0002b000 fd:00 2758097 > /usr/lib64/libnssutil3.so > > 3bee200000-3bee204000 r-xp 00000000 fd:00 2359337 /lib64/libplc4.so > > 3bee204000-3bee403000 ---p 00004000 fd:00 2359337 /lib64/libplc4.so > > 3bee403000-3bee404000 r--p 00003000 fd:00 2359337 /lib64/libplc4.so > > 3bee404000-3bee405000 rw-p 00004000 fd:00 2359337 /lib64/libplc4.so > > 3bee600000-3bee603000 r-xp 00000000 fd:00 2359341 /lib64/libplds4.so > > 3bee603000-3bee802000 ---p 00003000 fd:00 2359341 /lib64/libplds4.so > > 3bee802000-3bee803000 r--p 00002000 fd:00 2359341 /lib64/libplds4.so > > 3bee803000-3bee804000 rw-p 00003000 fd:00 2359341 /lib64/libplds4.so > > 3beea00000-3beeb37000 r-xp 00000000 fd:00 2769235 > /usr/lib64/libnss3.so > > 3beeb37000-3beed36000 ---p 00137000 fd:00 2769235 > /usr/lib64/libnss3.so > > 3beed36000-3beed3b000 r--p 00136000 fd:00 2769235 > /usr/lib64/libnss3.so > > 3beed3b000-3beed3d000 rw-p 0013b000 fd:00 2769235 > /usr/lib64/libnss3.so > > 3beed3d000-3beed3f000 rw-p 00000000 00:00 0 > > 3beee00000-3beee28000 r-xp 00000000 fd:00 2769244 > /usr/lib64/libsmime3.so > > 3beee28000-3bef027000 ---p 00028000 fd:00 2769244 > /usr/lib64/libsmime3.so > > 3bef027000-3bef02b000 r--p 00027000 fd:00 2769244 > /usr/lib64/libsmime3.so > > 3bef02b000-3bef02c000 rw-p 0002b000 fd:00 2769244 > /usr/lib64/libsmime3.so > > 3bef200000-3bef23b000 r-xp 00000000 fd:00 2769242 > /usr/lib64/libssl3.so > > 3bef23b000-3bef43b000 ---p 0003b000 fd:00 2769242 > /usr/lib64/libssl3.so > > 3bef43b000-3bef43e000 r--p 0003b000 fd:00 2769242 > /usr/lib64/libssl3.so > > 3bef43e000-3bef43f000 rw-p 0003e000 fd:00 2769242 > /usr/lib64/libssl3.so > > 3bef43f000-3bef440000 rw-p 00000000 00:00 0 > > 3bef600000-3bef619000 r-xp 00000000 fd:00 2779034 > /usr/lib64/libsasl2.so.2.0.23 > > 3bef619000-3bef818000 ---p 00019000 fd:00 2779034 > /usr/lib64/libsasl2.so.2.0.23 > > 3bef818000-3bef819000 r--p 00018000 fd:00 2779034 > /usr/lib64/libsasl2.so.2.0.23 > > 3bef819000-3bef81a000 rw-p 00019000 fd:00 2779034 > /usr/lib64/libsasl2.so.2.0.23 > > 3befa00000-3befa4d000 r-xp 00000000 fd:00 2359366 > /lib64/libldap-2.4.so.2.10.2 > > 3befa4d000-3befc4c000 ---p 0004d000 fd:00 2359366 > /lib64/libldap-2.4.so.2.10.2 > > 3befc4c000-3befc4e000 r--p 0004c000 fd:00 2359366 > /lib64/libldap-2.4.so.2.10.2 > > 3befc4e000-3befc50000 rw-p 0004e000 fd:00 2359366 > /lib64/libldap-2.4.so.2.10.2 > > 3e9ea00000-3e9ea20000 r-xp 00000000 fd:00 2359404 /lib64/ld-2.12.so > > 3e9ec1f000-3e9ec20000 r--p 0001f000 fd:00 2359404 /lib64/ld-2.12.so > > > Best regards, > > Pierre Smits > > ORRTIZ.COM <http://www.orrtiz.com> > OFBiz based solutions & services > > OFBiz Extensions Marketplace > http://oem.ofbizci.net/oci-2/ > > On Tue, Mar 29, 2016 at 8:29 PM, Pierre Smits <[email protected]> > wrote: > > > Hi all, > > > > After a restart of both the Trafodion 1.3 Sandbox and Apache ofbiz I > > get the subject of this posting returned in the OFBiz log. The > > excerpt > > shows: > > > > > > [java] Caused by: org.trafodion.jdbc.t4.HPT4Exception: The > > message > > id: problem_with_server_read > > > > [java] at > > org.trafodion.jdbc.t4.HPT4Messages.createSQLException(HPT4Messages.j > > av > > a:304) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.InputOutput.doIO(InputOutput.java:373) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.T4Connection.getReadBuffer(T4Connection.java:1 > > 54 > > ) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.T4Connection.EndTransaction(T4Connection.java: > > 39 > > 8) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.InterfaceConnection.endTransaction(InterfaceCo > > nn > > ection.java:1222) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.InterfaceConnection.rollback(InterfaceConnecti > > on > > .java:440) > > ~[jdbcT4.jar:?] > > > > [java] at > > org.trafodion.jdbc.t4.TrafT4Connection.rollback(TrafT4Connection.java: > > 1025) > > ~[jdbcT4.jar:?] > > > > > > This continued multiple times until the OFBiz instantiation halts. > > > > Can anybody shed a light what this means and why this is happening? > > > > Best regards, > > > > Pierre Smits > > > > ORRTIZ.COM <http://www.orrtiz.com> > > OFBiz based solutions & services > > > > OFBiz Extensions Marketplace > > http://oem.ofbizci.net/oci-2/ > > >
