hi could you do: gdb tdm_arkesp core.36479 and send in the trace? anoop -----Original Message----- From: Radu Marias [mailto:[email protected]] Sent: Friday, September 11, 2015 12:26 PM To: dev <[email protected]> Subject: Re: HammerDB issues
Dave, Narendra BTW, Amanda has access to our cluster. Please feel free to take a look if you think it helps in debugging the issue. On Fri, Sep 11, 2015 at 10:24 PM, Radu Marias <[email protected]> wrote: > $ file core.36479 > core.36479: ELF 64-bit LSB core file x86-64, version 1 (SYSV), > SVR4-style, from 'tdm_arkesp SQMON1.1 00000 00000 036479 $Z000US9 > <IP>:35596 00004 0000' > > > On Fri, Sep 11, 2015 at 10:22 PM, Narendra Goyal > <[email protected] > > wrote: > >> Hi Radu, >> >> Could you please run 'file core*' in the same directory. At least it >> will identify the program that core'd. >> >> Thanks, >> -Narendra >> >> -----Original Message----- >> From: Radu Marias [mailto:[email protected]] >> Sent: Friday, September 11, 2015 12:15 PM >> To: dev <[email protected]> >> Subject: Re: HammerDB issues >> >> yes, >> >> $ pwd >> /home/trafodion/trafodion-20150828_0830/sql/scripts >> $ ls -ltrh | grep core. >> -rwxr-x--- 1 trafodion trafodion 3.3K Aug 28 08:30 sqcorefile >> -rw------- 1 trafodion trafodion 152M Sep 11 13:27 core.42133 >> -rw------- 1 trafodion trafodion 162M Sep 11 17:36 core.9560 >> -rw------- 1 trafodion trafodion 162M Sep 11 17:39 core.10408 >> -rw------- 1 trafodion trafodion 152M Sep 11 18:27 core.36479 >> >> All are about 5.3 MB archived. Do you want to connect to the cluster >> and debug them? Or should I share over email? Because I don't know if >> sensitive information is included to post them to mailing list also. >> >> A quick look, but don't think it helps without symbol-file: >> >> Program terminated with signal 11, Segmentation fault. >> #0 0x00007f619ae22b56 in ?? () >> (gdb) bt >> #0 0x00007f619ae22b56 in ?? () >> #1 0x00007f619ae0d710 in ?? () >> #2 0x00007f618ceafb30 in ?? () >> #3 0x00007fffe7279c10 in ?? () >> #4 0x000000000040bd4d in ?? () >> #5 0x00007f618ceafb30 in ?? () >> #6 0x0000000000000000 in ?? () >> >> On Fri, Sep 11, 2015 at 9:53 PM, Dave Birdsall >> <[email protected]> >> wrote: >> >> > Sorry, I meant core dumps from abending processes. >> > >> > See any files with file name "core.<node name>.<pid>.<executable >> > name>" out there? >> > >> > If so might be interesting to use gdb to see the stack trace. >> > >> > -----Original Message----- >> > From: Radu Marias [mailto:[email protected]] >> > Sent: Friday, September 11, 2015 11:44 AM >> > To: dev <[email protected]> >> > Subject: Re: HammerDB issues >> > >> > By "core files" you mean mxssmp processes? If so then this is >> > present on all >> > nodes: >> > >> > $ ps x | grep mxssmp >> > 46157 ? SNl 0:02 mxssmp SQMON1.1 00004 00004 046157 $ZSM004 >> > 188.138.61.180:35762 00011 00004 00005 SPARE >> > >> > I tried this: >> > >> > sqstop (then needed to use ckillall because it was stuck) restarted >> > hdp sqstart >> > >> > Now this works from sqlci: >> > >> > CREATE UNIQUE INDEX CUSTOMER_I100 ON CUSTOMER (C_W_ID, C_D_ID, >> > C_LAST, C_FIRST, C_ID); >> > >> > But when I run hammerdb without SPJ it inserts data in some tables >> > and then it fails again with >> > >> > Vuser 1:Creating Index CUSTOMER_I2... >> > Vuser 1:Failed to Create Index >> > >> > And now if I run the create statement in sqlci I get the same error >> > "Operating system error 201". >> > >> > *So it seems that while hammerdb inserts data some trafodion >> > processes went away unexpectedly?* >> > >> > On Fri, Sep 11, 2015 at 8:47 PM, Dave Birdsall >> > <[email protected]> >> > wrote: >> > >> > > The first one ("Operating system error 201") means a process went >> > > away unexpectedly. Do you see any core files? >> > > >> > > -----Original Message----- >> > > From: Radu Marias [mailto:[email protected]] >> > > Sent: Friday, September 11, 2015 10:46 AM >> > > To: dev <[email protected]> >> > > Subject: Re: HammerDB issues >> > > >> > > This is what I get for the DDL you mentioned: >> > > >> > > >>CREATE UNIQUE INDEX CUSTOMER_I2 ON CUSTOMER (C_W_ID, C_D_ID, >> > > >>C_LAST, >> > > C_FIRST, C_ID); >> > > >> > > *** ERROR[2034] $Z00083Z:144: Operating system error 201 while >> > > communicating with server process $Z0201A7:26. >> > > >> > > *** ERROR[1081] Loading of index TRAFODION.TPCC.CUSTOMER_I2 >> > > failed unexpectedly. >> > > >> > > --- SQL operation failed with errors. >> > > >> > > The one for ORDERS is working: >> > > >> > > >>CREATE UNIQUE INDEX ORDERS_I2 ON ORDERS (O_W_ID, O_D_ID, >> > > >>O_C_ID, O_ID); >> > > >> > > --- SQL operation complete. >> > > >> > > $ sqcheck >> > > Checking if processes are up. >> > > Checking attempt: 1; user specified max: 2. Execution time in >> > > seconds: >> > > 0. >> > > >> > > The SQ environment is up! >> > > >> > > >> > > Process Configured Actual Down >> > > ------- ---------- ------ ---- >> > > DTM 5 5 >> > > RMS 10 10 >> > > MXOSRVR 20 20 >> > > >> > > Don't see any errors in trafodion logs. >> > > >> > > On Fri, Sep 11, 2015 at 8:19 PM, Radu Marias >> > > <[email protected]> >> > wrote: >> > > >> > > > pdsh is available but how should the jar files be replicated? >> > > > >> > > > I also installed HammerDB on the other 4 nodes and setup first >> > > > node as master and the 4 nodes as slaves as described here >> > > > http://hammerora.sourceforge.net/hammerdb_remote_modes.pdf >> > > > I also checked "*Copy Stored Procedures to Remote Nodes*" and >> > > > added the list of all 5 nodes but I don't see *NEWORDER.jar* on >> > > > the other >> > > > 4 nodes in *HammerDB-2.18* folder. I assume all the jars are >> > > > generated on the first node and copied to the others. But it >> > > > only generates >> > > > *NEWORDER.jar* on the first node and it fails and it's not >> > > > present on the other nodes. Also the rest of the stored >> > > > procedure jars are not created. >> > > > >> > > > For the create index DDL used, I changed config.xml file from >> > > > hammerdb to enable logs in /tmp and in that file I can only see >> > > > the trimmed statement. >> > > > Is there anywhere I can see additional logs? Or can I see in >> > > > trafodion logs the last statement executed from hammerdb over odbc? >> > > > >> > > > On Fri, Sep 11, 2015 at 6:38 PM, Suresh Subbiah < >> > > > [email protected]> wrote: >> > > > >> > > >> Hi Radu, >> > > >> >> > > >> Thanks for trying this out. >> > > >> >> > > >> I think what is happening here is that the jar file did not >> > > >> get pushed to all the nodes. HammerDB has so far been tested >> > > >> on single node instances or multiple nodes where we did some >> > > >> intermediate steps manually. I don't think HammerDB uses pdsh >> > > >> to push the jar file to all nodes, or maybe pdsh is somehow >> > > >> not available. Could you please check if the jar file is >> > > >> available from every node in the cluster? If this does turn >> > > >> out to be the problem, I promise to add more info about >> > > >> HammerDB in the Trafodion wiki and mention a workaround for these >> > > >> issues. >> > > >> >> > > >> For the Failed To Create Index issue, is there an error message? >> > > >> IF not we can take the create index statement HammerDB is >> > > >> using and issue it from sqlci. I think thisis the DDL that is >> > > >> used. >> > > >> >> > > >> CREATE UNIQUE INDEX CUSTOMER_I2 ON CUSTOMER (C_W_ID, C_D_ID, >> > > >> C_LAST, C_FIRST, C_ID) ; CREATE UNIQUE INDEX ORDERS_I2 ON >> > > >> ORDERS (O_W_ID, O_D_ID, O_C_ID, O_ID) ; >> > > >> >> > > >> Thanks >> > > >> Suresh >> > > >> >> > > >> >> > > >> >> > > >> On Fri, Sep 11, 2015 at 9:27 AM, Radu Marias >> > > >> <[email protected]> >> > > >> wrote: >> > > >> >> > > >> > Hi, >> > > >> > >> > > >> > I'm trying to run HammerDB TPC-C from this tutorial >> > > >> > http://hammerora.sourceforge.net/hammerdb_quickstart_trafodion. >> > > >> > pd >> > > >> > f >> > > >> > >> > > >> > I have this environment: >> > > >> > >> > > >> > >> > > >> > *CentOS release 6.7 (Final)* *Ambari 2.1.1* *HDP 2.2* >> > > >> > *trafodion-20150828_0830* >> > > >> > >> > > >> > *HammerDB-2.18* >> > > >> > >> > > >> > *java version "1.7.0_79"* >> > > >> > >> > > >> > On *Schema Build* I get errors when stored procedures are >> created. >> > > >> > >> > > >> > *This is from the hammerdb logs:* >> > > >> > >> > > >> > Vuser 1:CREATING TPCC STORED PROCEDURES Vuser 1:Failed to >> > > >> > create library >> > > >> /home/trafodion/HammerDB-2.18/NEWORDER.jar >> > > >> > >> > > >> > *This is what I see in the UI (attached is also a >> > > >> > screenshot):* >> > > >> > >> > > >> > loading history file ... 0 events added Main console display >> > > >> > active >> > > >> > (Tcl8.6.0 / Tk8.6.0) The xml in config.xml is well-formed, >> > > >> > applying variables Error in Virtual User 1: [Trafodion ODBC >> > > >> > Driver][Trafodion Database] SQL >> > > >> > ERROR:*** ERROR[1382] JAR or DLL file >> > > >> > /home/trafodion/HammerDB-2.18/NEWORDER.jar not found. >> > > >> > [2015-09-11 >> > > >> > 07:50:53]À€ç¿™À€éŽˆç»„ç¿™À€éŽ„组翙À€éŽ€ç»„ç¿™À€é >> > > >> > 组翙À€é 组翙À€éŒ¸ç»„ç¿™À€å ¦è šç¿™À€ìªˆÊŒÀ€À€ì¹°Ê‰À€À€é >> > > >> > >> > > >> 组翙À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€ >> > > >> À€Õ >> > > >> €Ë >> > > >> ®À >> > > >> €À€ìŸ >> > > >> > Ì…À€À€À€À€À€À€À€À€À€À€ï¿¿ï¿¿À€À€ì« ï¿¿ >> > > >> À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€À€é >> > > >> > °À€À€À€ì² ʉÀ€À€îˆ°Ò»À€À€éŸ¼ç« ç¿™À€ë¾°Ç´À€À€å–€Ò À€À€ç½ >> > > >> > Ë—À€À€ì² ʉÀ€À€é’€ç»„ç¿™À€ã µç« ç¿™À€é‘ 组翙À€ìµ ʉÀ€À€é >> > > >> > °ç»„ (executing the statement) >> > > >> > (HammerDB-2.18) 1 % >> > > >> > >> > > >> > By looking in the tpcc schema I see some tables are created >> > > >> > and are populated with data: >> > > >> > >> > > >> > >>set schema tpcc; >> > > >> > >> > > >> > --- SQL operation complete. >> > > >> > >>get tables; >> > > >> > >> > > >> > Tables in Schema TRAFODION.TPCC >> > > >> > =============================== >> > > >> > >> > > >> > CUSTOMER >> > > >> > DISTRICT >> > > >> > HISTORY >> > > >> > ITEM >> > > >> > NEW_ORDER >> > > >> > ORDERS >> > > >> > ORDER_LINE >> > > >> > STOCK >> > > >> > WAREHOUSE >> > > >> > >> > > >> > The file '*/home/trafodion/HammerDB-2.18/NEWORDER.jar*' >> > > >> > exists, attached is the NEWORDER.java >> > > >> > >> > > >> > *When trying to create the stored procedure from sqlci I get >> > > >> > this:* >> > > >> > >> > > >> > set schema tpcc; >> > > >> > create library testrs file >> > > >> > '/home/trafodion/HammerDB-2.18/NEWORDER.jar'; >> > > >> > create procedure NEWORD() >> > > >> > language java >> > > >> > parameter style java >> > > >> > external name 'NEWORDER.NEWORD' >> > > >> > dynamic result sets 1 >> > > >> > library testrs; >> > > >> > >> > > >> > *** ERROR[11239] No compatible Java methods named 'NEWORD' >> > > >> > were found in Java class 'NEWORDER'. >> > > >> > >> > > >> > *** ERROR[1231] User-defined routine TRAFODION.TPCC.NEWORDER >> > > >> > could not >> > > >> be >> > > >> > created. >> > > >> > >> > > >> > Is there an API change on methods for stored procedures in >> > > >> > the latest trafodion and hammerdb uses the older syntax? >> > > >> > This is the >> > > >> > method: >> > > >> > >> > > >> > public static void NEWORD (int no_w_id, int no_max_w_id, int >> > > >> > no_d_id, >> > > >> int >> > > >> > no_c_id, int no_o_ol_cnt, BigDecimal[] no_c_discount, >> > > >> > String[] >> > > >> no_c_last, >> > > >> > String[] no_c_credit, BigDecimal[] no_d_tax, BigDecimal[] >> > > >> > no_w_tax, >> > > >> int[] >> > > >> > no_d_next_o_id, Timestamp tstamp, ResultSet[] opres) throws >> > > >> > SQLException >> > > >> > >> > > >> > Also If I disable "*Build Java Stored Procedures Locally*" >> > > >> > from hammerdb I get: >> > > >> > >> > > >> > Vuser 1:CREATING TPCC INDEXES Vuser 1:Creating Index >> > > >> > CUSTOMER_I2... >> > > >> > Vuser 1:Failed to Create Index >> > > >> > >> > > >> > -- >> > > >> > And in the end, it's not the years in your life that count. >> > > >> > It's the >> > > >> life >> > > >> > in your years. >> > > >> > >> > > >> >> > > > >> > > > >> > > > >> > > > -- >> > > > And in the end, it's not the years in your life that count. >> > > > It's the life in your years. >> > > > >> > > >> > > >> > > >> > > -- >> > > And in the end, it's not the years in your life that count. It's >> > > the life in your years. >> > > >> > >> > >> > >> > -- >> > And in the end, it's not the years in your life that count. It's >> > the life in your years. >> > >> >> >> >> -- >> And in the end, it's not the years in your life that count. It's the >> life in your years. >> > > > > -- > And in the end, it's not the years in your life that count. It's the life > in your years. > -- And in the end, it's not the years in your life that count. It's the life in your years.
