Harm

We can reproduce this issue.. I will get back on this.. could you raise a JIRA issue for this?

asankha

Harm Verhagen wrote:
Asankha,

I have the ESB hanging now.
kill -3 <PID> doesn't give anything.. where should I see the thread dump ?
(I checked stdout/stderr of esb, I checked wso2esb/logs/*)


Regards,
Harm

2008/9/25 Asankha C. Perera <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>

    Harm

    Can you issue a "kill - 3 <pid_of_esb>" and get me a thread dump?

    Evanthika, can you test this scenario tomorrow to try to reproduce
    this issue?

    asankha

    Harm Verhagen wrote:
    Hoi,

    I'm experiencing something that looks like a deadlock in wso2esb.

    I have a sequence with a number of database accesses. (using mysql).
    Basicaly I want to store every message in a database.
    When stress testing the esb sometimes locks up, not processing
    any messages anymore.
    Only stop/start seems to help
    The logging seems to indicate its related to the databasepool
    management.


    *ESB logging*
    2008-09-25 17:31:56,199 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBReportMediator End : DBReport mediator
    2008-09-25 17:31:56,199 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Start : DBLookup mediator
    2008-09-25 17:31:56,199 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Getting a connection
    from DataSource jdbc:mysql://localhost:3306/esb and preparing
    statement : select id from transaction_uuid where uuid = ?
    2008-09-25 17:31:56,200 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Setting as parameter
    : 1 value : urn:uuid:953256D70E1C3B31A21222356730663 as JDBC Type
    : 12(see java.sql.Types for valid types)
    2008-09-25 17:31:56,200 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Successfully prepared
    statement : select id from transaction_uuid where uuid = ?
    against DataSource : jdbc:mysql://localhost:3306/esb
    2008-09-25 17:31:56,200 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Processing the first
    row returned : select id from transaction_uuid where uuid = ?
    2008-09-25 17:31:56,200 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator Column : 1 returned
    value : 6004 Setting this as the message property : transaction_id
    2008-09-25 17:31:56,201 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBLookupMediator End : DBLookup mediator
    2008-09-25 17:31:56,201 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBReportMediator Start : DBReport mediator
    2008-09-25 17:31:56,201 [127.0.0.1-vloeki_v01]
    [HttpServerWorker-5] DEBUG DBReportMediator Getting a connection
    from DataSource jdbc:mysql://localhost:3306/esb and preparing
    statement : delete from transaction_uuid where id = ?
    ...
    Here it hangs.. I can get more debug prints out of it by pressing
    Configuration -> save   but _no_ messages are processed.
    When the ESB hangs, Mysql is still up & running fine, I can
    happily view the database with mysql querybrowser, make new
    connections etc.

    Any Idea what goes on ?

    *how to reproduce*
    issue soap request with a stress test tool, 5 threads
    simultaneously doing a medium load (30-60 request/second)
    Its pretty reproducable, after a few thousens requests I get wso2
    to hang. so withing a few minutues.

    *The sequence:*
      <syn:sequence name="md_seq">
            <syn:property name="Authorization" value="Basic
    ZnJlZGRpZTpoZWluZWtlbg==" scope="transport"/>
            <syn:log level="headers" separator=",">
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="type" expression="get-property('axis2','messageType')"/>
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="test" expression="get-property('transport', 'Authorization')"/>
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="RemoteIP" expression="get-property('axis2','REMOTE_ADDR')"/>
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="time" expression="get-property('SYSTEM_DATE', 'yyyy-MM-dd
    HH:mm:ss z')"/>
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="uid" expression="get-property('transport', 'MessageID')"/>
                <syn:property name="text" value="XXXXXX"/>
                <syn:property
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    name="uid" expression="get-property('MessageID')"/>
            </syn:log>
            <syn:dblookup>
                <syn:connection>
                    <syn:pool>
                        <syn:password>esb</syn:password>
                        <syn:user>esb</syn:user>
<syn:url>jdbc:mysql://localhost:3306/esb</syn:url>
                        <syn:driver>com.mysql.jdbc.Driver</syn:driver>
                        <syn:property name="poolstatements"
    value="true"/>
                    </syn:pool>
                </syn:connection>
                <syn:statement>
                    <syn:sql><![CDATA[select * from customer c where
    token = '00000000-00';]]></syn:sql>
                    <syn:result name="customer_id" column="id"/>
                </syn:statement>
            </syn:dblookup>
            <syn:dbreport>
                <syn:connection>
                    <syn:pool>
                        <syn:password>esb</syn:password>
                        <syn:user>esb</syn:user>
<syn:url>jdbc:mysql://localhost:3306/esb</syn:url>
                        <syn:driver>com.mysql.jdbc.Driver</syn:driver>
                        <syn:property name="autocommit" value="false"/>
                    </syn:pool>
                </syn:connection>
                <syn:statement>
                    <syn:sql><![CDATA[insert into transaction (
    customer_id, token, ip, transaction_type ) values ( ?, ?, ?,1
    );]]></syn:sql>
                    <syn:parameter
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    expression="get-property('customer_id')" type="INTEGER"/>
                    <syn:parameter value="A0000000-02BBDDE"
    type="VARCHAR"/>
                    <syn:parameter
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    expression="get-property('axis2','REMOTE_ADDR')" type="VARCHAR"/>
                </syn:statement>
                <syn:statement>
                    <syn:sql><![CDATA[insert into
    transaction_uuid(id, uuid) values(LAST_INSERT_ID(),?)]]></syn:sql>
                    <syn:parameter
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    expression="get-property('MessageID')" type="VARCHAR"/>
                </syn:statement>
            </syn:dbreport>
            <syn:dblookup>
                <syn:connection>
                    <syn:pool>
                        <syn:password>esb</syn:password>
                        <syn:user>esb</syn:user>
<syn:url>jdbc:mysql://localhost:3306/esb</syn:url>
                        <syn:driver>com.mysql.jdbc.Driver</syn:driver>
                    </syn:pool>
                </syn:connection>
                <syn:statement>
                    <syn:sql><![CDATA[select id from transaction_uuid
    where uuid = ?]]></syn:sql>
                    <syn:parameter
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    expression="get-property('MessageID')" type="VARCHAR"/>
                    <syn:result name="transaction_id" column="1"/>
                </syn:statement>
            </syn:dblookup>
            <syn:dbreport>
                <syn:connection>
                    <syn:pool>
                        <syn:password>esb</syn:password>
                        <syn:user>esb</syn:user>
<syn:url>jdbc:mysql://localhost:3306/esb</syn:url>
                        <syn:driver>com.mysql.jdbc.Driver</syn:driver>
                        <syn:property name="autocommit" value="false"/>
                    </syn:pool>
                </syn:connection>
                <syn:statement>
                    <syn:sql><![CDATA[delete from transaction_uuid
    where id = ?]]></syn:sql>
                    <syn:parameter
    xmlns:ns1="http://org.apache.synapse/xsd";
    xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
    expression="get-property('transaction_id')" type="INTEGER"/>
                </syn:statement>
            </syn:dbreport>
            <syn:send>
                <syn:endpoint>
                    <syn:address
    uri="http://127.0.0.1:8180/boost-platform/services/messageDelivery"/>
                </syn:endpoint>
            </syn:send>
        </syn:sequence>



    *version
    *OS: debian
    wso2 v1.7.1
    mysql 5.0
    mysql-connector-java-5.0.4
    ------------------------------------------------------------------------

    _______________________________________________
    Esb-java-user mailing list
    [email protected] <mailto:[email protected]>
    http://mailman.wso2.org/cgi-bin/mailman/listinfo/esb-java-user

    _______________________________________________
    Esb-java-user mailing list
    [email protected] <mailto:[email protected]>
    http://mailman.wso2.org/cgi-bin/mailman/listinfo/esb-java-user


------------------------------------------------------------------------

_______________________________________________
Esb-java-user mailing list
[email protected]
http://mailman.wso2.org/cgi-bin/mailman/listinfo/esb-java-user
_______________________________________________
Esb-java-user mailing list
[email protected]
http://mailman.wso2.org/cgi-bin/mailman/listinfo/esb-java-user

Reply via email to