Hi all (especially Bela :-) ),
I've spent the following week trying to get TreeCache setup across a two node
cluster on JBoss. The two nodes are setup to access a single oracle database.
We are using hibernate 2.1 on the two nodes and we wanted to use TreeCache as
the second level cache in hibernate.
We are trying to meet the following requirements:
A) Cache persistent hibernate objects on each JVM
B) Maintain cache consistency across the cluster (i.e. updates to database from
one node are propagated to the caches on the other nodes, or at a minimum, the
caches get invalidated)
C) Use SERIALIZABLE transaction isolation level
I'll attach our existing configuration files at the bottom, but initially I'd
like to discribe the problems that we encountered trying to achieve our goals:
Problem 1: we couldn't get simple replication between caches across the two
nodes to work properly. The second node when it started up fetched the
transient tree status from the first node (using the RpcDelegatingCacheLoader).
It then started a read-only transaction (Hibernate flush-mode set to NEVER).
Unfortunately, it retrieved everything from the database (instead of the
replicated second-level cache ) probably becasue the Hibernate query cache was
empty. When it was committing the transaction, the reader blocked indefinately
(we got no exception). Looking at the stack of the reader thread we think it
was blocking on:
RpcDelegatingCacheLoader.delegatePut(...)
ie, it was perhaps trying to replicate the local puts.
Problem 2: as a consequence of Problem 1, the first node times out and
generates the following exception:
| 2005-07-20 12:34:33,087 ERROR [org.jgroups.blocks.RpcDispatcher] failed
invoking method
| org.jboss.util.NestedRuntimeException: - nested throwable:
(java.lang.InterruptedException)
| at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:3121)
| at org.jboss.cache.TreeCache.put(TreeCache.java:1762)
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
| at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
| at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
| at java.lang.reflect.Method.invoke(Method.java:585)
| at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:286)
| at org.jgroups.blocks.RpcDispatcher.handle(RpcDispatcher.java:236)
| at
org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:618)
| at
org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:515)
| at
org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:326)
| at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUp(MessageDispatcher.java:734)
| at
org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.access$300(MessageDispatcher.java:566)
| at
org.jgroups.blocks.MessageDispatcher$1.run(MessageDispatcher.java:703)
| at java.lang.Thread.run(Thread.java:595)
| Caused by: java.lang.InterruptedException
| at java.lang.Object.wait(Native Method)
| at
EDU.oswego.cs.dl.util.concurrent.QueuedSemaphore$WaitQueue$WaitNode.doTimedWait(QueuedSemaphore.java:123)
| at
EDU.oswego.cs.dl.util.concurrent.QueuedSemaphore.attempt(QueuedSemaphore.java:47)
| at
org.jboss.cache.lock.IdentityLock.acquireReadLock(IdentityLock.java:206)
| at org.jboss.cache.Node.acquireReadLock(Node.java:499)
| at org.jboss.cache.Node.acquire(Node.java:471)
| at
org.jboss.cache.interceptors.LockInterceptor.lock(LockInterceptor.java:245)
| at
org.jboss.cache.interceptors.LockInterceptor.invoke(LockInterceptor.java:153)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:41)
| at
org.jboss.cache.interceptors.CacheLoaderInterceptor.invoke(CacheLoaderInterceptor.java:120)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:41)
| at
org.jboss.cache.interceptors.UnlockInterceptor.invoke(UnlockInterceptor.java:35)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:41)
| at
org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:54)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:41)
| at
org.jboss.cache.interceptors.CacheStoreInterceptor.invoke(CacheStoreInterceptor.java:104)
| at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:3116)
| ... 14 more
|
Here are our configuration files. Maybe we are missing some stuff with our
configuration. Really, any help would be so much appreciated as we're at our
wits end with this!
TreeCache.xml
| <?xml version="1.0" encoding="UTF-8"?>
|
| <!-- =====================================================================
-->
| <!--
-->
| <!-- Sample TreeCache Service Configuration
-->
| <!--
-->
| <!-- =====================================================================
-->
|
| <server>
|
| <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>
|
|
| <!--
==================================================================== -->
| <!-- Defines TreeCache configuration
-->
| <!--
==================================================================== -->
|
| <mbean code="org.jboss.cache.TreeCache"
| name="jboss.cache:service=TreeCache">
|
| <depends>jboss:service=Naming</depends>
| <depends>jboss:service=TransactionManager</depends>
| <depends>jboss:service=ClientUserTransaction</depends>
|
| <!--
| Configure the TransactionManager
| -->
| <attribute
name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
|
| <!--
| Node isolation level : SERIALIZABLE
| REPEATABLE_READ (default)
| READ_COMMITTED
| READ_UNCOMMITTED
| NONE
| -->
| <attribute name="IsolationLevel">SERIALIZABLE</attribute>
|
| <!--
| Valid modes are LOCAL
| REPL_ASYNC
| REPL_SYNC
| -->
| <attribute name="CacheMode">REPL_SYNC</attribute>
|
| <!-- Name of cluster. Needs to be the same for all clusters, in
order
| to find each other
| -->
| <attribute name="ClusterName">TreeCache-Cluster</attribute>
|
| <!-- JGroups protocol stack properties. Can also be a URL,
| e.g. file:/home/bela/default.xml
| <attribute name="ClusterProperties"></attribute>
| -->
|
| <attribute name="ClusterConfig">
| <config>
| <!-- UDP: if you have a multihomed machine,
| set the bind_addr attribute to the appropriate NIC IP
address -->
| <!-- UDP: On Windows machines, because of the media sense
feature
| being broken with multicast (even after disabling media
sense)
| set the loopback attribute to true -->
| <UDP mcast_addr="228.1.3.160" mcast_port="45567"
bind_addr="192.120.240.160"
| ip_ttl="64" ip_mcast="true"
| mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
| ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
| loopback="false"/>
| <PING timeout="2000" num_initial_members="3"
| up_thread="false" down_thread="false"/>
| <MERGE2 min_interval="10000" max_interval="20000"/>
| <FD shun="true" up_thread="true" down_thread="true"/>
| <VERIFY_SUSPECT timeout="1500"
| up_thread="false" down_thread="false"/>
| <pbcast.NAKACK gc_lag="50"
retransmit_timeout="600,1200,2400,4800"
| up_thread="false" down_thread="false"/>
| <pbcast.STABLE desired_avg_gossip="20000"
| up_thread="false" down_thread="false"/>
| <UNICAST timeout="600,1200,2400" window_size="100"
min_threshold="10"
| down_thread="false"/>
| <FRAG frag_size="8192"
| down_thread="false" up_thread="false"/>
| <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
| shun="true" print_local_addr="true"/>
| <pbcast.STATE_TRANSFER up_thread="false"
down_thread="false"/>
| </config>
| </attribute>
|
| <!--
| Max number of entries in the cache. If this is exceeded, the
| eviction policy will kick some entries out in order to make
| more room
| -->
| <attribute name="MaxCapacity">20000</attribute>
|
| <!--
| The max amount of time (in milliseconds) we wait until the
| initial state (ie. the contents of the cache) are retrieved from
| existing members in a clustered environment
| -->
| <attribute name="InitialStateRetrievalTimeout">20000</attribute>
|
| <!--
| Number of milliseconds to wait until all responses for a
| synchronous call have been received.
| -->
| <attribute name="SyncReplTimeout">10000</attribute>
|
| <!-- Max number of milliseconds to wait for a lock acquisition
(15000 by default) -->
| <attribute name="LockAcquisitionTimeout">400000</attribute>
|
| <!-- Max number of milliseconds we hold a lock (not currently
| implemented) -->
| <attribute name="LockLeaseTimeout">600000</attribute>
|
| <!-- Name of the eviction policy class. Not supported now. -->
| <attribute name="EvictionPolicyClass"></attribute>
|
| <attribute name="FetchStateOnStartup">true</attribute>
| <attribute
name="CacheLoaderClass">org.jboss.cache.loader.RpcDelegatingCacheLoader</attribute>
| <attribute name="CacheLoaderShared">true</attribute>
| <attribute
name="CacheLoaderFetchTransientState">true</attribute>
| <attribute
name="CacheLoaderFetchPersistentState">false</attribute>
|
| </mbean>
|
| </server>
|
next, hibernate configuration xml file
| <?xml version='1.0' encoding='utf-8'?>
| <!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate
Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-2.0.dtd";>
| <hibernate-configuration>
| <session-factory>
|
| <property name="query.substitutions">true 1, false 0, yes 'Y', no
'N'</property>
|
| <property
name="dialect">net.sf.hibernate.dialect.OracleDialect</property>
| <property name="connection.datasource">java:/jdbc/inetdriver</property>
|
| <property name="connection.pool_size">10</property>
| <property name="statement_cache.size">100</property>
|
| <!-- Use JDBC Transaction factory for a local (per-JVM) application
| <property
name="transaction.factory_class">net.sf.hibernate.transaction.JDBCTransactionFactory</property>
| -->
|
| <!-- Use JTA Trancation factory for a distributed (inter-JVM)
application -->
| <property
name="transaction.factory_class">net.sf.hibernate.transaction.JTATransactionFactory</property>
|
| <!-- Added the JBoss Transaction Manager Lookup class (PH) for the
distributed (inter-JVM) application -->
| <property
name="transaction.manager_lookup_class">net.sf.hibernate.transaction.JBossTransactionManagerLookup</property>
|
| <!-- The EHCache is a per-JVM (local) second-level cache
| <property
name="cache.provider_class">net.sf.ehcache.hibernate.Provider</property>
| -->
| <!-- JBoss TreeCache is a distributed (inter-JVM) transactional cache
with replication -->
| <property
name="cache.provider_class">net.sf.hibernate.cache.TreeCacheProvider</property>
|
| <!-- PH: we must use query cache in order to use the second-level
cache! -->
| <property name="cache.use_query_cache">true</property>
|
| <property name="cache.use_minimal_puts">false</property>
|
| <!-- Magic number 8 is serializable, 2 is read_committed. NOTE: this
doesn't work if JBoss is managing the JDBC datasource -
| in that case you need to set it in the Oracle-ds.xml file, so
the below doesn't work. -->
| <property name="connection.isolation">8</property>
|
| <property name="show_sql">true</property>
| <property name="jdbc.fetch_size">5000</property>
| <property name="max_fetch_depth">8</property>
| <property name="jdbc.use_get_generated_keys">true</property>
| <property name="jdbc.batch_size">1000</property>
| <property name="jdbc.use_scrollable_resultset">true</property>
| <property name="jdbc.use_streams_for_binary">true</property>
| <property name="use_outer_join">true</property>
| <property name="cglib.use_reflection_optimizer">true</property>
|
| <!-- mapping files -->
| <mapping
resource="com/capetechnologies/inetdriver/domain/Archive.hbm.xml"/>
| <mapping
resource="com/capetechnologies/inetdriver/domain/DataPoint.hbm.xml"/>
| <mapping
resource="com/capetechnologies/inetdriver/domain/MonitoredProperty.hbm.xml"/>
| </session-factory>
| </hibernate-configuration>
|
this is the relevant transaction section from our conf/jboss-service.xml file
| <!--
==================================================================== -->
| <!-- Transactions
-->
| <!--
==================================================================== -->
| <!-- The configurable Xid factory. For use with Oracle, set pad to true
-->
| <mbean code="org.jboss.tm.XidFactory"
| name="jboss:service=XidFactory">
| <!--attribute name="Pad">true</attribute-->
| </mbean>
|
| <!--
| | The fast in-memory transaction manager.
| -->
| <mbean code="org.jboss.tm.TransactionManagerService"
| name="jboss:service=TransactionManager"
| xmbean-dd="resource:xmdesc/TransactionManagerService-xmbean.xml">
| <attribute name="TransactionTimeout">300</attribute>
|
| <depends
optional-attribute-name="XidFactory">jboss:service=XidFactory</depends>
| </mbean>
|
| <!-- EAR deployer, remove if you are not using Web layers -->
| <mbean code="org.jboss.deployment.EARDeployer"
name="jboss.j2ee:service=EARDeployer">
| </mbean>
|
|
| <!--
| | UserTransaction support.
| -->
| <mbean code="org.jboss.tm.usertx.server.ClientUserTransactionService"
| name="jboss:service=ClientUserTransaction"
| xmbean-dd="resource:xmdesc/ClientUserTransaction-xmbean.xml">
| <depends>
| <mbean code="org.jboss.invocation.jrmp.server.JRMPProxyFactory"
|
name="jboss:service=proxyFactory,target=ClientUserTransactionFactory">
| <attribute
name="InvokerName">jboss:service=invoker,type=jrmp</attribute>
| <attribute
name="TargetName">jboss:service=ClientUserTransaction</attribute>
| <attribute
name="JndiName">UserTransactionSessionFactory</attribute>
| <attribute
name="ExportedInterface">org.jboss.tm.usertx.interfaces.UserTransactionSessionFactory</attribute>
| <attribute name="ClientInterceptors">
| <interceptors>
|
<interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
|
<interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
| </interceptors>
| </attribute>
| <depends>jboss:service=invoker,type=jrmp</depends>
| </mbean>
| </depends>
| <depends optional-attribute-name="TxProxyName">
| <mbean code="org.jboss.invocation.jrmp.server.JRMPProxyFactory"
| name="jboss:service=proxyFactory,target=ClientUserTransaction">
| <attribute
name="InvokerName">jboss:service=invoker,type=jrmp</attribute>
| <attribute
name="TargetName">jboss:service=ClientUserTransaction</attribute>
| <attribute name="JndiName"></attribute>
| <attribute
name="ExportedInterface">org.jboss.tm.usertx.interfaces.UserTransactionSession</attribute>
| <attribute name="ClientInterceptors">
| <interceptors>
|
<interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
|
<interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
| </interceptors>
| </attribute>
| <depends>jboss:service=invoker,type=jrmp</depends>
| </mbean>
| </depends>
|
| </mbean>
|
thanks a million for your help!!!!!
Brian
View the original post :
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3885688#3885688
Reply to the post :
http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3885688
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
JBoss-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jboss-user
