Re: Fast producer, slow consumer with spooling to disk?

[Rob Davies]

Hi Albert,

from the latest in SVN it's possible to configure the cursor used for  
a topic subscriber - which by default is file based. If you use a VM  
based cursor, as messages aren't removed from memory (to spool to  
disk) - the producer should block. You can configure the cursor type  
using destination policies: http://incubator.apache.org/activemq/ 
xbean-xml-reference-41.html#XBeanXMLReference4.1-policyEntryelement


An example is:

<beans>
  <bean  
class="org.springframework.beans.factory.config.PropertyPlaceholderConfi 
gurer"/>

  <broker persistent="false" xmlns="http://activemq.org/config/1.0";>

    <!--  lets define the dispatch policy -->
    <destinationPolicy>
      <policyMap>
        <policyEntries>
          <policyEntry topic="org.apache.>">
            <dispatchPolicy>
              <strictOrderDispatchPolicy />
            </dispatchPolicy>
            <deadLetterStrategy>
              <individualDeadLetterStrategy  topicPrefix="Test.DLQ." />
            </deadLetterStrategy>
            <pendingSubscriberPolicy>
                <vmCursor />
            </pendingSubscriberPolicy>
          </policyEntry>

          <policyEntry queue="org.apache.>">
            <dispatchPolicy>
              <strictOrderDispatchPolicy />
            </dispatchPolicy>
            <deadLetterStrategy>
              <individualDeadLetterStrategy queuePrefix="Test.DLQ."/>
            </deadLetterStrategy>
            <pendingQueuePolicy>
                <vmQueueCursor />
            </pendingQueuePolicy>
          </policyEntry>

        </policyEntries>
      </policyMap>
    </destinationPolicy>
  </broker>

</beans>

where the all topic subscribers to destinations starting with  
org.apache will have a  VMCursor for their pending messages.

hope that helps

cheers,

Rob

On 26 Jan 2007, at 17:53, Albert Strasheim wrote:

Hey Rob,

On Fri, 26 Jan 2007, Rob Davies wrote:

Hi Albert,
I think you could need to set the prefetch limit as well - it could
be the clients running out of memory.

Thanks, I'll experiment with the prefetch limit a bit.

I agree we need to allow for configuration of blocking a producer
whilst waiting for space - be that disk or memory - this may take a
few more days to get right :(

Will you have time to look into this? Could you provide some hints  
as to
where one should start looking to implement this feature.

Thanks for the quick reply and the patches so far!

Cheers,

Albert