yeah, that could be confusing:) the calculation is projecting how much
product is available, each day, for the next 42 days (6 weeks). I
initially configured the system saga to process each day's calculation
as a separate message (42 messages).
public void Consume(Message message)
{
IsComplete = message.Date < State.BeganOn.AddDays(43);
if (IsComplete) return;
calculator.Calculate(message.Date);
Bus.Send(new Message{Date = message.Date.AddDays(1); });
}
I now use a loop in the method body of the consumer to loop through
all 42 days of calculations in 1 message
public void Consume(Message message)
{
for(var i = 0; i < 43; i++)
{
calculator.Calculate(message.Date.AddDays(i);
}
}
On Sep 9, 11:21 am, Ayende Rahien <[email protected]> wrote:
> Jason,I am happy that you found a solution.
> But I would like to know what you mean, 42 days of messaging.
>
> On Wed, Sep 9, 2009 at 6:08 PM, Jason Meckley <[email protected]>wrote:
>
>
>
> > I did some more testing today and found that processing 42 days with
> > individual messages was pegging the development servers CPU for the
> > duration of the saga. the saga was taking minutes to complete.
> > probably with all the read/writes from PHT and database. I abandoned
> > the saga altogether and preformed the 42 day loop within a single
> > consume(message) method. This had a very positive effect across the
> > entire system.
>
> > 1. the load was lessened on the server as a whole
> > 2. the duration of the load was drastically shorter
> > 3. all 42 days worth of calculations complete faster
> > 4. No write locks with saga persister because I'm not using sagas
>
> > I haven't run this through NHProf, but I'm sure it will yell about 42
> > days worth of queries. with a HiLo id scheme, multi-queries and
> > batched writes I can't think of any more optimizations.
>
> > All in all this was a good learning experience about messaging and an
> > introduction into multithreading. I'm still novice with deadlocking,
> > but I sort of know why/how it's happening. I really like how messaging
> > can decouple background work from the UI with "minimal" effort. The
> > actual code is minimal, understanding the architecture is more
> > complex.
>
> > RSB is great and I can see this opening huge possibilities for my
> > projects moving forward!
>
> > On Sep 8, 5:37 pm, Ayende Rahien <[email protected]> wrote:
> > > Jason,That is pretty much for the same reason as before.
> > > Although I would like to see if you can get a repro, I'll see if I can
> > > reduce contention in the PHT or RQ.
>
> > > On Wed, Sep 9, 2009 at 12:33 AM, Jason Meckley <[email protected]
> > >wrote:
>
> > > > spent the day researching dead locks and experimenting with isolation
> > > > levels. One thing I did come to realize is there are no indexes on my
> > > > tables. I applied some indexing to the tables in question. the 2
> > > > biggest impacts I see are: the responsiveness of the webpages is much
> > > > better and the dead lock messages have decreased significantly.
>
> > > > I am receiving an EsentErrorException: Error WriteConflict
> > > > (JET_errWriteConflict, Write lock failed due to outstanding write
> > > > lock). This may be caused by the nature of my saga. I was reading
> > > > another post (can't find the thread now). the users was getting write
> > > > conflicts because they where using PHT in a tight loop with messaging.
> > > > I'm doing something similar with my saga. when MessageX is consummed
> > > > the process performs selects and updates and then sends another
> > > > MessageX. after MessageX is consumed 42 times (6 weeks) the saga is
> > > > complete.
>
> > > > I figured this would be a better approach because it removes multiple
> > > > database calls in a loop and keeps the UI responsive. there is still a
> > > > loop (42 days) but the calculations are autonomous.
>
> > > > The Esent Exception didn't stop the processing, so that's good. I'll
> > > > keep digging into this more and stress the system again tomorrow.
>
> > > > On Sep 5, 5:56 am, Ayende Rahien <[email protected]> wrote:
> > > > > inline
>
> > > > > On Fri, Sep 4, 2009 at 11:29 PM, Jason Meckley <
> > [email protected]
> > > > >wrote:
>
> > > > > > I have a messaging system that runs calculations in the background
> > and
> > > > > > stores the results in the database. If I run on more than 1 thread
> > and
> > > > > > insert records through the web interface rapidly I will end up with
> > > > > > the follow 2 exceptions in my service log:
>
> > > > > > NHibernate.Event.Default.AbstractFlushingEventListener [(null)] -
> > > > > > Could not synchronize database state with session
> > > > > > NHibernate.HibernateException: An exception occurred when executing
> > > > > > batch queries ---> System.Data.SqlClient.SqlException: Transaction
> > > > > > (Process ID 51) was deadlocked on lock resources with another
> > process
> > > > > > and has been chosen as the deadlock victim. Rerun the transaction.
>
> > > > > > NHibernate.Impl.AbstractSessionImpl [(null)] - DTC transaction
> > prepre
> > > > > > phase failed
> > > > > > NHibernate.HibernateException: An exception occurred when executing
> > > > > > batch queries ---> System.Data.SqlClient.SqlException: Transaction
> > > > > > (Process ID 51) was deadlocked on lock resources with another
> > process
> > > > > > and has been chosen as the deadlock victim. Rerun the transaction.
>
> > > > > Yep, those are expected when you execute multiple threads that
> > conflict
> > > > with
> > > > > one another.
> > > > > With RSB, this will automatically retry the transaction, so the
> > action is
> > > > > actually being performed.
> > > > > You might need to look at transaction contention rate reduction
> > > > techniques
> > > > > or how to reduce deadlocks, but the basic problem is that you have
> > two
> > > > > concurrent transactions that each wants resources that the other has
> > > > > locked.
> > > > > Pay attention to the fact that by default we are running in RSB with
> > > > > serializable isolation level.
>
> > > > > > If I switch over to a single thread the problem appears to stop. I
> > > > > > think this makes sense to me. with >1 threads 2 threads attempt to
> > > > > > access the same record at the same time = dead lock.
>
> > > > > > the message tells me to re-run the transaction. how do I do that in
> > > > > > RSB? is there a Transaction.TryAgain(); method? I thought part of
> > the
> > > > > > advantage of messaging was parallel processing. If I only run on 1
> > > > > > thread is there any real benefit to messaging?
>
> > > > > Nope, your message will be retried automatically. If you will look at
> > the
> > > > > results you'll see that the action *is* being performed.
> > > > > That is one of the really nice things about messaging approach.
>
> > > > > > some other back ground on this procedure.
> > > > > > 1. in a NH post insert/update/delete event I send a message to
> > start
> > > > > > calculating. the service picks up this message and works its magic.
>
> > > > > 2. the isolation level for the windows service the RSB default,
>
> > > > > > serializable i believe. the isolation level for the website is read
> > > > > > committed.
> > > > > > 3. it appears that after the deadlock the current saga stops
> > > > > > altogether.
>
> > > > > That isn't right. Can you post the log from RSB for that? You should
> > see
> > > > the
> > > > > message retrying.
>
> > > > > > 4. right now I'm testing on an older box I'm pretty sure it's a
> > single
> > > > > > proc box with a gig of RAM.
> > > > > > 5. both the website and service are hitting the database. the
> > website
> > > > > > does CRUD for the immediate input. the service runs the
> > calculations
> > > > > > and CRUD against the database.
> > > > > > 6. the tables updated by the website are read by the service and
> > vice
> > > > > > versa. wouldn't think this would be an issue though.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Rhino Tools Dev" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/rhino-tools-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
