Re: [rt-users] RT not sending emails to owners for specific queue
The only difference between this and my other queues appears to be that whereas for all my other queues all requestors are in Groups, with quite a few rights, the requestors on this queue do not, and only have CreateTicket and ReplyToTicket from the Everyone group. I've got another Scrip that sends IM messages on correspond, and that doesn't work on that queue either for these external requestors. However if I put a reply on with my account it works Do you need ShowTicket or SeeQueue to get these scrips to work or something? Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 15:49, Justin Hayes wrote: I can't see anything applicable in the logs (either the debug log of the syslog) after that line. The template is the default global correspondence one. - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 15:03, Kevin Falcone wrote: On Mon, Oct 03, 2011 at 12:13:03PM +0100, Justin Hayes wrote: It appears to fire the emails for Scrips that are at the TransactionCreate stage, but my notify owner Scrip is at TransactionBatch. For a ticket in this queue I see the following line of logging: [Mon Oct 3 08:00:28 2011] [debug]: Found 2 scrips for TransactionBatch stage with applicable type(s) (/opt/rt_support.openbet .com/bin/../lib/RT/Scrips_Overlay.pm:370) But then nothing after that. At the least, it should tell you that those 2 weren't applicable in the logs. Check also that your Template for the Notify Owner scrip is valid in the queue where the scrip isn't working. -kevin On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
Re: [rt-users] RT not sending emails to owners for specific queue
I've switched to TransactionCreate for these Scrips and my emails are working now - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 4 Oct 2011, at 09:24, Justin Hayes wrote: The only difference between this and my other queues appears to be that whereas for all my other queues all requestors are in Groups, with quite a few rights, the requestors on this queue do not, and only have CreateTicket and ReplyToTicket from the Everyone group. I've got another Scrip that sends IM messages on correspond, and that doesn't work on that queue either for these external requestors. However if I put a reply on with my account it works Do you need ShowTicket or SeeQueue to get these scrips to work or something? Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 15:49, Justin Hayes wrote: I can't see anything applicable in the logs (either the debug log of the syslog) after that line. The template is the default global correspondence one. - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 15:03, Kevin Falcone wrote: On Mon, Oct 03, 2011 at 12:13:03PM +0100, Justin Hayes wrote: It appears to fire the emails for Scrips that are at the TransactionCreate stage, but my notify owner Scrip is at TransactionBatch. For a ticket in this queue I see the following line of logging: [Mon Oct 3 08:00:28 2011] [debug]: Found 2 scrips for TransactionBatch stage with applicable type(s) (/opt/rt_support.openbet .com/bin/../lib/RT/Scrips_Overlay.pm:370) But then nothing after that. At the least, it should tell you that those 2 weren't applicable in the logs. Check also that your Template for the Notify Owner scrip is valid in the queue where the scrip isn't working. -kevin On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
Re: [rt-users] RT not sending emails to owners for specific queue
Sorry have been on leave I've had debug logging enabled and turned on recording of outgoing mail so will see if anything shows up. Thanks, Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
Re: [rt-users] RT not sending emails to owners for specific queue
It appears to fire the emails for Scrips that are at the TransactionCreate stage, but my notify owner Scrip is at TransactionBatch. For a ticket in this queue I see the following line of logging: [Mon Oct 3 08:00:28 2011] [debug]: Found 2 scrips for TransactionBatch stage with applicable type(s) (/opt/rt_support.openbet .com/bin/../lib/RT/Scrips_Overlay.pm:370) But then nothing after that. Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
Re: [rt-users] RT not sending emails to owners for specific queue
The Scrip in question is definitely working on other queues: [Mon Oct 3 08:47:39 2011] [debug]: Committing scrip #24 on txn #2130835 of ticket #68440 (/opt/rt_support.openbet.com/bin/../lib/RT/Scrips_Overlay.pm:190) [Mon Oct 3 08:47:39 2011] [debug]: Calling SetRecipientDigests for transaction RT::Transaction=HASH(0x7f3d85ebed08), id 2130835 (/opt/rt_support.openbet.com/bin/../local/lib/RT/Action/SendEmail.pm:640) - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 12:13, Justin Hayes wrote: It appears to fire the emails for Scrips that are at the TransactionCreate stage, but my notify owner Scrip is at TransactionBatch. For a ticket in this queue I see the following line of logging: [Mon Oct 3 08:00:28 2011] [debug]: Found 2 scrips for TransactionBatch stage with applicable type(s) (/opt/rt_support.openbet .com/bin/../lib/RT/Scrips_Overlay.pm:370) But then nothing after that. Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
Re: [rt-users] RT not sending emails to owners for specific queue
I can't see anything applicable in the logs (either the debug log of the syslog) after that line. The template is the default global correspondence one. - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 On 3 Oct 2011, at 15:03, Kevin Falcone wrote: On Mon, Oct 03, 2011 at 12:13:03PM +0100, Justin Hayes wrote: It appears to fire the emails for Scrips that are at the TransactionCreate stage, but my notify owner Scrip is at TransactionBatch. For a ticket in this queue I see the following line of logging: [Mon Oct 3 08:00:28 2011] [debug]: Found 2 scrips for TransactionBatch stage with applicable type(s) (/opt/rt_support.openbet .com/bin/../lib/RT/Scrips_Overlay.pm:370) But then nothing after that. At the least, it should tell you that those 2 weren't applicable in the logs. Check also that your Template for the Notify Owner scrip is valid in the queue where the scrip isn't working. -kevin On 30 Sep 2011, at 16:13, Kevin Falcone wrote: On Fri, Sep 30, 2011 at 09:27:01AM +0100, Justin Hayes wrote: RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? And what do your debug logs say happens on correspondence? -kevin RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA ˜ October 18 19, 2011 * Washington DC, USA ˜ October 31 November 1, 2011 * Melbourne VIC, Australia ˜ November 28 29, 2011 * Barcelona, Spain ˜ November 28 29, 2011 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
[rt-users] RT not sending emails to owners for specific queue
RT 3.6.8 I've got a queue that we're using to track emails from recruiters. Because we want them to be able to create tickets and reply to them, but not actually see the tickets themselves, on the queue I've set it up so that the Everyone group has the CreateTicket and ReplyToTicket rights. The recruiters themselves have accounts which are enabled, and allowed to be assigned rights. They are not in any groups however so have no directly assigned rights. I have the standard scrip that emails owners on ticket reply. This works fine for all other queues. However on this new queue none of my owners are getting emails when the requestor replies to the ticket via email. The reply is added to the ticket (as they have the ReplyToTicket) right, so surely it should email out as per the scrip? Any suggestions welcome. Justin - Justin Hayes OpenBet Head of Support justin.ha...@openbet.com +44 (0)20 8987 0715 +44 (0)7808 772 059 RT Training Sessions (http://bestpractical.com/services/training.html) * San Francisco, CA, USA October 18 19, 2011 * Washington DC, USA October 31 November 1, 2011 * Melbourne VIC, Australia November 28 29, 2011 * Barcelona, Spain November 28 29, 2011
[rt-users] New tickets - link creation not working
I'm running 3.8.8 and if I click on one of the 'create' links in the Links panel to create a linked ticket (refers to, depends on etc) I get a new ticket screen with a URL like this: https://domain/Ticket/Create.html?Queue=1CloneTicket=56932DependsOn-new=56932 However when I submit the page to create the new ticket no link is created. I'm sure this used to work, and now doesn't. I've turned debug logging on and can't see any errors to explain it. Anyone got any ideas on where to look or what might be wrong? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com
Re: [rt-users] Content searching takes a long time and runs multiple queries
Thanks Ken. I certainly prefer the idea of using inbuilt DB server functionality to improve performance than an external tool hooked in. I'll probably look into a PostgreSQL migration first and see where that takes me. I hope it's *just* a case of creating the schema by running a 3.8.8 install against it, dumping the data and loading it in ;) Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 10 Dec 2010, at 14:46, Kenneth Marshall wrote: On Fri, Dec 10, 2010 at 08:26:58AM +, Justin Hayes wrote: Any view on which is faster Jesse (postgres or mysql/sphinx)? Also how much faster than the old Content search are we talking? Orders of magnitude, or just 'faster'? Thanks again, Justin Hi Justin, I have not benchmarked the MySQL/Sphinx relative to the PostgreSQL implementation but the few I have found using a search showed their performance to be pretty comparable. I have not had a chance to look at how RT integrates with Sphinx, but when I wanted to test it with a PostgreSQL database, it did not keep its indexes up realtime, but required a periodic job to run for updates and periodic reindexes. The PostgreSQL fulltext indexes are kept in sync at all times. This helps avoid the the ticket is there but the search did not find it syndrome. As far as the speed with the fulltext indexing versus no fulltext indexing, a sample search that I ran while testing took 20 minutes for the table scan versus a couple of seconds for the search with the index support, 600X faster. Of course, the bigger win is that your I/O system is not tapped out while you are searching in the content and you database scales much, much more gracefully. Cheers, Ken - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 8 Dec 2010, at 16:48, Jesse Vincent wrote: On Wed 8.Dec'10 at 16:44:57 +, Justin Hayes wrote: I was about to say that should have been 3.9 :) Been a while since I checked out bestpractical.com. Just saw the post about the dev release, and looks like things are going well which is great news. The only downside is the time needed to migrate to postgres, Or you could try the sphinx-based fts for mysql that's baked in and then the time it's going to take to port all my custom code into the new version. But that's my problem, and one I have every time I upgrade, so can't complain!
Re: [rt-users] Content searching takes a long time and runs multiple queries
Any view on which is faster Jesse (postgres or mysql/sphinx)? Also how much faster than the old Content search are we talking? Orders of magnitude, or just 'faster'? Thanks again, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 8 Dec 2010, at 16:48, Jesse Vincent wrote: On Wed 8.Dec'10 at 16:44:57 +, Justin Hayes wrote: I was about to say that should have been 3.9 :) Been a while since I checked out bestpractical.com. Just saw the post about the dev release, and looks like things are going well which is great news. The only downside is the time needed to migrate to postgres, Or you could try the sphinx-based fts for mysql that's baked in and then the time it's going to take to port all my custom code into the new version. But that's my problem, and one I have every time I upgrade, so can't complain!
Re: [rt-users] Content searching takes a long time and runs multiple queries
Hmm we were looking at sphinx. Would you suggest plugging that in rather than migrating to postgres (which we're also familiar with)? Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Dec 2010, at 19:40, Kenneth Marshall wrote: Hi Justin, In the wiki, there are fulltext index modifications for Oracle and PostgreSQL. I based the PostgreSQL version on the Oracle version and we use it here. It works very well indeed. It looks like the pre version rt-4, a.k.a rt-3.9.6 support Oracle and PostgreSQL using their fulltext support and MySQL using sphinx, pretty cool. Regards, Ken On Tue, Dec 07, 2010 at 05:49:12PM +, Justin Hayes wrote: Hi Ken, I was just thinking the same about the counts - it has to do that for pagination. Though I guess it could have been written to run 1 query for all the data, and just display the first 50 etc. Which DB backend would work faster? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Dec 2010, at 17:40, Kenneth Marshall wrote: You need to use a DB backend that supports fulltext indexing for content searchs to be fast. The actual query that you stated runs quickly, is only for the first 50 tickets. I do agree that running the same count() query twice for the same search is sub-optimal. I do not see how you could avoid the count query totally if you are paginating the results. Cheers, Ken On Tue, Dec 07, 2010 at 05:31:17PM +, Justin Hayes wrote: Guys, Searching for ticket content takes forever. I've done a bit of digging and for a single search in one of my queues over the last year, RT spawned 3 separate queries. 2 are counts (which appear to be identical), and 1 gets the actual content. Is there anyway round this? Losing loads of time just to get counts seems rather counter-productive? The final select was actually pretty quick. I've added the queries below. Many thanks, Justin # Time: 101207 17:24:09 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 57.722237 Lock_time: 0.000183 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742649; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:38 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 28.780620 Lock_time: 0.000510 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742678; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:42 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 4.492875 Lock_time: 0.000175 Rows_sent: 50 Rows_examined: 100799 SET timestamp=1291742682; SELECT DISTINCT main.* FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id) ORDER BY main.id ASC LIMIT 50; - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com
Re: [rt-users] Content searching takes a long time and runs multiple queries
I hadn't actually realised it was possible to get hold of 4.9, though if that's beta I guess I'd want to wait. Postgres might be the way to go then. Will have to look into how difficult a migration would be. http://requesttracker.wikia.com/wiki/PostgreSQLFullText Is that the relevant wiki page for getting the searching to run fast Ken? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 8 Dec 2010, at 13:54, Kenneth Marshall wrote: Given that you are familiar with PostgreSQL already, I would use it because the current versions of RT support the fulltext indexing already and you have fewer moving pieces. If you are already running the 4.9.x series, then you could certainly test the sphinx integration. Cheers, Ken On Wed, Dec 08, 2010 at 09:55:06AM +, Justin Hayes wrote: Hmm we were looking at sphinx. Would you suggest plugging that in rather than migrating to postgres (which we're also familiar with)? Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Dec 2010, at 19:40, Kenneth Marshall wrote: Hi Justin, In the wiki, there are fulltext index modifications for Oracle and PostgreSQL. I based the PostgreSQL version on the Oracle version and we use it here. It works very well indeed. It looks like the pre version rt-4, a.k.a rt-3.9.6 support Oracle and PostgreSQL using their fulltext support and MySQL using sphinx, pretty cool. Regards, Ken On Tue, Dec 07, 2010 at 05:49:12PM +, Justin Hayes wrote: Hi Ken, I was just thinking the same about the counts - it has to do that for pagination. Though I guess it could have been written to run 1 query for all the data, and just display the first 50 etc. Which DB backend would work faster? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Dec 2010, at 17:40, Kenneth Marshall wrote: You need to use a DB backend that supports fulltext indexing for content searchs to be fast. The actual query that you stated runs quickly, is only for the first 50 tickets. I do agree that running the same count() query twice for the same search is sub-optimal. I do not see how you could avoid the count query totally if you are paginating the results. Cheers, Ken On Tue, Dec 07, 2010 at 05:31:17PM +, Justin Hayes wrote: Guys, Searching for ticket content takes forever. I've done a bit of digging and for a single search in one of my queues over the last year, RT spawned 3 separate queries. 2 are counts (which appear to be identical), and 1 gets the actual content. Is there anyway round this? Losing loads of time just to get counts seems rather counter-productive? The final select was actually pretty quick. I've added the queries below. Many thanks, Justin # Time: 101207 17:24:09 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 57.722237 Lock_time: 0.000183 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742649; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:38 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 28.780620 Lock_time: 0.000510 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742678; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:42 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 4.492875 Lock_time: 0.000175 Rows_sent: 50 Rows_examined: 100799 SET timestamp=1291742682; SELECT DISTINCT main.* FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id) ORDER BY main.id ASC LIMIT 50
Re: [rt-users] Content searching takes a long time and runs multiple queries
I was about to say that should have been 3.9 :) Been a while since I checked out bestpractical.com. Just saw the post about the dev release, and looks like things are going well which is great news. The only downside is the time needed to migrate to postgres, and then the time it's going to take to port all my custom code into the new version. But that's my problem, and one I have every time I upgrade, so can't complain! Looking forward to a production-ready version Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 8 Dec 2010, at 16:33, Jesse Vincent wrote: On Wed 8.Dec'10 at 16:31:11 +, Justin Hayes wrote: I hadn't actually realised it was possible to get hold of 4.9, though if that's beta I guess I'd want to wait. 3.9. And we're aiming for a release candidate by 12/25. You can download test builds from http://download.bestpractical.com/pub/rt/devel/ Best, Jesse
[rt-users] Content searching takes a long time and runs multiple queries
Guys, Searching for ticket content takes forever. I've done a bit of digging and for a single search in one of my queues over the last year, RT spawned 3 separate queries. 2 are counts (which appear to be identical), and 1 gets the actual content. Is there anyway round this? Losing loads of time just to get counts seems rather counter-productive? The final select was actually pretty quick. I've added the queries below. Many thanks, Justin # Time: 101207 17:24:09 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 57.722237 Lock_time: 0.000183 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742649; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:38 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 28.780620 Lock_time: 0.000510 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742678; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:42 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 4.492875 Lock_time: 0.000175 Rows_sent: 50 Rows_examined: 100799 SET timestamp=1291742682; SELECT DISTINCT main.* FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id) ORDER BY main.id ASC LIMIT 50; - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com
Re: [rt-users] Content searching takes a long time and runs multiple queries
Hi Ken, I was just thinking the same about the counts - it has to do that for pagination. Though I guess it could have been written to run 1 query for all the data, and just display the first 50 etc. Which DB backend would work faster? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Dec 2010, at 17:40, Kenneth Marshall wrote: You need to use a DB backend that supports fulltext indexing for content searchs to be fast. The actual query that you stated runs quickly, is only for the first 50 tickets. I do agree that running the same count() query twice for the same search is sub-optimal. I do not see how you could avoid the count query totally if you are paginating the results. Cheers, Ken On Tue, Dec 07, 2010 at 05:31:17PM +, Justin Hayes wrote: Guys, Searching for ticket content takes forever. I've done a bit of digging and for a single search in one of my queues over the last year, RT spawned 3 separate queries. 2 are counts (which appear to be identical), and 1 gets the actual content. Is there anyway round this? Losing loads of time just to get counts seems rather counter-productive? The final select was actually pretty quick. I've added the queries below. Many thanks, Justin # Time: 101207 17:24:09 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 57.722237 Lock_time: 0.000183 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742649; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:38 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 28.780620 Lock_time: 0.000510 Rows_sent: 1 Rows_examined: 122794 SET timestamp=1291742678; SELECT COUNT(DISTINCT main.id) FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id); # Time: 101207 17:24:42 # u...@host: rt_support[rt_support] @ localhost [] # Query_time: 4.492875 Lock_time: 0.000175 Rows_sent: 50 Rows_examined: 100799 SET timestamp=1291742682; SELECT DISTINCT main.* FROM Tickets main JOIN Transactions Transactions_1 ON ( Transactions_1.ObjectId = main.id ) JOIN Attachments Attachments_2 ON ( Attachments_2.TransactionId = Transactions_1.id ) WHERE (Transactions_1.ObjectType = 'RT::Ticket') AND (main.Status != 'deleted') AND (main.Created '2010-01-01 00:00:00' AND main.Queue = '4' AND Attachments_2.Content LIKE '%testing%') AND (main.Type = 'ticket') AND (main.EffectiveId = main.id) ORDER BY main.id ASC LIMIT 50; - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com
[rt-users] Multiple instances of RT web - 1 DB
Is it ok to have multiple webservers running the front end for an RT instance all pointing at the same RT DB? I assume it should be but just want confirmation first. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History
I'll have a look at that Jesse, thanks. I think part of the problem is not really knowing how fast it should actually be. Perhaps my install is working as per normal and I just think it slow. Certainly after adding that patch of Tim's and running faster hardware it's a lot perkier. Ruslan also suggested that the ticket history has had a rewrite in 3.10. Any idea how far off that is? Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 10 Sep 2010, at 03:52, Jesse Vincent wrote: On Thu, Sep 09, 2010 at 08:15:37AM +0100, Justin Hayes wrote: We gave those patches a try. Shaved ~4s from our 20s, which isn't bad so we might look at rolling them into Live. Justin, Have you tried running ticket loads under Devel::NYTProf with the standalone server to try to identify what's up here for you? RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
We've tried all of those. We've also tried the RT inbuilt webserver and results were the same in all cases. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:36, Torsten Brumm wrote: Are you running mod_perl2, FastCGI or FCGID? 2010/9/7 Justin Hayes justin.ha...@openbet.com Thanks Torsten. Will have a look at some of those suggestions. We'd ruled the DB out as we couldn't find an slow ones in the logs, and also because we tried a totally fresh DB with only 1 ticket in it, and it was still slow. But it might just be lots of tiny queries so we'll have another look at that. We've also tried running the web services on a separate server to the DB. This also made no difference. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Torsten. We'll have a look at those settings as well. Appreciate the help! Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 12:45, Torsten Brumm wrote: Hi Justin, just created inside a RT Test VM (slow one with 500mb ram) a single ticket with around 60 replies and some comments. Tested the speed with different users 1. root user to open this ticket: around 26 sec - 870 single sql queries in around 4 sec! (Queries: http://pastebin.com/7Yekfx2Y) 2. user with full access (take, own, modify etc): around same time and queries like root (Queries: http://pastebin.com/U0HnPcJL) 3. user with less rights (no take, no own, only showticket, seequeue): time around 15 sec and 600 sql queries in around 2 sec! (Queries: http://pastebin.com/fXDHu6im) After this the apache starts to render the page from the results and push them to the browser. The page is for my few comments/replies already 206KB without any apache optimizations After adding: SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary ExpiresActive On ExpiresByType text/css A604800 ExpiresByType image/x-icon A31536000 ExpiresByType image/gif A604800 ExpiresByType image/jpg A604800 ExpiresByType image/jpeg A604800 ExpiresByType image/png A604800 ExpiresByType application/x-javascript A3600 Header set Cache-Control must-revalidate to the rt vhost, the page load time goes down from 26 sec to 8 sec and from 206 kb to 10kb you should try. Torsten 2010/9/7 Justin Hayes justin.ha...@openbet.com Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote
Re: [rt-users] Slow Ticket History 3.8.8
Aren't those options just compressing the page to send out to the browser and caching the output? We're on an internal gigabit network so seems unlikely that would help. All our time goes on the server actually building the page to send out I think. Can try it though :) Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 12:45, Torsten Brumm wrote: Hi Justin, just created inside a RT Test VM (slow one with 500mb ram) a single ticket with around 60 replies and some comments. Tested the speed with different users 1. root user to open this ticket: around 26 sec - 870 single sql queries in around 4 sec! (Queries: http://pastebin.com/7Yekfx2Y) 2. user with full access (take, own, modify etc): around same time and queries like root (Queries: http://pastebin.com/U0HnPcJL) 3. user with less rights (no take, no own, only showticket, seequeue): time around 15 sec and 600 sql queries in around 2 sec! (Queries: http://pastebin.com/fXDHu6im) After this the apache starts to render the page from the results and push them to the browser. The page is for my few comments/replies already 206KB without any apache optimizations After adding: SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary ExpiresActive On ExpiresByType text/css A604800 ExpiresByType image/x-icon A31536000 ExpiresByType image/gif A604800 ExpiresByType image/jpg A604800 ExpiresByType image/jpeg A604800 ExpiresByType image/png A604800 ExpiresByType application/x-javascript A3600 Header set Cache-Control must-revalidate to the rt vhost, the page load time goes down from 26 sec to 8 sec and from 206 kb to 10kb you should try. Torsten 2010/9/7 Justin Hayes justin.ha...@openbet.com Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect
Re: [rt-users] Slow Ticket History
We gave those patches a try. Shaved ~4s from our 20s, which isn't bad so we might look at rolling them into Live. Obviously if you're getting 20min loads then something must be much more fundamentally wrong with your setup I'd say. We've never had anything even remotely that long. Thanks for the patch! Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 12:24, Tim Cutts wrote: On 6 Sep 2010, at 5:54 pm, rt-users-requ...@lists.bestpractical.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; From what I can tell, the real problem here is repeated scanning of both the @attachments and @attachment_content arrays, which makes the execution speed of the ShowHistory element O(N^2) with respect to the number of transactions; painful, to say the least. 1) The %$trans_content hash can be made up front in a single pass, for all the attachments in the ticket, turning this into an O(N) operation, rather than O(N^2) 2) The $trans_content variable is only used in one place; it's passed to ShowTransaction, where it is then passed on to ShowTransactionAttachments. 3) In there, there are some errors which cause some autovivification of hash members which needn't happen. 4) You can do much the same up-front calculation with $trans_attachments as well, so you don't have to keep grepping through it I've now made those changes, and on a reasonably large ticket (216 transactions) it reduced the ticket rendering time on my system from 2.5 minutes to 34 seconds, which is a pretty good improvement, I think. On a more extreme ticket, with 417 transactions, the 3.8.8 release code takes over 20 minutes to render the ticket (I gave up waiting), so in fact it's considerably worse order execution time than I thought. My patched code takes 2.2 minutes - still not brilliant but hey, this ticket is now renderable, which it was not before. Just for some background, the hardware I'm running on: RT database: 2 CPU virtual machine, 8GB RAM, MySQL, running on vSphere 4.1 on a 2.0 GHz E5504 Nehalem system RT web server: 2 CPU virtual machine, 2GB RAM, on the same type of physical hardware as the database server As far as I can tell, I have not semantically altered the code, but others may want to test more thoroughly. I have not yet put this on my production web server - I cloned my web server VM and made my changes to the clone (God, I love VMs for this sort of thing!) Regards, Tim Here are my patches to Ticket/Elements/ShowHistory: --- /opt/rt3/share/html/Ticket/Elements/ShowHistory 2010-05-14 13:58:15.0 +0100 +++ /opt/rt3/local/html/Ticket/Elements/ShowHistory 2010-09-07 11:59:30.0 +0100 @@ -84,6 +84,10 @@ %perl my @attachments = @{$Attachments-ItemsArrayRef()}; my @attachment_content = @{$AttachmentContent-ItemsArrayRef()}; +my $trans_content = {}; +map { $trans_content-{$_-TransactionId}-{$_-Id} = $_ } @attachment_content; +my $trans_attachments = {}; +map { push (@{$trans_attachments-{$_-TransactionId}}, $_) } @attachments; while ( my $Transaction = $Transactions-Next ) { my $skip = 0; @@ -97,12 +101,6 @@ $i++; -my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; - -my $trans_content = {}; -grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; - - my $IsLastTransaction = 0; if ( $OldestFirst ) { $IsLastTransaction = $Transactions-IsLast; @@ -118,7 +116,7 @@ Transaction = $Transaction, ShowHeaders = $ShowHeaders, RowNum = $i, - Attachments = \...@trans_attachments, + Attachments = $trans_attachments-{$Transaction-id}, AttachmentContent= $trans_content, LastTransaction = $IsLastTransaction ); and here's the patch to ShowTransactionAttachments (both files need to be patched): --- /opt/rt3/share/html/Ticket/Elements/ShowTransactionAttachments
Re: [rt-users] Slow Ticket History 3.8.8
We've just tried a brand new webserver. Intel(R) Xeon(R) CPU E5640 @ 2.67GHz Quad (with hyperthreading) 12GB RAM Load time has dropped from 20-30s down to 11s. So throwing some newer hardware at it definitely improves things. Hopefully this will tide us over until RT3.10 which has the history rewrite in it. Going to try the patches from that other slow ticket history thread as well to see if that brings it down further. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 9 Sep 2010, at 08:13, Justin Hayes wrote: Aren't those options just compressing the page to send out to the browser and caching the output? We're on an internal gigabit network so seems unlikely that would help. All our time goes on the server actually building the page to send out I think. Can try it though :) Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 12:45, Torsten Brumm wrote: Hi Justin, just created inside a RT Test VM (slow one with 500mb ram) a single ticket with around 60 replies and some comments. Tested the speed with different users 1. root user to open this ticket: around 26 sec - 870 single sql queries in around 4 sec! (Queries: http://pastebin.com/7Yekfx2Y) 2. user with full access (take, own, modify etc): around same time and queries like root (Queries: http://pastebin.com/U0HnPcJL) 3. user with less rights (no take, no own, only showticket, seequeue): time around 15 sec and 600 sql queries in around 2 sec! (Queries: http://pastebin.com/fXDHu6im) After this the apache starts to render the page from the results and push them to the browser. The page is for my few comments/replies already 206KB without any apache optimizations After adding: SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary ExpiresActive On ExpiresByType text/css A604800 ExpiresByType image/x-icon A31536000 ExpiresByType image/gif A604800 ExpiresByType image/jpg A604800 ExpiresByType image/jpeg A604800 ExpiresByType image/png A604800 ExpiresByType application/x-javascript A3600 Header set Cache-Control must-revalidate to the rt vhost, the page load time goes down from 26 sec to 8 sec and from 206 kb to 10kb you should try. Torsten 2010/9/7 Justin Hayes justin.ha...@openbet.com Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who
Re: [rt-users] Slow Ticket History 3.8.8
I actually liked the incremental page load, as I could read the start of the ticket while the rest was loading, thus saving me a bit of time ;) However this seems to have stopped since installing 3.8.8. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 9 Sep 2010, at 13:50, Kenneth Marshall wrote: One big win with enabling compression was that pages loaded in bigger pieces and you have less problems with users trying to type in an page that is unfinished with unpredictable results. With the DEFLATE on, the page all decompresses on the fast client instead of dribbling out from the server. Cheers, Ken On Thu, Sep 09, 2010 at 08:13:28AM +0100, Justin Hayes wrote: Aren't those options just compressing the page to send out to the browser and caching the output? We're on an internal gigabit network so seems unlikely that would help. All our time goes on the server actually building the page to send out I think. Can try it though :) Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 12:45, Torsten Brumm wrote: Hi Justin, just created inside a RT Test VM (slow one with 500mb ram) a single ticket with around 60 replies and some comments. Tested the speed with different users 1. root user to open this ticket: around 26 sec - 870 single sql queries in around 4 sec! (Queries: http://pastebin.com/7Yekfx2Y) 2. user with full access (take, own, modify etc): around same time and queries like root (Queries: http://pastebin.com/U0HnPcJL) 3. user with less rights (no take, no own, only showticket, seequeue): time around 15 sec and 600 sql queries in around 2 sec! (Queries: http://pastebin.com/fXDHu6im) After this the apache starts to render the page from the results and push them to the browser. The page is for my few comments/replies already 206KB without any apache optimizations After adding: SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary ExpiresActive On ExpiresByType text/css A604800 ExpiresByType image/x-icon A31536000 ExpiresByType image/gif A604800 ExpiresByType image/jpg A604800 ExpiresByType image/jpeg A604800 ExpiresByType image/png A604800 ExpiresByType application/x-javascript A3600 Header set Cache-Control must-revalidate to the rt vhost, the page load time goes down from 26 sec to 8 sec and from 206 kb to 10kb you should try. Torsten 2010/9/7 Justin Hayes justin.ha...@openbet.com Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,?debug?); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Jeffrey. We're already using UseSQLForACLChecks and don't use web external auth. Tried the combination you've given though and unfortunately no difference :( - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 00:44, Jeffrey Fearn wrote: Hi Justin, I've recently been using siege to bash on RT, and have been testing the following two settings in our RT_SiteConfig.pm Set($UseSQLForACLChecks, 1); Set($WebExternalAuthContinuous, 0); The combined effect has been a serious reduction in rendering speed in general, and particularly so for long tickets. Cheers, Jeff. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; Both are greps. Does this imply that perl itself is just slow? IF so why would our perl be slow compared to other people's? We've tried compiling it from source and that made no difference. ATM we're at a bit of a loss Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 01 July 2010 11:39 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine
Re: [rt-users] Slow Ticket History 3.8.8
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; Both are greps. Does this imply that perl itself is just slow? IF so why would our perl be slow compared to other people's? We've tried compiling it from source and that made no difference. ATM we're at a bit of a loss Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 01 July 2010 11:39 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Torsten. Will have a look at some of those suggestions. We'd ruled the DB out as we couldn't find an slow ones in the logs, and also because we tried a totally fresh DB with only 1 ticket in it, and it was still slow. But it might just be lots of tiny queries so we'll have another look at that. We've also tried running the web services on a separate server to the DB. This also made no difference. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; Both are greps. Does this imply that perl itself is just slow? IF so why would our perl be slow compared to other people's? We've tried compiling it from source and that made no difference. ATM we're at a bit of a loss Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11
Re: [rt-users] Slow Ticket History 3.8.8
Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; Both are greps. Does this imply that perl itself is just slow? IF so why would our perl be slow compared to other people's? We've tried compiling it from source and that made no difference. ATM we're at a bit of a loss Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields
Re: [rt-users] Slow Ticket History 3.8.8
Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com wrote: So far we've tried installing RT on different hardware, both 32 and 64bit versions of linux. RT is still very slow for long tickets. All the time is taken up by the perl/apache process maxing out a core of CPU. We've even gone as far as trying to profile the code. We came up with this graph of where the time was going: TIMING.png We then tried to go further into those functions but can't find a single smoking gun call that is taking all the time. For example in a ticket that takes 22s to render approx 5 secs goes on these 2 lines: File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8 my @trans_attachments = grep { $_-TransactionId == $Transaction-Id } @attachments; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; Both are greps. Does this imply that perl itself is just slow? IF so why would our perl be slow compared to other people's? We've tried compiling it from source and that made no difference. ATM we're at a bit of a loss Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 01 July 2010 11:39 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total
[rt-users] Incremental page load stopped with 3.8.8
When we were running 3.8.4 my pages used to render content incrementally, which was great as I could read initial content, metadata etc while the page was loading the rest. Since 3.8.8 that's stopped, and I have to wait for the whole page to load before my browser renders. Is that a deliberate change? Anyone know if there's a way to set it back to incremental? Thanks! Justin Justin Hayes Support Manager T: +44 208 742 1600 +44 208 987 0715 M: +44 779 337 2866 E: justin.ha...@openbet.com W: www.openbet.com OpenBet Chiswick Park Building 9 566 Chiswick High Rd London W4 5XT This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmas...@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Technologies Limited. Registered Office: One London Road, Staines, Middlesex, TW18 4EX, United Kingdom. A company registered in England and Wales. Registered no. 6712030. VAT no. GB927523623 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
[rt-users] Add a user to a group via RT CLI?
Is it possible to add a user to a group via the CLI? I can find the command to create a new user, and I can create a new group. But how can I put one in the other? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] 3.8.4 - Customers able to see tickets for queues they don't have rights on
Thanks Jesse. I know it's beta, but does fix some other issues so having it on was better than not, if you see what I mean :) I'm upgrading to 3.8.8 soon anyway which will fix it. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 14 Jul 2010, at 14:49, Jesse Vincent wrote: This is on 3.8.4 - we've got 3.8.8 on a test system and it doesn't seem to be showing the same problem on there. Anyone noticed this before?? I use UseSQLForACLChecks = 1. If I turn that off then at least they can't see things they shouldn't, but now the search results are very messed up and you might have to page until you can find a visible ticket. I suspect that this is We've fixed a bug in UseSQLForACLChecks more than anything else. There's a reason (several actually) that we describe UseSQLForACLChecks as beta ;) Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Scrips and Recipients section not picking up TransactionBatch scrips - 3.8.8
I normally do - I think I must have changed this one early on in the process before I got used to altering RT_SiteConfig.pm. So the problem still stands - I'm surprised no-one else has seen this before, unless not many people use Batch? Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 13 Jul 2010, at 19:36, Kenneth Crocker wrote: Justin, Yep. But I'd make the change in RT_SiteConfig.pm. I like to leave original files/info alone in case I ever need to reference an original setting, etc. RT will automatically override the setting from the RT_SiteConfig.pm file. Kenn LBNL On Tue, Jul 13, 2010 at 8:59 AM, Justin Hayes justin.ha...@openbet.com wrote: Is that this one Kenn? RT_Config.pm:Set($UseTransactionBatch, 1); Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 13 Jul 2010, at 16:52, Kenneth Crocker wrote: Justin, I hate to ask, but did you make sure your configuration setting is for Batch? Kenn LBNL On Tue, Jul 13, 2010 at 8:03 AM, Justin Hayes justin.ha...@openbet.com wrote: I've noticed that the 'Scrips and Recipients' section of the Ticket update page doesn't pick up scrips that are TransactionBatch. Only TransactionCreate ones are shown. I use TransactionBatch a lot of the time (I'm seem to remember their being good reasons for this) and so I can't actually see who's going to be sent mails. Does anyone know if this is a known bug/issue or deliberate? I've not found anything on the wiki Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Scrips and Recipients section not picking up TransactionBatch scrips - 3.8.8
That makes quite a bit of sense Roy. I'm only really interested in giving people some idea about who's going to be getting standard automated responses, so I should probably look at switching some scrips back to TransactionCreate where possible. As for my performance problems, hopefully my guy's going to be trying some of the settings you sent me, as there are lots of innoDB configs in there that we've not set at all. Hopefully we'll get somewhere. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 15 Jul 2010, at 11:10, Raed El-Hames wrote: Hi Justin; Ø I've noticed that the 'Scrips and Recipients' section of the Ticket update page doesn't pick up scrips that are TransactionBatch. Only TransactionCreate ones are shown. This is just pure guess on my part, but I think you don’t see the list of recipients with TransactionBatch because its not possible to determine the list until you hit submit, because its possible there will be scrips that depends on other scrips or actions, eg scrip 1 on correspondence update custom field 1 if the word ‘hello’ is in the update, scrip 2 on if custom field 1 changed email b...@blah So you see its not possible to determine b...@blah will be emailed until the Transaction is created. As I said, this is my guess, possibly some one from BP can confirm. Roy Ps:how did you get on with the performance issue you had? From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 15 July 2010 07:49 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Scrips and Recipients section not picking up TransactionBatch scrips - 3.8.8 I normally do - I think I must have changed this one early on in the process before I got used to altering RT_SiteConfig.pm. So the problem still stands - I'm surprised no-one else has seen this before, unless not many people use Batch? Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 13 Jul 2010, at 19:36, Kenneth Crocker wrote: Justin, Yep. But I'd make the change in RT_SiteConfig.pm. I like to leave original files/info alone in case I ever need to reference an original setting, etc. RT will automatically override the setting from the RT_SiteConfig.pm file. Kenn LBNL On Tue, Jul 13, 2010 at 8:59 AM, Justin Hayes justin.ha...@openbet.com wrote: Is that this one Kenn? RT_Config.pm:Set($UseTransactionBatch, 1); Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 13 Jul 2010, at 16:52, Kenneth Crocker wrote: Justin, I hate to ask, but did you make sure your configuration setting is for Batch? Kenn LBNL On Tue, Jul 13, 2010 at 8:03 AM, Justin Hayes justin.ha...@openbet.com wrote: I've noticed that the 'Scrips and Recipients' section of the Ticket update page doesn't pick up scrips that are TransactionBatch. Only TransactionCreate ones are shown. I use TransactionBatch a lot of the time (I'm seem to remember their being good reasons for this) and so I can't actually see who's going to be sent mails. Does anyone know if this is a known bug/issue or deliberate? I've not found anything on the wiki Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Scrips and Recipients section not picking up TransactionBatch scrips - 3.8.8
Is that this one Kenn? RT_Config.pm:Set($UseTransactionBatch, 1); Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 13 Jul 2010, at 16:52, Kenneth Crocker wrote: Justin, I hate to ask, but did you make sure your configuration setting is for Batch? Kenn LBNL On Tue, Jul 13, 2010 at 8:03 AM, Justin Hayes justin.ha...@openbet.com wrote: I've noticed that the 'Scrips and Recipients' section of the Ticket update page doesn't pick up scrips that are TransactionBatch. Only TransactionCreate ones are shown. I use TransactionBatch a lot of the time (I'm seem to remember their being good reasons for this) and so I can't actually see who's going to be sent mails. Does anyone know if this is a known bug/issue or deliberate? I've not found anything on the wiki Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] 3.8.4 - Customers able to see tickets for queues they don't have rights on
One of my customers has just alerted me to the fact that by doing a certain search they can list tickets they shouldn't be able to see. For example they build this search Status = 'open' OR Status = 'stalled' in Advanced and they can see rows returned for queues they do not have See Queue and Show Ticket rights for However if you put ()s round the search it works correctly (Status = 'open' OR Status = 'stalled') This is on 3.8.4 - we've got 3.8.8 on a test system and it doesn't seem to be showing the same problem on there. Anyone noticed this before?? I use UseSQLForACLChecks = 1. If I turn that off then at least they can't see things they shouldn't, but now the search results are very messed up and you might have to page until you can find a visible ticket. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps], 222K conn, TX: 637B, RX: 39B) [--] Reads / Writes: 98% / 2% [--] Total buffers: 602.0M global + 134.8M per thread (150 max threads) [!!] Maximum possible memory usage: 20.3G (262% of installed RAM) [OK] Slow queries: 0% (229K/110M) [!!] Highest connection usage: 100% (151/150) [OK] Key buffer size / total MyISAM indexes: 512.0M/6.7M [OK] Key buffer hit rate: 100.0% (84M cached / 7K reads) [OK] Query cache efficiency: 71.4% (76M cached / 107M selects) [!!] Query cache prunes per day: 661360 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2M sorts) [!!] Joins performed without indexes: 112714 [!!] Temporary tables created on disk: 33% (968K on disk / 2M total) [OK] Thread cache hit rate: 99% (1K created / 222K connections) [OK] Table cache hit rate: 36% (318 open / 880 opened) [OK] Open file limit used: 14% (166/1K) [OK] Table locks acquired immediately: 99% (39M immediate / 39M locks) [!!] InnoDB data size / buffer pool: 10.1G/8.0M Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Reduce your overall MySQL memory footprint for system stability Reduce or eliminate persistent connections to reduce connection usage Adjust your join queries to always utilize indexes When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries without LIMIT clauses Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** max_connections ( 150) wait_timeout ( 28800) interactive_timeout ( 28800) query_cache_size ( 16M) join_buffer_size ( 2.0M, or always use indexes with joins) tmp_table_size ( 128M) max_heap_table_size ( 64M) innodb_buffer_pool_size (= 10G) - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:22, Jason Doran wrote: Hi, If you are using mysqld have a look at mysqltuner.pl perl script (google) This has fixed quickly many performance issues on both RT and other web-based software we use. I run this every few weeks and apply suggested changes and then simply restart mysqld when things are quite. Regards, Jason Doran Computer Centre NUI, Maynooth On 29 Jun 2010, at 14:09, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just
Re: [rt-users] Slow Ticket History 3.8.8
Thanks a lot for this response Kim. r.e. the joins without indexes. How can we work out which indexes we need? I'd have thought this would cause slow queries but we're not seeing any... We're trying to tune on our test server before making any changes to live, however the hardware etc is different so not sure it it's going to be that useful an excercise. This is what we're not getting on test - we still get long ticket load times and we've done a lot of the recommended tuning: [OK] Logged in using credentials passed on the command line General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.0.75-0ubuntu10.2 [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 131M (Tables: 15) [--] Data in InnoDB tables: 10G (Tables: 81) [!!] Total fragmented tables: 2 Performance Metrics - [--] Up for: 3m 44s (9K q [41.304 qps], 73 conn, TX: 17M, RX: 2M) [--] Reads / Writes: 99% / 1% [--] Total buffers: 11.0G global + 3.5M per thread (150 max threads) [!!] Maximum possible memory usage: 11.5G (149% of installed RAM) [OK] Slow queries: 0% (20/9K) [OK] Highest usage of available connections: 9% (14/150) [OK] Key buffer size / total MyISAM indexes: 3.0G/4.6M [OK] Key buffer hit rate: 97.8% (453 cached / 10 reads) [OK] Query cache efficiency: 89.0% (8K cached / 9K selects) [OK] Query cache prunes per day: 0 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 22 sorts) [OK] Temporary tables created on disk: 14% (28 on disk / 195 total) [OK] Thread cache hit rate: 80% (14 created / 73 connections) [OK] Table cache hit rate: 95% (121 open / 127 opened) [OK] Open file limit used: 7% (80/1K) [OK] Table locks acquired immediately: 100% (773 immediate / 773 locks) [!!] InnoDB data size / buffer pool: 10.7G/6.0G Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance MySQL started within last 24 hours - recommendations may be inaccurate Reduce your overall MySQL memory footprint for system stability Enable the slow query log to troubleshoot bad queries Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** innodb_buffer_pool_size (= 10G) - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 30 Jun 2010, at 08:55, Kim Pedersen wrote: Hi Justin, Our RT instance is much smaller than yours, but we had very similar problems. Optimizing MySQL using the output from the tuner script made nearly all the speed issues go away. The one remaining issue we had with slow queries was resolved after the upgrade to 3.8.8 Your key buffer size looks like overkill - probably want to reduce that, and you have many joins done without indexes - that's a performance killer. Your InnoDB memory allocation looks like it needs to be increasing. Overall I'd say your database performance is skewed towards ISAM performance and not InnoDB (Which is what RT3 uses by default) Kim On 2010-06-29 10:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps], 222K conn, TX: 637B, RX: 39B) [--] Reads / Writes: 98% / 2% [--] Total buffers: 602.0M global + 134.8M per thread (150 max threads) [!!] Maximum possible memory usage: 20.3G (262% of installed RAM) [OK] Slow queries: 0% (229K/110M) [!!] Highest connection usage: 100% (151/150) [OK] Key buffer size / total MyISAM indexes: 512.0M/6.7M [OK] Key buffer hit rate: 100.0% (84M cached / 7K reads) [OK] Query cache efficiency: 71.4% (76M cached / 107M selects) [!!] Query cache prunes per day: 661360 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2M sorts) [!!] Joins performed without indexes: 112714 [!!] Temporary tables created on disk: 33% (968K on disk / 2M total) [OK] Thread cache hit rate: 99% (1K created / 222K connections
Re: [rt-users] Slow Ticket History 3.8.8
I'd heard of this before and thought I'd removed them. Just checked on live and they're still there. However removing them seems to have made no difference. Still taking 20+seconds for a ticket with 130 comments/replies (+ other status changes etc). Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 01 July 2010 11:39 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps], 222K conn, TX: 637B, RX: 39B) [--] Reads / Writes: 98% / 2% [--] Total buffers: 602.0M global + 134.8M per thread (150 max threads) [!!] Maximum possible memory usage: 20.3G (262% of installed RAM) [OK] Slow queries: 0% (229K/110M) [!!] Highest connection usage: 100% (151/150) [OK] Key buffer size / total MyISAM indexes: 512.0M/6.7M [OK] Key buffer hit rate: 100.0% (84M cached / 7K reads) [OK] Query cache efficiency: 71.4% (76M cached / 107M selects) [!!] Query cache prunes per day: 661360 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2M sorts) [!!] Joins performed without indexes: 112714 [!!] Temporary tables created on disk: 33% (968K on disk / 2M total) [OK] Thread cache hit rate: 99% (1K created / 222K connections) [OK] Table cache hit rate: 36% (318 open / 880 opened) [OK] Open file limit used: 14% (166/1K) [OK] Table locks acquired immediately: 99% (39M immediate / 39M locks) [!!] InnoDB data size / buffer pool: 10.1G/8.0M Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Reduce your overall MySQL memory footprint for system stability Reduce or eliminate persistent connections to reduce connection usage Adjust your join queries to always utilize indexes When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries without LIMIT clauses Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** max_connections ( 150) wait_timeout ( 28800) interactive_timeout ( 28800) query_cache_size ( 16M) join_buffer_size ( 2.0M, or always use indexes with joins) tmp_table_size ( 128M) max_heap_table_size ( 64M) innodb_buffer_pool_size (= 10G
Re: [rt-users] Slow Ticket History 3.8.8
Hi Roy, We've logged the SQL statements on the test server - virtually every query is .000s of a second. Yet the ticket we're using on there still takes 12s to render. So the queries seem ok. However I know that having a huge ticket in a small DB is much faster (say 2s). So it *seems* to be DB or server related, I just don't know how... Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 12:28, Raed El-Hames wrote: Hi Justin; Our database is ~ 59G, the attachments table is ~ 40G. In our setup , the web server is different hardware to the db servers , and the db servers are 2 * dual core 2.6 AMD processors with 16G Ram, A stupid question, but did you restart apache after you commented out the ShowTransaction lines ? What version of RT by the way? Do you use UseSQLForACLChecks?? I found it slows things a bit as it tries to build long joins with cached group members and groups etc. In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. Then try to load the ticket, then from your rt.log grab some of the sql statements generated and try executing them on the sql server via sql console, and see if any are slow , for any slow queries try an explain and see what indexes are used. Something else I also try is to put debug lines with timestamps within the html page itself and see which component taking the longest time, then do the same within the slowest component breaking it into various parts to determine which bit of code is taking the longest time etc etc. Feel free to mail me again if you need further help. Regards; Roy From: Justin Hayes [mailto:justin.ha...@openbet.com] Sent: 01 July 2010 12:08 To: Raed El-Hames Subject: Re: [rt-users] Slow Ticket History 3.8.8 Hi Raed, How big is your DB? We've got about 10gb in Attachments. If we put a really long ticket in an empty DB performance is excellent. So something is wrong with the server/DB, we just don't know what. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes Sent: 01 July 2010 11:39 To: Kenneth Crocker Cc: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Alberto. Will look at those as well. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 1 Jul 2010, at 14:42, Alberto Villanueva wrote: Have you read next links? Maybe you could be good for your server. http://wiki.bestpractical.com/view/PerformanceTuning http://wiki.bestpractical.com/view/CleanupSessions Good luck! Hi Roy, We've logged the SQL statements on the test server - virtually every query is .000s of a second. Yet the ticket we're using on there still takes 12s to render. So the queries seem ok. However I know that having a huge ticket in a small DB is much faster (say 2s). So it *seems* to be DB or server related, I just don't know how... Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com mailto:justin.ha...@openbet.com On 1 Jul 2010, at 12:28, Raed El-Hames wrote: Hi Justin; Our database is ~ 59G, the attachments table is ~ 40G. In our setup , the web server is different hardware to the db servers , and the db servers are 2 * dual core 2.6 AMD processors with 16G Ram, A stupid question, but did you restart apache after you commented out the ShowTransaction lines ? What version of RT by the way? Do you use UseSQLForACLChecks?? I found it slows things a bit as it tries to build long joins with cached group members and groups etc. In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. Then try to load the ticket, then from your rt.log grab some of the sql statements generated and try executing them on the sql server via sql console, and see if any are slow , for any slow queries try an explain and see what indexes are used. Something else I also try is to put debug lines with timestamps within the html page itself and see which component taking the longest time, then do the same within the slowest component breaking it into various parts to determine which bit of code is taking the longest time etc etc. Feel free to mail me again if you need further help. Regards; Roy *From:* Justin Hayes [mailto:justin.ha...@openbet.com] *Sent:* 01 July 2010 12:08 *To:* Raed El-Hames *Subject:* Re: [rt-users] Slow Ticket History 3.8.8 Hi Raed, How big is your DB? We've got about 10gb in Attachments. If we put a really long ticket in an empty DB performance is excellent. So something is wrong with the server/DB, we just don't know what. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com mailto:justin.ha...@openbet.com On 1 Jul 2010, at 11:51, Raed El-Hames wrote: Justin, Do you use Transaction custom fields, if you do n’t ; try and comment out lines 70,71,72 from html/Ticket/Elements/ShowTransaction % if ( $Transaction-CustomFieldValues-Count ) { /Elements/ShowCustomFields, Object = $Transaction % } See if that improves things for you. Some of our monitoring tickets can have up to 500 updates, such tickets use to take up to 20s to load, once I commented out the above lines, load time is now down to less than 5 seconds. Regards; Roy *From:* rt-users-boun...@lists.bestpractical.com mailto:rt-users-boun...@lists.bestpractical.com [mailto:rt-users-boun...@lists.bestpractical.com] *On Behalf Of *Justin Hayes *Sent:* 01 July 2010 11:39 *To:* Kenneth Crocker *Cc:* rt-users@lists.bestpractical.com mailto:rt-users@lists.bestpractical.com *Subject:* Re: [rt-users] Slow Ticket History 3.8.8 We do Kenneth, but most tickets don't have many file attachments, so I assume that's not an issue? Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com mailto:justin.ha...@openbet.com On 29 Jun 2010, at 17:54, Kenneth Crocker wrote: Justin, I didn't see this mentioned and may have missed it, but are you displaying attachements inline? That might cut back on the I/O for History. Just a thought. Kenn LBNL On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com mailto:justin.ha...@openbet.com wrote: As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com mailto:justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics
[rt-users] Slow Ticket History 3.8.8
Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
Hi Charles, But as far as we can tell from our profiling the time is spent in the perl rendering the ticket, not in the query used to retrieve the ticket and attachments in the first place (which happens once at the start I believe). The loop that renders the ticket history and each attachment takes anything up to 20+ seconds. So I'm not sure how the DB can be an issue here? Unless I'm wrong about it doing 1 big query at the start and there are loads of little queries inside the loop. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:14, Charles Johnson wrote: Quite outside of the speed of RT, what database are you using as a back-end? How long has it been since you did some care and feeding of your database? How big are your tables? If you are using mysql or postgres have you used their tools to examine your database, especially the indexes and statistics? This may be an issue with indexes needing to be rebuilt, or asking the database to gather new statistics for the queries your are running. Cheers-- Charles On Jun 29, 2010, at 8:09 AM, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Charles Johnson, Vanderbilt University Advanced Computing Center for Research Education Mailing Address: Peabody #34, 230 Appleton Place, Nashville, TN 37203 Shipping Address: 1231 18th Avenue South, Hill Center, Suite 143, Nashville, TN 37212 Office: 615-343-4134 Cell: 615-478-5743 Fax: 615-343-7216 charles.john...@accre.vanderbilt.edu Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Jason - I'll give that a go as well. We have got a large DB (~10gb) so keeping it tuned is definitely important. However as stated the time seems to be lost in code, after I'd have thought it had run the query for the ticket (though I may be wrong in that assumption). Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:22, Jason Doran wrote: Hi, If you are using mysqld have a look at mysqltuner.pl perl script (google) This has fixed quickly many performance issues on both RT and other web-based software we use. I run this every few weeks and apply suggested changes and then simply restart mysqld when things are quite. Regards, Jason Doran Computer Centre NUI, Maynooth On 29 Jun 2010, at 14:09, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
Hehe thanks Charles. I think you might have a point anyway. Running that script recommended by Jason has thrown up a few things that need tweaking with mysql. Not sure what some of them mean yet, and not sure if it'll help, but we'll see. I appreciate you helping believe me! Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:28, Charles Johnson wrote: Justin, thank you for the kindness of your reply. I didn't intend to send you chasing rabbits, so to speak. Just offering something to add to the checklist of items to be crossed off before digging too deeply into the perl code. If you are convinced that mysql is performing at its best then please ignore my noise! :) Cheers-- Charles On Jun 29, 2010, at 9:18 AM, Justin Hayes wrote: Hi Charles, But as far as we can tell from our profiling the time is spent in the perl rendering the ticket, not in the query used to retrieve the ticket and attachments in the first place (which happens once at the start I believe). The loop that renders the ticket history and each attachment takes anything up to 20+ seconds. So I'm not sure how the DB can be an issue here? Unless I'm wrong about it doing 1 big query at the start and there are loads of little queries inside the loop. Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:14, Charles Johnson wrote: Quite outside of the speed of RT, what database are you using as a back-end? How long has it been since you did some care and feeding of your database? How big are your tables? If you are using mysql or postgres have you used their tools to examine your database, especially the indexes and statistics? This may be an issue with indexes needing to be rebuilt, or asking the database to gather new statistics for the queries your are running. Cheers-- Charles On Jun 29, 2010, at 8:09 AM, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Charles Johnson, Vanderbilt University Advanced Computing Center for Research Education Mailing Address: Peabody #34, 230 Appleton Place, Nashville, TN 37203 Shipping Address: 1231 18th Avenue South, Hill Center, Suite 143, Nashville, TN 37212 Office: 615-343-4134 Cell: 615-478-5743 Fax: 615-343-7216 charles.john...@accre.vanderbilt.edu -- Charles Johnson, Vanderbilt University Advanced Computing Center for Research Education Mailing Address: Peabody #34, 230 Appleton Place, Nashville, TN 37203 Shipping Address: 1231 18th Avenue South, Hill Center, Suite 143, Nashville, TN 37212 Office: 615-343-4134 Cell: 615-478-5743 Fax: 615-343-7216 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps], 222K conn, TX: 637B, RX: 39B) [--] Reads / Writes: 98% / 2% [--] Total buffers: 602.0M global + 134.8M per thread (150 max threads) [!!] Maximum possible memory usage: 20.3G (262% of installed RAM) [OK] Slow queries: 0% (229K/110M) [!!] Highest connection usage: 100% (151/150) [OK] Key buffer size / total MyISAM indexes: 512.0M/6.7M [OK] Key buffer hit rate: 100.0% (84M cached / 7K reads) [OK] Query cache efficiency: 71.4% (76M cached / 107M selects) [!!] Query cache prunes per day: 661360 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2M sorts) [!!] Joins performed without indexes: 112714 [!!] Temporary tables created on disk: 33% (968K on disk / 2M total) [OK] Thread cache hit rate: 99% (1K created / 222K connections) [OK] Table cache hit rate: 36% (318 open / 880 opened) [OK] Open file limit used: 14% (166/1K) [OK] Table locks acquired immediately: 99% (39M immediate / 39M locks) [!!] InnoDB data size / buffer pool: 10.1G/8.0M Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Reduce your overall MySQL memory footprint for system stability Reduce or eliminate persistent connections to reduce connection usage Adjust your join queries to always utilize indexes When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries without LIMIT clauses Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** max_connections ( 150) wait_timeout ( 28800) interactive_timeout ( 28800) query_cache_size ( 16M) join_buffer_size ( 2.0M, or always use indexes with joins) tmp_table_size ( 128M) max_heap_table_size ( 64M) innodb_buffer_pool_size (= 10G) - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:22, Jason Doran wrote: Hi, If you are using mysqld have a look at mysqltuner.pl perl script (google) This has fixed quickly many performance issues on both RT and other web-based software we use. I run this every few weeks and apply suggested changes and then simply restart mysqld when things are quite. Regards, Jason Doran Computer Centre NUI, Maynooth On 29 Jun 2010, at 14:09, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
Hi Nicola, I've just tried upgrading from 3.8.4 which I've been on for ages, on the hope that it would be better, but isn't. So I don't think it's something that has either been introduced or fixed by 3.8.8. I'll take a look at that index though. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:49, Foggi, Nicola wrote: I did just have a problem with 3.8.8 and one of the indexes (didn't have the problem on 3.8.6 or below). It turned out to be the index GROUPS2 on the Groups table... dropped the index, and performance was back to normal (pre 3.8.8) Nicola -Original Message- From: rt-users-boun...@lists.bestpractical.com on behalf of Justin Hayes Sent: Tue 6/29/2010 9:28 AM To: Jason Doran Cc: rt Users Subject: Re: [rt-users] Slow Ticket History 3.8.8 Thanks Jason - I'll give that a go as well. We have got a large DB (~10gb) so keeping it tuned is definitely important. However as stated the time seems to be lost in code, after I'd have thought it had run the query for the ticket (though I may be wrong in that assumption). Cheers, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:22, Jason Doran wrote: Hi, If you are using mysqld have a look at mysqltuner.pl perl script (google) This has fixed quickly many performance issues on both RT and other web-based software we use. I run this every few weeks and apply suggested changes and then simply restart mysqld when things are quite. Regards, Jason Doran Computer Centre NUI, Maynooth On 29 Jun 2010, at 14:09, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow Ticket History 3.8.8
As a test we've just created a long ticket in an empty RT DB and it's very fast. So does look to be DB related - contrary to our earlier investigations. I guess it must still access the DB resultset during the ticket rendering (which isn't how we thought it would work). Time to tune the hell out of mysql then... Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:53, Justin Hayes wrote: Seem to be quite a few things to look at Jason. Need to figure out what they all mean first. Justin General Statistics -- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log [OK] Operating on 64-bit architecture Storage Engine Statistics --- [--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 611M (Tables: 8) [--] Data in InnoDB tables: 10G (Tables: 20) [!!] Total fragmented tables: 21 Performance Metrics - [--] Up for: 19d 19h 32m 37s (110M q [64.266 qps], 222K conn, TX: 637B, RX: 39B) [--] Reads / Writes: 98% / 2% [--] Total buffers: 602.0M global + 134.8M per thread (150 max threads) [!!] Maximum possible memory usage: 20.3G (262% of installed RAM) [OK] Slow queries: 0% (229K/110M) [!!] Highest connection usage: 100% (151/150) [OK] Key buffer size / total MyISAM indexes: 512.0M/6.7M [OK] Key buffer hit rate: 100.0% (84M cached / 7K reads) [OK] Query cache efficiency: 71.4% (76M cached / 107M selects) [!!] Query cache prunes per day: 661360 [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2M sorts) [!!] Joins performed without indexes: 112714 [!!] Temporary tables created on disk: 33% (968K on disk / 2M total) [OK] Thread cache hit rate: 99% (1K created / 222K connections) [OK] Table cache hit rate: 36% (318 open / 880 opened) [OK] Open file limit used: 14% (166/1K) [OK] Table locks acquired immediately: 99% (39M immediate / 39M locks) [!!] InnoDB data size / buffer pool: 10.1G/8.0M Recommendations - General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Reduce your overall MySQL memory footprint for system stability Reduce or eliminate persistent connections to reduce connection usage Adjust your join queries to always utilize indexes When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries without LIMIT clauses Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** max_connections ( 150) wait_timeout ( 28800) interactive_timeout ( 28800) query_cache_size ( 16M) join_buffer_size ( 2.0M, or always use indexes with joins) tmp_table_size ( 128M) max_heap_table_size ( 64M) innodb_buffer_pool_size (= 10G) - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 29 Jun 2010, at 15:22, Jason Doran wrote: Hi, If you are using mysqld have a look at mysqltuner.pl perl script (google) This has fixed quickly many performance issues on both RT and other web-based software we use. I run this every few weeks and apply suggested changes and then simply restart mysqld when things are quite. Regards, Jason Doran Computer Centre NUI, Maynooth On 29 Jun 2010, at 14:09, Justin Hayes wrote: Hi everyone, I've raised this before, but we've had another look at it and still can't see how to improve things. We put a lot of comments/replies in our tickets. Often there can be 50-100 entries in a ticket, mostly plain text. Loading such a ticket can take 10-20secs. We don't have any slow queries - all the time seems to be in the code rendering the history of the ticket. We've had a go at stripping functions out of ShowHistory, ShowTransaction and ShowTransactionAttachmments but not had much success. FWIW our RT runs on quad 3ghz Xeons with 8gb of ram. I'd like to try and determine if we're just slow, or if this is just how long RT takes. Maybe perl is just slow. Can anyone shed any light on how long it takes them to render long tickets in their systems? If you look at the page source it gives you a value e.g. spanTime to display: 24.996907/span Can anyone share some numbers from theirs for longer tickets? It would be really appreciated. Thanks, Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden
Re: [rt-users] Time taken to open ticket history
You'd want to prune at the DB query level though I'd have thought? Otherwise it still has to loop over everything to decide whether or not to show things. I'm already doing this (my Display doesnt show quite a lot compared to History) but there's not much speed difference. Perhaps I'm pruning at too high a level in the code? Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 7 Jun 2010, at 21:53, Kenneth Marshall wrote: Hi Justin, The last time I looked at this problem, the slow step is simply preparing all of the transactions for the web. The biggest gains were made by pruning the list of displayed transactions to the bare minimum, by default. Then have a button to allow seeing all of the transactions, if needed. Regards, Ken On Thu, Jun 03, 2010 at 09:27:45AM +0100, Justin Hayes wrote: RT 3.8.4 Does anyone have any suggestions on how to improve the time taken for RT to display ticket history. Depending on the number of entries on the ticket it can take 6-30seconds to render the ticket. I've had a look in the mysql logs for slow queries and can't see any (though it's only logging ones that take longer than 1s so maybe it's doing lots and lots of faster ones that can still be optimised). I've put some debug logging in and all the time is going in ShowHistory, so I guess in the loop that loops over all the various transactions and things to display for that ticket. Any suggestions or tweaks that people have done would be great, before I start investigating what's going on at a lower level. Thanks, Justin Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Time taken to open ticket history
I'm using a SkipTransation callback in ShowHistory that does something like %init my $ttype; $ttype = $Transaction-Type; $$skip = 1 if (($_SkipSystemMessages) ((($Transaction-Creator eq RT-Config-Get('SystemUserID')) ($Transaction-Type ne 'Status') ($Transaction-Type ne 'Comment')) || (($Transaction-Creator eq RT-Config-Get('CronUserID')) (($Transaction-Type ne 'Give') ($Transaction-Type ne 'Correspond'))) || ($Transaction-Type eq 'CustomField') || ($Transaction-Type eq 'Set' $Transaction-Field eq 'TimeWorked')) ); my $type = $Transaction-Type; /%init But I guess it's too late at this point for performance reasons, as we're already processing the entry? Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 7 Jun 2010, at 21:53, Kenneth Marshall wrote: Hi Justin, The last time I looked at this problem, the slow step is simply preparing all of the transactions for the web. The biggest gains were made by pruning the list of displayed transactions to the bare minimum, by default. Then have a button to allow seeing all of the transactions, if needed. Regards, Ken On Thu, Jun 03, 2010 at 09:27:45AM +0100, Justin Hayes wrote: RT 3.8.4 Does anyone have any suggestions on how to improve the time taken for RT to display ticket history. Depending on the number of entries on the ticket it can take 6-30seconds to render the ticket. I've had a look in the mysql logs for slow queries and can't see any (though it's only logging ones that take longer than 1s so maybe it's doing lots and lots of faster ones that can still be optimised). I've put some debug logging in and all the time is going in ShowHistory, so I guess in the loop that loops over all the various transactions and things to display for that ticket. Any suggestions or tweaks that people have done would be great, before I start investigating what's going on at a lower level. Thanks, Justin Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Time taken to open ticket history
RT 3.8.4 Does anyone have any suggestions on how to improve the time taken for RT to display ticket history. Depending on the number of entries on the ticket it can take 6-30seconds to render the ticket. I've had a look in the mysql logs for slow queries and can't see any (though it's only logging ones that take longer than 1s so maybe it's doing lots and lots of faster ones that can still be optimised). I've put some debug logging in and all the time is going in ShowHistory, so I guess in the loop that loops over all the various transactions and things to display for that ticket. Any suggestions or tweaks that people have done would be great, before I start investigating what's going on at a lower level. Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Search for tickets with unresolved dependants
Hi, Is it possible to search for tickets that have some unresolved dependant tickets, or alternatively where all dependants are resolved? I'd like an easy way to see which of my master tickets can be resolved and which can't. Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Requestor email address search hangs apache process
RT3.8.4 If I use an unprivileged user (or a privileged user with no rights to see queues or tickets) to create a very simple search that looks like this: Requestor.EmailAddress Like 'email address' then the apache process handling the request maxes out CPU and hangs forever. If I create the same search with a privileged user that has rights to see queues etc it's fine and returns instantly. This is a big problem as the SelfService interface screens work via requestor email address, so anyone accessing SelfService severely affects performance of my RT. Anyone got any ideas Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Requestor email address search hangs apache process
Hi Kenneth, I'm using MYSQL, How can I tell which query it is? I get no logging and nothing in the MYSQL slow search log, I assume because it never completes? Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 29 Apr 2010, at 13:39, Kenneth Marshall wrote: On Thu, Apr 29, 2010 at 08:24:22AM +0100, Justin Hayes wrote: RT3.8.4 If I use an unprivileged user (or a privileged user with no rights to see queues or tickets) to create a very simple search that looks like this: Requestor.EmailAddress Like 'email address' then the apache process handling the request maxes out CPU and hangs forever. If I create the same search with a privileged user that has rights to see queues etc it's fine and returns instantly. This is a big problem as the SelfService interface screens work via requestor email address, so anyone accessing SelfService severely affects performance of my RT. Anyone got any ideas Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Hi Justin, Perhaps you are missing an index. What query is hitting the database for the slow search? What DB backend are you using? Cheers, Ken Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Requestor email address search hangs apache process
Hi Ruslan, I have both of those indexes already, and in SelfService I have no control over the query being run - it's just trying to show the 'My open tickets' panel. Also there is no problem with SelfService with a privileged account that has rights to see queues/tickets, so it seems to be something else that's the problem. Something rights related? Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 29 Apr 2010, at 17:27, Ruslan Zakirov wrote: Hi Justin, First of all use Requestor.EmailAddress = 'email address', so use = instead of LIKE, in the query builder it is is operator. Second, add index on CachedGroupMembers(MemberId, GroupId, Disabled) and on Users(EmailAddress) if these are not there already. On Thu, Apr 29, 2010 at 11:24 AM, Justin Hayes justin.ha...@orbisuk.com wrote: RT3.8.4 If I use an unprivileged user (or a privileged user with no rights to see queues or tickets) to create a very simple search that looks like this: Requestor.EmailAddress Like 'email address' then the apache process handling the request maxes out CPU and hangs forever. If I create the same search with a privileged user that has rights to see queues etc it's fine and returns instantly. This is a big problem as the SelfService interface screens work via requestor email address, so anyone accessing SelfService severely affects performance of my RT. Anyone got any ideas Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Best regards, Ruslan. Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Very slow ticket search creation/edit screen 3.8.4
This is the query that takes all the time: # Query_time: 37.415600 Lock_time: 0.000329 Rows_sent: 0 Rows_examined: 4474859 use rt_contdel; SET timestamp=1267024665; SELECT DISTINCT main.* FROM Users main CROSS JOIN ACL ACL_4 JOIN Principals Principals_1 ON ( Principals_1.id = main.id ) JOIN CachedGroupMembers CachedGroupMembers_2 ON ( CachedGroupMembers_2.MemberId = Principals_1.id ) JOIN Groups Groups_3 ON ( Groups_3.id = CachedGroupMembers_2.GroupId ) WHERE (Principals_1.Disabled = '0') AND (ACL_4.PrincipalType = Groups_3.Type) AND (Principals_1.id != '1') AND (Principals_1.PrincipalType = 'User') AND (ACL_4.RightName = 'OwnTicket' OR ACL_4.RightName = 'SuperUser') AND (Groups_3.Domain = 'RT::Queue-Role') AND ((ACL_4.ObjectType = 'RT::Queue') OR (ACL_4.ObjectType = 'RT::System')) ORDER BY main.Name ASC; - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 17:42, Justin Hayes wrote: Ah ok, so even if all my users have OwnTicket for a queue getting that list is still faster than showing all users on the Search screen, even though the resulting list is the same length? Total rows in Users is 543. Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 17:33, Emmanuel Lacour wrote: On Mon, Feb 15, 2010 at 05:18:53PM +, Justin Hayes wrote: That doesn't seem to be an answer. So it's just slow if you want to have a lot of owners?? It doesn't take 40 secs to render the owner dropdown on ticket create/update, so why should the search screen be so bad? They both use the same element. bevause on a ticket, we already ha ve a queue selected and so we filter potential owners for this queue. in search screen, we load all people that may own ticket in any queue (all users that have OwnTicket right). this is a known issue, but that shouldn't be a problem on most systems where people with OwnTicket rights are a few hundreds. tell us more about how many people you have has potential owner in search screen. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Very slow ticket search creation/edit screen 3.8.4
On one of our RT installs it's taking 40secs to render the 'Add Criteria' section of the 'New Search' page (or the edit search screen for that matter). I've narrowed it down to the Owner dropdown. All the time is going to render that. Now the users of this RT allow pretty much anyone (staff and customers) to be ticket owners, so this list is rather long. However this seems like an awfully long time to get a list of owners. I assume it's doing permissions checks.I tried giving the 'privileged user' group 'Own Ticket' permissions to see if that remove the need for all the checking and speed things up, but it didn't help. We're using 3.8.4 Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Very slow ticket search creation/edit screen 3.8.4
That doesn't seem to be an answer. So it's just slow if you want to have a lot of owners?? It doesn't take 40 secs to render the owner dropdown on ticket create/update, so why should the search screen be so bad? They both use the same element. /Elements/SelectOwner Apologies if I'm missing something obvious :) Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 16:52, Jesse Vincent wrote: On Mon, Feb 15, 2010 at 04:30:32PM +, Justin Hayes wrote: On one of our RT installs it's taking 40secs to render the 'Add Criteria' section of the 'New Search' page (or the edit search screen for that matter). I've narrowed it down to the Owner dropdown. All the time is going to render that. Now the users of this RT allow pretty much anyone (staff and customers) to be ticket owners, so this list is rather long. However this seems like an awfully long time to get a list of owners. I assume it's doing permissions checks.I tried giving the 'privileged user' group 'Own Ticket' permissions to see if that remove the need for all the checking and speed things up, but it didn't help. http://wiki.bestpractical.com/view/FAQ Q: Why Bulk update and/or search builder pages are slow to load? === We're using 3.8.4 Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Very slow ticket search creation/edit screen 3.8.4
Sorry maybe I'm confusing things. The list is long, but that's deliberate. The guys who use this RT need everyone able to own tickets, as they pass the ticket back to the customer as owner of a ticket (which isn't how we use it in Support hence our owners list is much smaller - only Support own the tickets). I'm not sure I could get them to change how they work, and even if I could I still don't see why they owners list takes ages to generate on one screen compared to another. Thanks for taking the time to think about this Jesse. It's much appreciated. Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 17:20, Jesse Vincent wrote: On Mon, Feb 15, 2010 at 05:18:53PM +, Justin Hayes wrote: That doesn't seem to be an answer. So it's just slow if you want to have a lot of owners?? It doesn't take 40 secs to render the owner dropdown on ticket create/update, so why should the search screen be so bad? They both use the same element. Humor me and fix your ACLs so your end-users don't have OwnTicket rights. If that doesn't fix it, then maybe there's something else going on, but this is something we see misconfigured rather frequently. /Elements/SelectOwner Apologies if I'm missing something obvious :) Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 16:52, Jesse Vincent wrote: On Mon, Feb 15, 2010 at 04:30:32PM +, Justin Hayes wrote: On one of our RT installs it's taking 40secs to render the 'Add Criteria' section of the 'New Search' page (or the edit search screen for that matter). I've narrowed it down to the Owner dropdown. All the time is going to render that. Now the users of this RT allow pretty much anyone (staff and customers) to be ticket owners, so this list is rather long. However this seems like an awfully long time to get a list of owners. I assume it's doing permissions checks.I tried giving the 'privileged user' group 'Own Ticket' permissions to see if that remove the need for all the checking and speed things up, but it didn't help. http://wiki.bestpractical.com/view/FAQ Q: Why Bulk update and/or search builder pages are slow to load? === We're using 3.8.4 Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Very slow ticket search creation/edit screen 3.8.4
Ah ok, so even if all my users have OwnTicket for a queue getting that list is still faster than showing all users on the Search screen, even though the resulting list is the same length? Total rows in Users is 543. Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 15 Feb 2010, at 17:33, Emmanuel Lacour wrote: On Mon, Feb 15, 2010 at 05:18:53PM +, Justin Hayes wrote: That doesn't seem to be an answer. So it's just slow if you want to have a lot of owners?? It doesn't take 40 secs to render the owner dropdown on ticket create/update, so why should the search screen be so bad? They both use the same element. bevause on a ticket, we already ha ve a queue selected and so we filter potential owners for this queue. in search screen, we load all people that may own ticket in any queue (all users that have OwnTicket right). this is a known issue, but that shouldn't be a problem on most systems where people with OwnTicket rights are a few hundreds. tell us more about how many people you have has potential owner in search screen. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com 2010 RT Training Sessions! San Francisco, CA, USA - Feb 22 23 Dublin, Ireland - Mar 15 16 Boston, MA, USA - April 5 6 Washington DC, USA - Oct 25 26 Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] UseSQLForACLChecks causes slow performance
I've just upgraded to 3.8.4 and as part of that I've enabled UseSQLForACLChecks RT_SiteConfig.pm:Set($UseSQLForACLChecks, 1); However this really slows down load of the homepage. For example with my default layout load time goes from 1s to 6-7s. The Quick Search panel alone takes ~3s (for about 20 queues and 40k tickets) for a user that can see all 20 queues (for myself with SuperUser rights it's much faster). I really would like to keep that setting as it fixes a number of display bugs with homepage panels and ticket results. Anyone have any suggestions? I notice that 3.8.6 is now out. Were there improvements in that that would help with this setting? Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] UseSQLForACLChecks causes slow performance
Well we turned on slow query logging for anything longer than 1 second, and I didn't see any logged. So maybe there are lots of queries all taking 1 sec but the time adds up. I'll try to do some more debugging over the next couple of days, trouble is it's a live server now and its hard to just isolate the queries for just 1 user hitting the main page. Justin On 31 Oct 2009, at 16:09, Kenneth Marshall wrote: On Sat, Oct 31, 2009 at 11:21:08AM +, Justin Hayes wrote: I've just upgraded to 3.8.4 and as part of that I've enabled UseSQLForACLChecks RT_SiteConfig.pm:Set($UseSQLForACLChecks, 1); However this really slows down load of the homepage. For example with my default layout load time goes from 1s to 6-7s. The Quick Search panel alone takes ~3s (for about 20 queues and 40k tickets) for a user that can see all 20 queues (for myself with SuperUser rights it's much faster). I really would like to keep that setting as it fixes a number of display bugs with homepage panels and ticket results. Anyone have any suggestions? I notice that 3.8.6 is now out. Were there improvements in that that would help with this setting? Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com Hi Justin, Have you looked at the queries being run to see if there are obvious missing indexes? I know that it is relatively new as a feature so the tuning still has room for improvement. Regards, Ken - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Upgrade from 3.6.3 to 3.8.4 - image attachments missing/corrupt
I didn't mean the whole attachments table. Putting that in the filesystem would be crazy. I was more talking about files you manually attach (word docs, images etc). These tend to me more throwaway for me than the text of the replies/comments themselves, and we don't have anywhere near as many They could just live in the filesystem in neat subdirectories and be retrieved when someone actually clicks on one to look at. Backup would be easy - just rsync/tar/other option of your choice. But as long as mysql can handle the large DB sizes then I guess it's fine where it is :D Justin On 17 Sep 2009, at 00:33, Aaron Guise wrote: I fully agree Tom, SQL Servers totally own the filesystem equivalent in this regard. Our attachments table is huge and it would be rather difficult to keep a track of them all and ensure every last one is backed up without the MySQL storage system :-) Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Thu, Sep 17, 2009 at 11:00 AM, Tom Lahti t...@bitstatement.net wrote: Justin Hayes wrote: Thanks Aaron. I've always wondered why file attachments are stored in the db at all. I'd have thought those would have been better placed out in the filesystem. Egads! What if the storage database is not local to the web server? How will you perform comprehensive backups? What if your RT has a million attachments, or more? Not to mention the performance hit of using a filesystem as a database, especially with high concurrency at the HTTP level. I have a custom database application designed specifically to store PDFs in the database. It has 30 million documents in it, the database storage is over 4TB. The web-based front-end for it is efficient enough to saturate a 100MBit/sec Internet connection with a single Core-2 duo web server. When I tested this against our old filesystem version of the application, it outperformed the filesystem by more than 100%. Backup is done by dumping the database in chunks in a rotating schedule. Scalability can be accomplished with simple replication to additional read-only SQL servers and using a SQL relay to dispatch SQL commands in a load-balancing fashion. -- -- Tom Lahti BIT Statement LLC (425)251-0833 x 117 http://www.bitstatement.net/ -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Upgrade from 3.6.3 to 3.8.4 - image attachments missing/corrupt
Thanks Aaron for taking the time to dig them out. I'll take a look at them (though fingers crossed adding the binary format options to the DB dump seems to be working so far). Justin On 15 Oct 2009, at 22:09, Aaron Guise wrote: Hi Justin, Sorry it took so long. I was on leave and then couldn't test that my scripts still worked. I have found them now and tested it all out. They are attached here. If you have any trouble please let me know. Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Thu, Sep 17, 2009 at 12:33 PM, Aaron Guise aa...@guise.net.nz wrote: I fully agree Tom, SQL Servers totally own the filesystem equivalent in this regard. Our attachments table is huge and it would be rather difficult to keep a track of them all and ensure every last one is backed up without the MySQL storage system :-) Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Thu, Sep 17, 2009 at 11:00 AM, Tom Lahti t...@bitstatement.net wrote: Justin Hayes wrote: Thanks Aaron. I've always wondered why file attachments are stored in the db at all. I'd have thought those would have been better placed out in the filesystem. Egads! What if the storage database is not local to the web server? How will you perform comprehensive backups? What if your RT has a million attachments, or more? Not to mention the performance hit of using a filesystem as a database, especially with high concurrency at the HTTP level. I have a custom database application designed specifically to store PDFs in the database. It has 30 million documents in it, the database storage is over 4TB. The web-based front-end for it is efficient enough to saturate a 100MBit/sec Internet connection with a single Core-2 duo web server. When I tested this against our old filesystem version of the application, it outperformed the filesystem by more than 100%. Backup is done by dumping the database in chunks in a rotating schedule. Scalability can be accomplished with simple replication to additional read-only SQL servers and using a SQL relay to dispatch SQL commands in a load-balancing fashion. -- -- Tom Lahti BIT Statement LLC (425)251-0833 x 117 http://www.bitstatement.net/ -- ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com GrabAndInsert.zip - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] 3.8.4 looks different in IE7
Hi guys,For me IE7 seems to render 3.8.4 differently to IE8, Safari and Firefox. It's missing all the nice round edges on the various buttons and panel headers etc, and generally doesn't look as nice.IE7:Safari:Is that expected, or is there something wrong with my install perhaps?I can see this in the source:!--[if lt IE 8] link rel="stylesheet" href="" type="text/css" media="all" / ![endif]-- !--[if lt IE 7] link rel="stylesheet" href="" type="text/css" media="all" / ![endif]--So maybe it's a CSS issue?Cheers,Justin -Justin HayesOrbis Support Managerjustin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] 3.8.4 looks different in IE7
Gotta love different browser implementations :) Still one less problem that I actually need to try and fix. Thanks for the quick reply Jesse! Justin On 7 Oct 2009, at 16:53, Jesse Vincent wrote: On Wed, Oct 07, 2009 at 04:48:40PM +0100, Justin Hayes wrote: Hi guys, For me IE7 seems to render 3.8.4 differently to IE8, Safari and Firefox. It's missing all the nice round edges on the various buttons and panel headers etc, and generally doesn't look as nice. Is that expected, or is there something wrong with my install perhaps? IE doesn't support the nice bit of CSS we use to round corners. (Neither does Opera) Adding corner rounding for IE without lots of hand-tooled images and ugly html hacking caused an upsetting performance degradation in page rendering speed. So, it's expected and intentional. But not something I'm happy about. Jesse ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Upgrade from 3.6.3 to 3.8.4 - image attachments missing/corrupt
Thanks a lot for the info and advise Aaron. Don't suppose you kept the scripts you used to dump the attachments and load them back in did you? I'm going to talk to my sysadmins and see if they are using that default-character-set option in the backup dump. If they aren't I'll get them to do me a new dump with that option on and see if it works that time. Cheers, Justin On 15 Sep 2009, at 23:36, Aaron Guise wrote: I had similar problems when moving upto 3.8.1. The previous sysadmin responsible for RT had failed to upgrade the DB properly when going from 3.6.5 to 3.8.0 some time back. All our attachments went screwy too when I tried to upgrade to 3.8.1. In the end what I did is dump the database before upgrade in case I need to go back. Dumped all attachment records to disk via perl, ran the RT upgrade scripts and then updated the attachments table from the ones I had dumped out earlier. This then made all the attachments become working again. RT itself also seemed to get a performance boost !YAY!. And the two ALTER entries in the upgrade script I found as well, Prior to running the upgrade I removed the ones that weren't binary columns e.g. VARBINARY so removing the lines which mentioned something like LONGBLOB. When you use mysqldump to backup the database you just need to make sure to place this --opt --default-character-set=binary in the commandline arguments. That will mean it exports in binary mode to avoid corruption. Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Wed, Sep 16, 2009 at 7:54 AM, Justin Hayes justin.ha...@orbisuk.com wrote: Hi guys, I'm just testing an upgrade from 3.6.3 to 3.8.4. I ran the rt-setup- database fine: /opt/rt_support.openbet.com/sbin/rt-setup-database -dba rt_support -- prompt-for-dba-password --action upgrade Then created the schema upgrade script: perl /opt/rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl *blah* *blah* *password* upgrade.sql Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Tickets.status has type VARCHAR however mapping is missing. Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Users.BlockImg has type CHAR however mapping is missing. -- ** NOTICE: No database changes have been made. ** -- Please review the generated SQL, ensure you have a full backup of your database -- and apply it to your database using a command like: -- mysql -u rt_support -p rt_support queries.sql; cat upgrade.sql ALTER DATABASE rt_support DEFAULT CHARACTER SET utf8; ALTER TABLE ACL DEFAULT CHARACTER SET utf8, MODIFY RightName VARBINARY(25) NOT NULL, MODIFY PrincipalType VARBINARY(25) NOT NULL, MODIFY ObjectType VARBINARY(25) NOT NULL; ALTER TABLE ACL MODIFY RightName VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY PrincipalType VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY ObjectType VARCHAR(25) CHARACTER SET ascii NOT NULL; ALTER TABLE Attachments DEFAULT CHARACTER SET utf8, MODIFY Subject VARBINARY(255) NULL DEFAULT NULL, MODIFY ContentType VARBINARY(80) NULL DEFAULT NULL, MODIFY Filename VARBINARY(255) NULL DEFAULT NULL, MODIFY Headers LONGBLOB NULL DEFAULT NULL, MODIFY MessageId VARBINARY(160) NULL DEFAULT NULL, MODIFY Content LONGBLOB NULL DEFAULT NULL, MODIFY ContentEncoding VARBINARY(80) NULL DEFAULT NULL; ALTER TABLE Attachments MODIFY Subject VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY ContentType VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY Filename VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY Headers LONGTEXT CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY MessageId VARCHAR(160) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY ContentEncoding VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL; . . . Now that looks a bit odd as there are 2 ALTERS per table and the second seems to reverse some bits of the first? Anyway I ran that into my DB. Now when I go into a ticket with an image attached and click on it no image is returned, which is a bit worrying. I'm wondering if it was a problem with the upgrade, or the original db dump provided by my IT systems guys. Perhaps the DB wasn't dumped using binary character set? How could I check that and how should the IT guys have dumped the DB to make sure it was in binary? Any thoughts? Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Upgrade from 3.6.3 to 3.8.4 - image attachments missing/corrupt
Thanks Aaron. I've always wondered why file attachments are stored in the db at all. I'd have thought those would have been better placed out in the filesystem. Cheers, Justin Sent from my iPhone On 16 Sep 2009, at 21:59, Aaron Guise aa...@guise.net.nz wrote: I'll have a look, I'm sure they are here somewhere. Might take a day though. Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Wed, Sep 16, 2009 at 9:20 PM, Justin Hayes justin.ha...@orbisuk.com wrote: Thanks a lot for the info and advise Aaron. Don't suppose you kept the scripts you used to dump the attachments and load them back in did you? I'm going to talk to my sysadmins and see if they are using that default-character-set option in the backup dump. If they aren't I'll get them to do me a new dump with that option on and see if it works that time. Cheers, Justin On 15 Sep 2009, at 23:36, Aaron Guise wrote: I had similar problems when moving upto 3.8.1. The previous sysadmin responsible for RT had failed to upgrade the DB properly when going from 3.6.5 to 3.8.0 some time back. All our attachments went screwy too when I tried to upgrade to 3.8.1. In the end what I did is dump the database before upgrade in case I need to go back. Dumped all attachment records to disk via perl, ran the RT upgrade scripts and then updated the attachments table from the ones I had dumped out earlier. This then made all the attachments become working again. RT itself also seemed to get a performance boost !YAY!. And the two ALTER entries in the upgrade script I found as well, Prior to running the upgrade I removed the ones that weren't binary columns e.g. VARBINARY so removing the lines which mentioned something like LONGBLOB. When you use mysqldump to backup the database you just need to make sure to place this --opt --default-character-set=binary in the commandline arguments. That will mean it exports in binary mode to avoid corruption. Regards, Aaron Guise 07 838 7793 027 212 6638 aa...@guise.net.nz On Wed, Sep 16, 2009 at 7:54 AM, Justin Hayes justin.ha...@orbisuk.com wrote: Hi guys, I'm just testing an upgrade from 3.6.3 to 3.8.4. I ran the rt-setup- database fine: /opt/rt_support.openbet.com/sbin/rt-setup-database -dba rt_support --prompt-for-dba-password --action upgrade Then created the schema upgrade script: perl /opt/rt_support.openbet.com/etc/upgrade/upgrade-mysql- schema.pl *blah* *blah* *password* upgrade.sql Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Tickets.status has type VARCHAR however mapping is missing. Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Users.BlockImg has type CHAR however mapping is missing. -- ** NOTICE: No database changes have been made. ** -- Please review the generated SQL, ensure you have a full backup of your database -- and apply it to your database using a command like: -- mysql -u rt_support -p rt_support queries.sql; cat upgrade.sql ALTER DATABASE rt_support DEFAULT CHARACTER SET utf8; ALTER TABLE ACL DEFAULT CHARACTER SET utf8, MODIFY RightName VARBINARY(25) NOT NULL, MODIFY PrincipalType VARBINARY(25) NOT NULL, MODIFY ObjectType VARBINARY(25) NOT NULL; ALTER TABLE ACL MODIFY RightName VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY PrincipalType VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY ObjectType VARCHAR(25) CHARACTER SET ascii NOT NULL; ALTER TABLE Attachments DEFAULT CHARACTER SET utf8, MODIFY Subject VARBINARY(255) NULL DEFAULT NULL, MODIFY ContentType VARBINARY(80) NULL DEFAULT NULL, MODIFY Filename VARBINARY(255) NULL DEFAULT NULL, MODIFY Headers LONGBLOB NULL DEFAULT NULL, MODIFY MessageId VARBINARY(160) NULL DEFAULT NULL, MODIFY Content LONGBLOB NULL DEFAULT NULL, MODIFY ContentEncoding VARBINARY(80) NULL DEFAULT NULL; ALTER TABLE Attachments MODIFY Subject VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY ContentType VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY Filename VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY Headers LONGTEXT CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY MessageId VARCHAR(160) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY ContentEncoding VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL; . . . Now that looks a bit odd as there are 2 ALTERS per table and the second seems to reverse some bits of the first? Anyway I ran that into my DB. Now when I go into a ticket with an image attached and click on it no image is returned, which is a bit worrying. I'm wondering if it was a problem with the upgrade, or the original db dump provided by my IT systems guys. Perhaps the DB wasn't dumped using binary character set? How could I check that and how should the IT guys have dumped the DB
[rt-users] Upgrade from 3.6.3 to 3.8.4 - image attachments missing/corrupt
Hi guys, I'm just testing an upgrade from 3.6.3 to 3.8.4. I ran the rt-setup- database fine: /opt/rt_support.openbet.com/sbin/rt-setup-database -dba rt_support -- prompt-for-dba-password --action upgrade Then created the schema upgrade script: perl /opt/rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl *blah* *blah* *password* upgrade.sql Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Tickets.status has type VARCHAR however mapping is missing. Use of uninitialized value in join or string at /opt/ rt_support.openbet.com/etc/upgrade/upgrade-mysql-schema.pl line 261. .Users.BlockImg has type CHAR however mapping is missing. -- ** NOTICE: No database changes have been made. ** -- Please review the generated SQL, ensure you have a full backup of your database -- and apply it to your database using a command like: -- mysql -u rt_support -p rt_support queries.sql; cat upgrade.sql ALTER DATABASE rt_support DEFAULT CHARACTER SET utf8; ALTER TABLE ACL DEFAULT CHARACTER SET utf8, MODIFY RightName VARBINARY(25) NOT NULL, MODIFY PrincipalType VARBINARY(25) NOT NULL, MODIFY ObjectType VARBINARY(25) NOT NULL; ALTER TABLE ACL MODIFY RightName VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY PrincipalType VARCHAR(25) CHARACTER SET ascii NOT NULL, MODIFY ObjectType VARCHAR(25) CHARACTER SET ascii NOT NULL; ALTER TABLE Attachments DEFAULT CHARACTER SET utf8, MODIFY Subject VARBINARY(255) NULL DEFAULT NULL, MODIFY ContentType VARBINARY(80) NULL DEFAULT NULL, MODIFY Filename VARBINARY(255) NULL DEFAULT NULL, MODIFY Headers LONGBLOB NULL DEFAULT NULL, MODIFY MessageId VARBINARY(160) NULL DEFAULT NULL, MODIFY Content LONGBLOB NULL DEFAULT NULL, MODIFY ContentEncoding VARBINARY(80) NULL DEFAULT NULL; ALTER TABLE Attachments MODIFY Subject VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY ContentType VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY Filename VARCHAR(255) CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY Headers LONGTEXT CHARACTER SET utf8 NULL DEFAULT NULL, MODIFY MessageId VARCHAR(160) CHARACTER SET ascii NULL DEFAULT NULL, MODIFY ContentEncoding VARCHAR(80) CHARACTER SET ascii NULL DEFAULT NULL; . . . Now that looks a bit odd as there are 2 ALTERS per table and the second seems to reverse some bits of the first? Anyway I ran that into my DB. Now when I go into a ticket with an image attached and click on it no image is returned, which is a bit worrying. I'm wondering if it was a problem with the upgrade, or the original db dump provided by my IT systems guys. Perhaps the DB wasn't dumped using binary character set? How could I check that and how should the IT guys have dumped the DB to make sure it was in binary? Any thoughts? Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Relationship Graphs not working in RT 3.8.4
Simply installing IPC::Run::SafeHandles did the trick. Thanks Kevin! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 5 Aug 2009, at 17:36, Kevin Falcone wrote: On Wed, Aug 05, 2009 at 05:11:22PM +0100, Justin Hayes wrote: I didn't do the initial install, but I'm guessing probably not... Is there any way to do it retrospectively? Just try installing IPC::Run::SafeHandles You can reconfigure while saying --enable-graphviz and just run make testdeps -kevin On 5 Aug 2009, at 16:58, Kevin Falcone wrote: On Wed, Aug 05, 2009 at 12:26:16PM +0100, Justin Hayes wrote: Anyone had any problems getting the ticket relation graphing stuff working under RT3.8.4? I've installed various libgd packages and installed GD and GraphViz over CPAN. I now get the 'Graph' link in the Links section of Ticket Metadata but clicking on it gives me this: Did you run ./configure with --enable-graphviz ? My guess is that you're missing IPC::Run::SafeHandles -kevin Error during compilation of /opt/rt3/share/html/Ticket/Graphs/ index.html: Attempt to reload RT/Graph/Tickets.pm aborted. Compilation failed in require at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/ perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/ HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/ HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/ mason_handler.fcgi:79] BEGIN failed--compilation aborted at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/ perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/ HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/ HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/ mason_handler.fcgi:79] Cheers, Justin - Justin Hayes Orbis Support Manager ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Dashboard subscriptions not working in RT 3.8.4
Hopefully my last question before I get 3.8.4 live :) I've playing around with the new 'dashboards' functionality. It all looks very useful, and I think my customers would like it as well, however I can't seem to get subscriptions to work. I've set up a dashboard and configured it to mail me daily at 1pm but I never get any emails. Are there any configs I have to turn on to get this to work? I'm getting other RT mails from the system so it's not like mail is broken or something. Cheers in advance, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Dashboard subscriptions not working in RT 3.8.4
Totally my bad - forgot to check the README. Thanks for that Joop and sorry for wasting your time! Justin 10 Set up automated recurring tasks (cronjobs): To generate email digest messages, you must arrange for the provided utility to be run once daily, and once weekly. You may also want to arrange for the rt-email-dashboards utility to be run hourly. For example, if your task scheduler is cron, you can configure it as follows: crontab -e# as the RT administrator (probably root) # insert the following lines: 0 0 * * * /opt/rt3/sbin/rt-email-digest -m daily 0 0 * * 0 /opt/rt3/sbin/rt-email-digest -m weekly 0 * * * * /opt/rt3/sbin/rt-email-dashboards - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 6 Aug 2009, at 13:38, Joop van de Wege wrote: Justin Hayes wrote: Hopefully my last question before I get 3.8.4 live :) I've playing around with the new 'dashboards' functionality. It all looks very useful, and I think my customers would like it as well, however I can't seem to get subscriptions to work. I've set up a dashboard and configured it to mail me daily at 1pm but I never get any emails. Are there any configs I have to turn on to get this to work? I'm getting other RT mails from the system so it's not like mail is broken or something. Did you setup your crontab to contain the following? 0 * * * * /opt/rt3/sbin/rt-email-dashboards Regards, Joop ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Relationship Graphs not working in RT 3.8.4
Anyone had any problems getting the ticket relation graphing stuff working under RT3.8.4? I've installed various libgd packages and installed GD and GraphViz over CPAN. I now get the 'Graph' link in the Links section of Ticket Metadata but clicking on it gives me this: Error during compilation of /opt/rt3/share/html/Ticket/Graphs/ index.html: Attempt to reload RT/Graph/Tickets.pm aborted. Compilation failed in require at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/opt/rt3/ share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/Mason/ Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/ perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/ Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/ usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/ Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/ share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/ CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/ opt/rt3/bin/mason_handler.fcgi:79] BEGIN failed--compilation aborted at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/ rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/ Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/ share/perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/ Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/ usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/ Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/ share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/ CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/ opt/rt3/bin/mason_handler.fcgi:79] Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Relationship Graphs not working in RT 3.8.4
I've restarted apache but I haven't restarted the box itself. The charts from search results started working after installing GD and GraphWiz so I assumed that the relationship ones would as well. I'll try a complete restart. Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 5 Aug 2009, at 14:27, Ruslan Zakirov wrote: Hello Justin, Have you restarted the server after install? On Wed, Aug 5, 2009 at 3:26 PM, Justin Hayesjustin.ha...@orbisuk.com wrote: Anyone had any problems getting the ticket relation graphing stuff working under RT3.8.4? I've installed various libgd packages and installed GD and GraphViz over CPAN. I now get the 'Graph' link in the Links section of Ticket Metadata but clicking on it gives me this: Error during compilation of /opt/rt3/share/html/Ticket/Graphs/ index.html: Attempt to reload RT/Graph/Tickets.pm aborted. Compilation failed in require at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/mason_handler.fcgi:79] BEGIN failed--compilation aborted at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/mason_handler.fcgi:79] Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Best regards, Ruslan. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Relationship Graphs not working in RT 3.8.4
Tried restarting the server itself, but still get the same error :( Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 5 Aug 2009, at 14:27, Ruslan Zakirov wrote: Hello Justin, Have you restarted the server after install? On Wed, Aug 5, 2009 at 3:26 PM, Justin Hayesjustin.ha...@orbisuk.com wrote: Anyone had any problems getting the ticket relation graphing stuff working under RT3.8.4? I've installed various libgd packages and installed GD and GraphViz over CPAN. I now get the 'Graph' link in the Links section of Ticket Metadata but clicking on it gives me this: Error during compilation of /opt/rt3/share/html/Ticket/Graphs/ index.html: Attempt to reload RT/Graph/Tickets.pm aborted. Compilation failed in require at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/mason_handler.fcgi:79] BEGIN failed--compilation aborted at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/mason_handler.fcgi:79] Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Best regards, Ruslan. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Relationship Graphs not working in RT 3.8.4
I didn't do the initial install, but I'm guessing probably not... Is there any way to do it retrospectively? Thanks again Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 5 Aug 2009, at 16:58, Kevin Falcone wrote: On Wed, Aug 05, 2009 at 12:26:16PM +0100, Justin Hayes wrote: Anyone had any problems getting the ticket relation graphing stuff working under RT3.8.4? I've installed various libgd packages and installed GD and GraphViz over CPAN. I now get the 'Graph' link in the Links section of Ticket Metadata but clicking on it gives me this: Did you run ./configure with --enable-graphviz ? My guess is that you're missing IPC::Run::SafeHandles -kevin Error during compilation of /opt/rt3/share/html/Ticket/Graphs/ index.html: Attempt to reload RT/Graph/Tickets.pm aborted. Compilation failed in require at /opt/rt3/share/html/Ticket/Graphs/index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/ perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/ HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/ perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/ Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/ HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/ mason_handler.fcgi:79] BEGIN failed--compilation aborted at /opt/rt3/share/html/Ticket/Graphs/ index.html line 78. Stack: [/opt/rt3/share/html/Ticket/Graphs/index.html:78] [/usr/share/ perl5/HTML/Mason/Interp.pm:811] [/usr/share/perl5/HTML/Mason/Interp.pm:441] [/usr/share/perl5/ HTML/Mason/Request.pm:246] [/usr/share/perl5/HTML/Mason/Request.pm:211] [/opt/rt3/bin/../lib/RT/Interface/Web/Request.pm:68] [/usr/share/ perl5/Class/Container.pm:275] [/usr/share/perl5/Class/Container.pm:353] [/usr/share/perl5/HTML/ Mason/Interp.pm:348] [/usr/share/perl5/HTML/Mason/Interp.pm:342] [/usr/share/perl5/ HTML/Mason/CGIHandler.pm:124] [/usr/share/perl5/HTML/Mason/CGIHandler.pm:73] [/opt/rt3/bin/ mason_handler.fcgi:79] Cheers, Justin - Justin Hayes Orbis Support Manager ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] MakeClicky call back example doesn't seem to work (RT3.8.4)
Ah that might be it - I have got the httpurl one turned on as well. I've got the patch and will let you know how it goes. Thanks again! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 22:43, Kevin Falcone wrote: On Tue, Jul 14, 2009 at 09:53:30PM +0100, Justin Hayes wrote: Here's the output from that debug you suggested adding: My colleage suspects you're running into a bug where the other two MakeClicky extensions are clobbering captures. You didn't mention having them enabled, so I was testing without them. He may followup with a patch to try -kevin Jul 14 21:50:42 cetus RT: $VAR1 = { 'all_matches' = [ 'Ticket #1', undef, undef, undef, undef, undef, undef, undef, undef ], 'Message' = [ { 'text' = 'Ticket #1 [b]this is bold[/ b]', 'empty' = '', 'quoter' = '', 'raw' = 'Ticket #1 [b]this is bold[/b]' }, { 'text' = '-', 'empty' = '', 'quoter' = '', 'raw' = '-' - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 21:14, Kevin Falcone wrote: On Tue, Jul 14, 2009 at 06:12:16PM +0100, Justin Hayes wrote: Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : I just tried with your code and with the code copied from pod and it worked for me without any tweaking (RT 3.8.4). I suggest you add the following piece of debugging and see what you get. Otherwise you're going to need to instrument MakeClicky itself. You don't mention your RT version of your perl version, both of which might be interesting. -kevin %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; use Data::Dumper; $RT::Logger-error(Dumper \%args); my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id$args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] MakeClicky call back example doesn't seem to work (RT3.8.4)
Works great. Thanks very much for fixing this so quickly!! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 22:40, Ruslan Zakirov wrote: Try attached patch. Don't forget to flush mason cache and send us feedback. On Wed, Jul 15, 2009 at 12:53 AM, Justin Hayesjustin.ha...@orbisuk.com wrote: Here's the output from that debug you suggested adding: Jul 14 21:50:42 cetus RT: $VAR1 = { 'all_matches' = [ 'Ticket #1', undef, undef, undef, undef, undef, undef, undef, undef ], 'Message' = [ { 'text' = 'Ticket #1 [b]this is bold[/ b]', 'empty' = '', 'quoter' = '', 'raw' = 'Ticket #1 [b]this is bold[/b]' }, { 'text' = '-', 'empty' = '', 'quoter' = '', 'raw' = '-' - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 21:14, Kevin Falcone wrote: On Tue, Jul 14, 2009 at 06:12:16PM +0100, Justin Hayes wrote: Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : I just tried with your code and with the code copied from pod and it worked for me without any tweaking (RT 3.8.4). I suggest you add the following piece of debugging and see what you get. Otherwise you're going to need to instrument MakeClicky itself. You don't mention your RT version of your perl version, both of which might be interesting. -kevin %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; use Data::Dumper; $RT::Logger-error(Dumper \%args); my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id $args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com -- Best regards, Ruslan. *** NDS UK IT scanned this email for malicious content *** *** IMPORTANT: Do not open attachments from unrecognized senders *** RT-3.8.4-make_click_all_matches.patch ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] MakeClicky call back example doesn't seem to work (RT3.8.4)
Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id $args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT It definitely does something, as my Ticket #1 text becomes clickable, however it links to a Display.html url with no ticket id Ticket/Display.html?id= I guess the my $id = $args{'all_matches'}[1]; bit just doesn't have a value, but this is the example in the official docs so I'm surprised it doesn't work Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] MakeClicky call back example doesn't seem to work(RT3.8.4)
Also tried the following: $actions-{'bold_text'} = sub { my %args = @_; my $id = $args{'all_matches'}[1]; return qq{b$idb}; }; push @$types, { # name, that should be used in config to activate action name = 'bold_text', # regular expression that matches text 'ticket #xxx' regex = qr{\[b\](.*)\[\/b\]}i, # name of the action that should be applied action = 'bold_text', }; In this case return qq{b$idb}; gives nothing return qq{b$args{value}b}; returns the whole pattern matched (which obviously I don't want as I want to strip the [b][/b] tags off). So it really looks like $args{'all_matches'}[1]; etc doesn't get populated with the sub-matches in a regex like qr{\[b\](.*)\[\/b\]}i, I would expect [1] to contain the stuff in the (.*) sub-match (unless the example I'm basing mine on is wrong and I'm doing something stupid) Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 18:12, Justin Hayes wrote: Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id $args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT It definitely does something, as my Ticket #1 text becomes clickable, however it links to a Display.html url with no ticket id Ticket/Display.html?id= I guess the my $id = $args{'all_matches'}[1]; bit just doesn't have a value, but this is the example in the official docs so I'm surprised it doesn't work Cheers, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] MakeClicky call back example doesn't seem to work (RT3.8.4)
Thanks Kevin. RT version is 3.8.4 as given in the subject of my mail. Perl is 5.10.0 I believe. I'll try adding that debug and see what I get. Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 21:14, Kevin Falcone wrote: On Tue, Jul 14, 2009 at 06:12:16PM +0100, Justin Hayes wrote: Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : I just tried with your code and with the code copied from pod and it worked for me without any tweaking (RT 3.8.4). I suggest you add the following piece of debugging and see what you get. Otherwise you're going to need to instrument MakeClicky itself. You don't mention your RT version of your perl version, both of which might be interesting. -kevin %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; use Data::Dumper; $RT::Logger-error(Dumper \%args); my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id $args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] MakeClicky call back example doesn't seem to work (RT3.8.4)
Here's the output from that debug you suggested adding: Jul 14 21:50:42 cetus RT: $VAR1 = { 'all_matches' = [ 'Ticket #1', undef, undef, undef, undef, undef, undef, undef, undef ], 'Message' = [ { 'text' = 'Ticket #1 [b]this is bold[/b]', 'empty' = '', 'quoter' = '', 'raw' = 'Ticket #1 [b]this is bold[/b]' }, { 'text' = '-', 'empty' = '', 'quoter' = '', 'raw' = '-' - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com On 14 Jul 2009, at 21:14, Kevin Falcone wrote: On Tue, Jul 14, 2009 at 06:12:16PM +0100, Justin Hayes wrote: Hi, Has anyone tried the example MakeClicky Callback in the docs (extending_clickable_links.pod)? I've added the following as a callback as it says : I just tried with your code and with the code copied from pod and it worked for me without any tweaking (RT 3.8.4). I suggest you add the following piece of debugging and see what you get. Otherwise you're going to need to instrument MakeClicky itself. You don't mention your RT version of your perl version, both of which might be interesting. -kevin %ARGS $types = [] $actions = {} /%ARGS %INIT my $web_path = RT-Config-Get('WebPath'); # action that takes ticket ID as argument and returns link to the ticket $actions-{'link_ticket'} = sub { my %args = @_; use Data::Dumper; $RT::Logger-error(Dumper \%args); my $id = $args{'all_matches'}[1]; return qq{a href=$web_path/Ticket/Display.html?id=$id $args{value}/a}; }; # add action to the list push @$types, { # name, that should be used in config to activate action name = 'short_ticket_link', # regular expression that matches text 'ticket #xxx' regex = qr{ticket\s+#(\d+)}i, # name of the action that should be applied action = 'link_ticket', }; /%INIT ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Bug in 3.6.3 At A Glance saved search display
Ok many thanks Kevin. I'm hoping to set up a 3.8 test environment so I'll have a play around with that setting on there. Justin On 11 May 2009, at 18:02, Kevin Falcone wrote: On May 11, 2009, at 12:50 PM, Justin Hayes wrote: Many thanks Kevin. Does it have any other effects? Just trying to understand why it's off by default. Looks like another reason to work on the upgrade. It is documented in the config file, but the summary is that we're not convinced it works in every case (this is a really complex query). -kevin On 11 May 2009, at 15:20, Kevin Falcone wrote: On May 11, 2009, at 2:36 AM, Justin Hayes wrote: I create a saved search which will display tickets created in the last week. For me (who can see all queues) I see 10 results (the last 10 tickets created). However another user might only have permission to see 1 queue. Instead of seeing the last 10 tickets created in their queue they only seem to see whichever of the last 10 are in their queue, so they might only see 1 or 2 tickets, which is fairly misleading. I guess it's picking the last 10 from the resultset and THEN applying what you can view, which to me seems wrong. It should work out what the user can see, then show the last 10. Does anyone know if this can be fixed, or if it is fixed in 3.8? There is an option for this in 3.8 that isn't enabled by default. You want the config option UseSQLForACLChecks -kevin ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Bug in 3.6.3 At A Glance saved search display
Hi, Here's an example: I create a saved search which will display tickets created in the last week. For me (who can see all queues) I see 10 results (the last 10 tickets created). However another user might only have permission to see 1 queue. Instead of seeing the last 10 tickets created in their queue they only seem to see whichever of the last 10 are in their queue, so they might only see 1 or 2 tickets, which is fairly misleading. I guess it's picking the last 10 from the resultset and THEN applying what you can view, which to me seems wrong. It should work out what the user can see, then show the last 10. Does anyone know if this can be fixed, or if it is fixed in 3.8? Thanks, Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Bug in 3.6.3 At A Glance saved search display
Many thanks Kevin. Does it have any other effects? Just trying to understand why it's off by default. Looks like another reason to work on the upgrade. Cheers, Justin On 11 May 2009, at 15:20, Kevin Falcone wrote: On May 11, 2009, at 2:36 AM, Justin Hayes wrote: I create a saved search which will display tickets created in the last week. For me (who can see all queues) I see 10 results (the last 10 tickets created). However another user might only have permission to see 1 queue. Instead of seeing the last 10 tickets created in their queue they only seem to see whichever of the last 10 are in their queue, so they might only see 1 or 2 tickets, which is fairly misleading. I guess it's picking the last 10 from the resultset and THEN applying what you can view, which to me seems wrong. It should work out what the user can see, then show the last 10. Does anyone know if this can be fixed, or if it is fixed in 3.8? There is an option for this in 3.8 that isn't enabled by default. You want the config option UseSQLForACLChecks -kevin ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow loading of long tickets in 3.6.3
Thanks Ken but I think I'm already doing that in my SkipTransaction callback: %init my $ttype; $ttype = $Transaction-Type; $$skip = 1 if (($_SkipSystemMessages) ((($Transaction-Creator eq 1) ($Transaction-Type ne 'Status') ($Transaction-Type ne 'Comment')) || (($Transaction-Creator eq 177) (($Transaction-Type ne 'Give') ($Transaction-Type ne 'Correspond'))) || ($Transaction- Type eq 'CustomField') || ($Transaction-Type eq 'Set' $Transaction-Field eq 'TimeWorked')) ); my $type = $Transaction-Type; /%init I'm basically only showing comment/correspondence and status changes. I'm not sure I can skip anything else. So I think I'm going to need to look into ShowTransaction and ShowTransactionAttachments to see if there are savings to be had. I only need to save a couple of hundredths of a second per transaction to make a big difference on the longer tickets. I'm also looking at moving to 3.8.2, but until I can get one set up to benchmark against I don't know if it's any faster in this regard. Cheers, Justin On 28 Apr 2009, at 18:54, Kenneth Marshall wrote: Just looking at what we have: while ( my $Transaction = $Transactions-Next ) { my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; my $transcreator = $Transaction-Creator; next if ( $transcreator == 1 and $ShowHeaders != 1 ); # RT_System next if ( $transcreator == 96711 and $ShowHeaders != 1 ); # escalate $i++; ... I skip all the RT_System transactions unless the full option is specified. That seemed to make the biggest difference. Everything else ended up being a micro-optimization at best. As you have noticed, the key is to reduce the amount of transactions processed completely. Just nuking the RT_System transactions really helped. My two cents, Ken On Tue, Apr 28, 2009 at 06:29:13PM +0100, Justin Hayes wrote: I'm already skipping a load of transactions, but there's a minimum I can cut down to. In ShowHistory we have this loop, where all the time goes. I've added interval logging round the 3 main sections 1) The skip checking 2) The attachment grepping 3) The call to ShowTransaction while ( my $Transaction = $Transactions-Next ) { my $t10 = [gettimeofday]; my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; $i++; my $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop1 - time taken: $elapsed10); my @trans_attachments = grep { $_-TransactionId == $Transaction- Id } @attachments; my $trans_content = {}; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop2 - time taken: $elapsed10); #Args is first because we're clobbering the Attachments parameter $m-comp( 'ShowTransaction', %ARGS, AttachPath = $AttachPath, UpdatePath = $UpdatePath, Ticket = $Ticket, Transaction = $Transaction, ShowHeaders = $ShowHeaders, Collapsed= $Collapsed, RowNum = $i, EntryNum = (( $Transaction-Type =~ /^(Create|Correspond|Comment)$/ ) ? (++ $EntryNum) : ), ShowTitleBarCommands = $ShowTitleBarCommands, Attachments = \...@trans_attachments, AttachmentContent= $trans_content, LastTransaction = $Transactions-IsLast ); # manually flush the content buffer after each txn, so the user sees # some update $m-flush_buffer(); $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop3 - time taken: $elapsed10); } We get output like this (time taken to get from the start of the iteration to the logging point) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004148 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.010705 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.048624 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:133) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004812 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.011379 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.024753 (/opt/rt3/local/html/Ticket/Elements
Re: [rt-users] Slow loading of long tickets in 3.6.3
Alternatively I might look into only showing say the first 10 transactions and last 30 for tickets with over 100 transactions or something (numbers are initial guesses). I could pass a flag into ShowHistory to decide whether or not to do this (so I can disable it on the full History screen) and then work out how much to show based on how many transactions the ticket has. Justin On 29 Apr 2009, at 07:35, Justin Hayes wrote: Thanks Ken but I think I'm already doing that in my SkipTransaction callback: %init my $ttype; $ttype = $Transaction-Type; $$skip = 1 if (($_SkipSystemMessages) ((($Transaction-Creator eq 1) ($Transaction-Type ne 'Status') ($Transaction-Type ne 'Comment')) || (($Transaction-Creator eq 177) (($Transaction-Type ne 'Give') ($Transaction-Type ne 'Correspond'))) || ($Transaction- Type eq 'CustomField') || ($Transaction-Type eq 'Set' $Transaction-Field eq 'TimeWorked')) ); my $type = $Transaction-Type; /%init I'm basically only showing comment/correspondence and status changes. I'm not sure I can skip anything else. So I think I'm going to need to look into ShowTransaction and ShowTransactionAttachments to see if there are savings to be had. I only need to save a couple of hundredths of a second per transaction to make a big difference on the longer tickets. I'm also looking at moving to 3.8.2, but until I can get one set up to benchmark against I don't know if it's any faster in this regard. Cheers, Justin On 28 Apr 2009, at 18:54, Kenneth Marshall wrote: Just looking at what we have: while ( my $Transaction = $Transactions-Next ) { my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; my $transcreator = $Transaction-Creator; next if ( $transcreator == 1 and $ShowHeaders != 1 ); # RT_System next if ( $transcreator == 96711 and $ShowHeaders != 1 ); # escalate $i++; ... I skip all the RT_System transactions unless the full option is specified. That seemed to make the biggest difference. Everything else ended up being a micro-optimization at best. As you have noticed, the key is to reduce the amount of transactions processed completely. Just nuking the RT_System transactions really helped. My two cents, Ken On Tue, Apr 28, 2009 at 06:29:13PM +0100, Justin Hayes wrote: I'm already skipping a load of transactions, but there's a minimum I can cut down to. In ShowHistory we have this loop, where all the time goes. I've added interval logging round the 3 main sections 1) The skip checking 2) The attachment grepping 3) The call to ShowTransaction while ( my $Transaction = $Transactions-Next ) { my $t10 = [gettimeofday]; my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; $i++; my $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop1 - time taken: $elapsed10); my @trans_attachments = grep { $_-TransactionId == $Transaction- Id } @attachments; my $trans_content = {}; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop2 - time taken: $elapsed10); #Args is first because we're clobbering the Attachments parameter $m-comp( 'ShowTransaction', %ARGS, AttachPath = $AttachPath, UpdatePath = $UpdatePath, Ticket = $Ticket, Transaction = $Transaction, ShowHeaders = $ShowHeaders, Collapsed= $Collapsed, RowNum = $i, EntryNum = (( $Transaction-Type =~ /^(Create|Correspond|Comment)$/ ) ? (++ $EntryNum) : ), ShowTitleBarCommands = $ShowTitleBarCommands, Attachments = \...@trans_attachments, AttachmentContent= $trans_content, LastTransaction = $Transactions-IsLast ); # manually flush the content buffer after each txn, so the user sees # some update $m-flush_buffer(); $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop3 - time taken: $elapsed10); } We get output like this (time taken to get from the start of the iteration to the logging point) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004148 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.010705 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.048624 (/opt/rt3/local
Re: [rt-users] Slow loading of long tickets in 3.6.3
I've implemented stripping out the middle portion of long tickets (set to more than 200 transactions atm) and here's how I did it in case you would like to do something similar. For a ticket with 100 comments/ replies that was taking 10s to load it now takes 3s, and it won't get much slower even if the tickets get longer. This is my diff from Ticket/Elements/ShowHistory: /opt/rt3/local/html/Ticket/Elements$ diff ../../../../share/html/ Ticket/Elements/ShowHistory ShowHistory 84a85,87 my $trans_count = $Transactions-Count; my $speed_msg_shown = 0; 85a89,117 if ($TruncateLongTicket $trans_count 200 (($j 10) ($j ($trans_count - 50 { $j++; if (( $Transaction-Type =~ /^(Create|Correspond|Comment) $/ )) { $EntryNum++; } if ($speed_msg_shown == 0) { /%perl div class=ticket-transaction links% $i % 2 ? ' odd' : ' even' % table width=100% cellspacing=0 cellpadding=2 border=0 trtd rowspan=2 valign=top class=typenbsp;/td td class=description brbrbrbrbr brbrbrbrbr brbrbrbrbr Some entries have been removed to speed the loading of this long ticket.brPlease see the a href=% $RT::WebPath %/Ticket/ History.html?id=% $Ticket-Id %History/a for a full Journal of this Ticket. brbrbrbrbr brbrbrbrbr brbrbrbrbr /td/tr /table /div %perl } $speed_msg_shown = 1; next; } $j++; 100d131 111a143 EntryNum= (( $Transaction-Type =~ /^(Create| Correspond|Comment)$/ ) ? (++$EntryNum) : ), 146a179,181 my $j; my $LastTransId; my $EntryNum; 161a197 $TruncateLongTicket = 1 You can then use TruncateLongTicket as an argument to ShowHistory from Display.html, History.html etc etc to decide when you want to do the truncating.. Hope this is useful to someone! Justin On 29 Apr 2009, at 08:01, Justin Hayes wrote: Alternatively I might look into only showing say the first 10 transactions and last 30 for tickets with over 100 transactions or something (numbers are initial guesses). I could pass a flag into ShowHistory to decide whether or not to do this (so I can disable it on the full History screen) and then work out how much to show based on how many transactions the ticket has. Justin On 29 Apr 2009, at 07:35, Justin Hayes wrote: Thanks Ken but I think I'm already doing that in my SkipTransaction callback: %init my $ttype; $ttype = $Transaction-Type; $$skip = 1 if (($_SkipSystemMessages) ((($Transaction-Creator eq 1) ($Transaction-Type ne 'Status') ($Transaction-Type ne 'Comment')) || (($Transaction-Creator eq 177) (($Transaction- Type ne 'Give') ($Transaction-Type ne 'Correspond'))) || ($Transaction- Type eq 'CustomField') || ($Transaction-Type eq 'Set' $Transaction-Field eq 'TimeWorked')) ); my $type = $Transaction-Type; /%init I'm basically only showing comment/correspondence and status changes. I'm not sure I can skip anything else. So I think I'm going to need to look into ShowTransaction and ShowTransactionAttachments to see if there are savings to be had. I only need to save a couple of hundredths of a second per transaction to make a big difference on the longer tickets. I'm also looking at moving to 3.8.2, but until I can get one set up to benchmark against I don't know if it's any faster in this regard. Cheers, Justin On 28 Apr 2009, at 18:54, Kenneth Marshall wrote: Just looking at what we have: while ( my $Transaction = $Transactions-Next ) { my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; my $transcreator = $Transaction-Creator; next if ( $transcreator == 1 and $ShowHeaders != 1 ); # RT_System next if ( $transcreator == 96711 and $ShowHeaders != 1 ); # escalate $i++; ... I skip all the RT_System transactions unless the full option is specified. That seemed to make the biggest difference. Everything else ended up being a micro-optimization at best. As you have noticed, the key is to reduce the amount of transactions processed completely. Just nuking the RT_System transactions really helped. My two cents, Ken On Tue, Apr 28, 2009 at 06:29:13PM +0100, Justin Hayes wrote: I'm already skipping a load of transactions, but there's a minimum I can cut down to. In ShowHistory we have this loop, where all the time goes. I've added interval logging round the 3 main sections 1) The skip checking 2) The attachment grepping 3) The call to ShowTransaction while ( my $Transaction = $Transactions-Next ) { my $t10 = [gettimeofday]; my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip
Re: [rt-users] Slow loading of long tickets in 3.6.3
Hope it comes in useful. It's a bit rough and ready but could easily be improved on. Justin On 29 Apr 2009, at 13:44, Kenneth Marshall wrote: Justin, Thank you for the code for your addition. It will be useful to have it in the bag-o-tricks when this problem rears its head here. Cheers, Ken On Wed, Apr 29, 2009 at 09:53:36AM +0100, Justin Hayes wrote: I've implemented stripping out the middle portion of long tickets (set to more than 200 transactions atm) and here's how I did it in case you would like to do something similar. For a ticket with 100 comments/ replies that was taking 10s to load it now takes 3s, and it won't get much slower even if the tickets get longer. This is my diff from Ticket/Elements/ShowHistory: /opt/rt3/local/html/Ticket/Elements$ diff ../../../../share/html/Ticket/Elements/ShowHistory ShowHistory 84a85,87 my $trans_count = $Transactions-Count; my $speed_msg_shown = 0; 85a89,117 if ($TruncateLongTicket $trans_count 200 (($j 10) ($j ($trans_count - 50 { $j++; if (( $Transaction-Type =~ /^(Create|Correspond|Comment) $/ )) { $EntryNum++; } if ($speed_msg_shown == 0) { /%perl div class=ticket-transaction links% $i % 2 ? ' odd' : ' even' % table width=100% cellspacing=0 cellpadding=2 border=0 trtd rowspan=2 valign=top class=typenbsp;/td td class=description brbrbrbrbr brbrbrbrbr brbrbrbrbr Some entries have been removed to speed the loading of this long ticket.brPlease see the a href=% $RT::WebPath %/Ticket/History.html?id=% $Ticket-Id %History/a for a full Journal of this Ticket. brbrbrbrbr brbrbrbrbr brbrbrbrbr /td/tr /table /div %perl } $speed_msg_shown = 1; next; } $j++; 100d131 111a143 EntryNum= (( $Transaction-Type =~ /^(Create|Correspond|Comment)$/ ) ? (++$EntryNum) : ), 146a179,181 my $j; my $LastTransId; my $EntryNum; 161a197 $TruncateLongTicket = 1 You can then use TruncateLongTicket as an argument to ShowHistory from Display.html, History.html etc etc to decide when you want to do the truncating.. Hope this is useful to someone! Justin On 29 Apr 2009, at 08:01, Justin Hayes wrote: Alternatively I might look into only showing say the first 10 transactions and last 30 for tickets with over 100 transactions or something (numbers are initial guesses). I could pass a flag into ShowHistory to decide whether or not to do this (so I can disable it on the full History screen) and then work out how much to show based on how many transactions the ticket has. Justin On 29 Apr 2009, at 07:35, Justin Hayes wrote: Thanks Ken but I think I'm already doing that in my SkipTransaction callback: %init my $ttype; $ttype = $Transaction-Type; $$skip = 1 if (($_SkipSystemMessages) ((($Transaction-Creator eq 1) ($Transaction-Type ne 'Status') ($Transaction-Type ne 'Comment')) || (($Transaction-Creator eq 177) (($Transaction- Type ne 'Give') ($Transaction-Type ne 'Correspond'))) || ($Transaction- Type eq 'CustomField') || ($Transaction-Type eq 'Set' $Transaction-Field eq 'TimeWorked')) ); my $type = $Transaction-Type; /%init I'm basically only showing comment/correspondence and status changes. I'm not sure I can skip anything else. So I think I'm going to need to look into ShowTransaction and ShowTransactionAttachments to see if there are savings to be had. I only need to save a couple of hundredths of a second per transaction to make a big difference on the longer tickets. I'm also looking at moving to 3.8.2, but until I can get one set up to benchmark against I don't know if it's any faster in this regard. Cheers, Justin On 28 Apr 2009, at 18:54, Kenneth Marshall wrote: Just looking at what we have: while ( my $Transaction = $Transactions-Next ) { my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; my $transcreator = $Transaction-Creator; next if ( $transcreator == 1 and $ShowHeaders != 1 ); # RT_System next if ( $transcreator == 96711 and $ShowHeaders != 1 ); # escalate $i++; ... I skip all the RT_System transactions unless the full option is specified. That seemed to make the biggest difference. Everything else ended up being a micro-optimization at best. As you have noticed, the key is to reduce the amount of transactions processed completely. Just nuking the RT_System transactions really helped. My two cents, Ken On Tue, Apr 28, 2009 at 06:29:13PM +0100, Justin Hayes wrote: I'm already skipping a load of transactions, but there's a minimum I can cut down to. In ShowHistory we have this loop, where all the time goes. I've added interval logging round the 3 main sections 1
[rt-users] Slow loading of long tickets in 3.6.3
Hi, Some of our tickets can get pretty long and the longer they get the longer the page takes to load. So for a long ticket with say 100 comments/replies and assorted status changes etc it might take 10+ seconds to render. A lot of the time seems to be looping over every transaction, checking permissions etc etc, I guess maybe to decide whether to display the reply/comment options and other things. Have any improvements been made to the history display code in 3.8? I would have thought it was more sensible to check permissions once for the ticket and then quickly render all the attachments, but I'm sure that's a simplistic view of it. Any thoughts welcome! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow loading of long tickets in 3.6.3
Thanks for the advice Roy, but I think I'd already tried that at some point in the past: tr td colspan=4 class=content %perl $m-print('hidden') if $Collapsed==2;/%perl id=txn-body-%$Transaction-id% %# if ($Transaction-CustomFieldValues-Count) { %# /Elements/ShowCustomFields, Object = $Transaction %# } % $m-comp('ShowTransactionAttachments', %ARGS, Parent = 0) unless ($Collapsed==1 ||!$ShowBody); /td /tr Justin On 28 Apr 2009, at 11:00, Raed El-Hames wrote: Justin; I 've had similar slow loading tickets in the past, and it turned out (in my case) the problem was with transactions custom fields, do you use those?? if you don't you can comment out the following lines from Ticket/Elements/ShowTransaction 65 % if ($Transaction-CustomFieldValues-Count) { 66/Elements/ShowCustomFields, Object = $Transaction 67 % } The numbers you see there are the line numbers Roy Justin Hayes wrote: Hi, Some of our tickets can get pretty long and the longer they get the longer the page takes to load. So for a long ticket with say 100 comments/replies and assorted status changes etc it might take 10+ seconds to render. A lot of the time seems to be looping over every transaction, checking permissions etc etc, I guess maybe to decide whether to display the reply/comment options and other things. Have any improvements been made to the history display code in 3.8? I would have thought it was more sensible to check permissions once for the ticket and then quickly render all the attachments, but I'm sure that's a simplistic view of it. Any thoughts welcome! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Slow loading of long tickets in 3.6.3
I've put some high resolution timestamping into the code and found some interesting things. Am example ticket that took 10 seconds to render spent 8.17 seconds in the while ( my $Transaction = $Transactions-Next ) { loop inside ShowHistory: Apr 28 16:10:05 calypso RT: After Transaction Loop - time taken: 8.172556 (/opt/rt3/local/html/Ticket/Elements/ShowHistory:151) So then I put some logging in ShowTransaction Each INIT takes milliseconds (so I think we can discount that): Apr 28 16:10:05 calypso RT: - ShowTransaction INIT - time taken: 0.006261 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:253) But each Body of ShowTransaction takes between 0.03 and 0.15 seconds eg: Apr 28 16:09:59 calypso RT: After show transaction - time taken: 0.130829 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:09:59 calypso RT: After show transaction - time taken: 0.038523 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:10:00 calypso RT: After show transaction - time taken: 0.033974 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:10:00 calypso RT: After show transaction - time taken: 0.027394 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:10:00 calypso RT: After show transaction - time taken: 0.030395 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:10:00 calypso RT: After show transaction - time taken: 0.005656 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) Apr 28 16:10:00 calypso RT: After show transaction - time taken: 0.030067 (/opt/rt3/local/html/Ticket/Elements/ShowTransaction:102) So multiply that for 100 comments + 100 other transactions (status changes etc) and it's easy to see where the time goes... Justin On 28 Apr 2009, at 16:06, Raed El-Hames wrote: Justin; One of the other things I done here is to hide show summary by default, as I noticed the ShowSummary and in particular /Ticket/Elements/ShowRequestor, Ticket = $Ticket (Thats the box that shows Other ticket by requestor etc) was taking a long time to render If you can play and test in your system try to comment out /Ticket/Elements/ShowSummary, Ticket = $TicketObj, Attachments = $attachments see if ticket loading is quicker, remember to restart apache after you do the change .. I am on 3.6.3 and some of our monitoring tickets are over 300 updates long .. having been through the pain you having, I tried all possible options Roy Justin Hayes wrote: Thanks for the advice Roy, but I think I'd already tried that at some point in the past: tr td colspan=4 class=content %perl $m-print('hidden') if $Collapsed==2;/%perl id=txn-body-%$Transaction-id% %# if ($Transaction-CustomFieldValues-Count) { %# /Elements/ShowCustomFields, Object = $Transaction %# } % $m-comp('ShowTransactionAttachments', %ARGS, Parent = 0) unless ($Collapsed==1 ||!$ShowBody); /td /tr Justin On 28 Apr 2009, at 11:00, Raed El-Hames wrote: Justin; I 've had similar slow loading tickets in the past, and it turned out (in my case) the problem was with transactions custom fields, do you use those?? if you don't you can comment out the following lines from Ticket/Elements/ShowTransaction 65 % if ($Transaction-CustomFieldValues-Count) { 66/Elements/ShowCustomFields, Object = $Transaction 67 % } The numbers you see there are the line numbers Roy Justin Hayes wrote: Hi, Some of our tickets can get pretty long and the longer they get the longer the page takes to load. So for a long ticket with say 100 comments/replies and assorted status changes etc it might take 10+ seconds to render. A lot of the time seems to be looping over every transaction, checking permissions etc etc, I guess maybe to decide whether to display the reply/comment options and other things. Have any improvements been made to the history display code in 3.8? I would have thought it was more sensible to check permissions once for the ticket and then quickly render all the attachments, but I'm sure that's a simplistic view of it. Any thoughts welcome! Justin - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com - Justin Hayes Orbis Support Manager justin.ha...@orbisuk.com
Re: [rt-users] Slow loading of long tickets in 3.6.3
I'm already skipping a load of transactions, but there's a minimum I can cut down to. In ShowHistory we have this loop, where all the time goes. I've added interval logging round the 3 main sections 1) The skip checking 2) The attachment grepping 3) The call to ShowTransaction while ( my $Transaction = $Transactions-Next ) { my $t10 = [gettimeofday]; my $skip = 0; $m-comp( '/Elements/Callback', _CallbackName = 'SkipTransaction', Transaction = $Transaction, skip = \$skip, %ARGS ); next if $skip; $i++; my $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop1 - time taken: $elapsed10); my @trans_attachments = grep { $_-TransactionId == $Transaction- Id } @attachments; my $trans_content = {}; grep { ($_-TransactionId == $Transaction-Id ) ($trans_content-{$_-Id} = $_) } @attachment_content; $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop2 - time taken: $elapsed10); #Args is first because we're clobbering the Attachments parameter $m-comp( 'ShowTransaction', %ARGS, AttachPath = $AttachPath, UpdatePath = $UpdatePath, Ticket = $Ticket, Transaction = $Transaction, ShowHeaders = $ShowHeaders, Collapsed= $Collapsed, RowNum = $i, EntryNum = (( $Transaction-Type =~ /^(Create|Correspond|Comment)$/ ) ? (++ $EntryNum) : ), ShowTitleBarCommands = $ShowTitleBarCommands, Attachments = \...@trans_attachments, AttachmentContent= $trans_content, LastTransaction = $Transactions-IsLast ); # manually flush the content buffer after each txn, so the user sees # some update $m-flush_buffer(); $elapsed10 = tv_interval ( $t10 ); $RT::Logger-debug(Transaction Loop3 - time taken: $elapsed10); } We get output like this (time taken to get from the start of the iteration to the logging point) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004148 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.010705 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.048624 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:133) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004812 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.011379 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.024753 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:133) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004143 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.010706 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.024104 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:133) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.004148 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.01071 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.049055 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:133) Apr 28 18:17:14 calypso RT: Transaction Loop1 - time taken: 0.00481 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:102) Apr 28 18:17:14 calypso RT: Transaction Loop2 - time taken: 0.011399 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:110) Apr 28 18:17:14 calypso RT: Transaction Loop3 - time taken: 0.024773 (/ opt/rt3/local/html/Ticket/Elements/ShowHistory:133) So very little time lost checking for skip, then maybe 0.01s to get past the attachment grep, and then another 0.01 - 0.04s to show the transaction itself. So times that by 300 transactions and you're up to 10 secs easily. Those grep commands seem to cost 0.01 (which is 3s over 300 transactions) and I'm not sure what they do.. Then you're into the ShowTransaction itself, where the bulk of the time goes, so maybe I need to look there next. Justin On 28 Apr 2009, at 18:12, Kenneth Marshall wrote: Hi, Justin, We found similar results from a slow ticket load investigation. For a fix, we added a brief mode and a full mode to the display code. The full mode show it all, but the brief mode skipped uninteresting transactions. Maybe an approach like that would help. Cheers, Ken On Tue, Apr 28, 2009 at 04:19:21PM +0100, Justin Hayes wrote: I've put some high resolution timestamping
Re: [rt-users] Search for logged-in user as requestor
Thanks for the suggestion Gabriele but unfortunately that doesn't seem to work for me :( Oh I'm using 3.6.3 btw. Justin On 7 Jan 2009, at 15:00, Franzini, Gabriele [Nervianoms] wrote: Hello,Justin, I use in similar saved searches for V3.8.1 the clause: Owner.Id = '__CurrentUser__' so maybe Requestor.Id would do? HTH, Gabriele Franzini ICT Applications Manager Nerviano Medical Sciences SRL 20014 Nerviano Italy - Message: 1 Date: Wed, 7 Jan 2009 09:43:12 + From: Justin Hayes justin.ha...@orbisuk.com Subject: [rt-users] Search for logged-in user as requestor Hi, I would like to be able to create saved searches that will find tickets where the logged in user is the Requestor. This is easy for Owner as you can useOwner = '__CurrentUser__'. Is there a way to do this for Requestor though? I tried Requestor = '__CurrentUser__'but that doesn't work. I found this on the wiki, but that only seems to apply to RT2: http://wiki.bestpractical.com/view/CurrentUserEmail Any help much appreciated!! Justin ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Search for logged-in user as requestor
In case anyone's interested I figured out how to do this in share/html/Elements/ShowSearch you have: $SearchArg-{'Query'} =~ s/__CurrentUser__/$session{'CurrentUser'}- Id/ge; I've added a line for CurrentUserEmail so now I have: $SearchArg-{'Query'} =~ s/__CurrentUser__/$session{'CurrentUser'}- Id/ge; $SearchArg-{'Query'} =~ s/__CurrentUserEmail__/ $session{'CurrentUser'}-EmailAddress/ge; You can then use the following in searches that you add to the At A Glance screen: Requestor.EmailAddress = '__CurrentUserEmail__' Cheers, Justin On 7 Jan 2009, at 09:43, Justin Hayes wrote: Hi, I would like to be able to create saved searches that will find tickets where the logged in user is the Requestor. This is easy for Owner as you can use Owner = '__CurrentUser__'. Is there a way to do this for Requestor though? I tried Requestor = '__CurrentUser__' but that doesn't work. I found this on the wiki, but that only seems to apply to RT2: http://wiki.bestpractical.com/view/CurrentUserEmail Any help much appreciated!! Justin ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Search for logged-in user as requestor
Hi, I would like to be able to create saved searches that will find tickets where the logged in user is the Requestor. This is easy for Owner as you can use Owner = '__CurrentUser__'. Is there a way to do this for Requestor though? I tried Requestor = '__CurrentUser__' but that doesn't work. I found this on the wiki, but that only seems to apply to RT2: http://wiki.bestpractical.com/view/CurrentUserEmail Any help much appreciated!! Justin___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: sa...@bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Locking accounts on password failure
Has anyone implemented locking accounts after so many login attempts? I'm sure I can add it myself but if there's a solution out there already. I'm using 3.6.3 Cheers, Justin ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
[rt-users] Resolve Ticket Option
I'd like to offer my customers an option to resolve a ticket themselves, however I don't want them to have the Modify Ticket permission (which I think is the one you need for changing status). Does anyone know of a way round this? I'm running 3.6.3 Cheers, Justin -- Justin Hayes Support Manager [EMAIL PROTECTED] ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Resolve Ticket Option
Hi Ben, Thanks for that suggestion - had forgotten you can scrip based on the change to a custom field. However can you grant permissions to specific custom fields? I have others and don't want the customer to be able to change those. I had been thinking somewhere along the lines of altering the SetStatus fucntion in Ticket_Overlay.pm It currently does this: #Check ACL if ( $args{Status} eq 'deleted') { unless ($self-CurrentUserHasRight('DeleteTicket')) { return ( 0, $self-loc('Permission Denied') ); } } else { unless ($self-CurrentUserHasRight('ModifyTicket')) { return ( 0, $self-loc('Permission Denied') ); } } I was thinking about trying: #Check ACL if ( $args{Status} eq 'deleted') { unless ($self-CurrentUserHasRight('DeleteTicket')) { return ( 0, $self-loc('Permission Denied') ); } elsif ( $args{Status} eq 'resolved') { # do nothing as we don't mind people resolving } else { unless ($self-CurrentUserHasRight('ModifyTicket')) { return ( 0, $self-loc('Permission Denied') ); } } I figured that would be ok as you'd have to have the permission to see the ticket in the first place. Your method might be safer though, as you can permission it. Justin -- Justin Hayes Support Manager [EMAIL PROTECTED] On 25 Jun 2008, at 09:45, Benjamin Weser wrote: Hi Justin, sure. Establish a custom field with the right ModifyCustomField for the user. Use the CF to trigger a scrip which sets the ticket to resolved. Ben Justin Hayes schrieb: I'd like to offer my customers an option to resolve a ticket themselves, however I don't want them to have the Modify Ticket permission (which I think is the one you need for changing status). Does anyone know of a way round this? I'm running 3.6.3 Cheers, Justin -- Justin Hayes Support Manager [EMAIL PROTECTED] ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com
Re: [rt-users] Resolve Ticket Option
Hmm another interesting suggestion, thanks! :) But aren't you checking that it's the owner that's doing the resolving? if ( $Ticket-OwnerObj-Name eq $session{'CurrentUser'}-Name ) I'm after letting the customer resolve, and they won't be the owner. Also you're calling the SetStatus function. Doesn't that check that you have ModifyTicket? } else { unless ($self-CurrentUserHasRight('ModifyTicket')) { return ( 0, $self-loc('Permission Denied') ); } } Or does that code in %INIT run as if you're the main RT user or something, thus you have permission if you run SetStatus in %INIT? Justin -- Justin Hayes Support Manager [EMAIL PROTECTED] On 25 Jun 2008, at 10:08, Brian Gallew wrote: Justin Hayes wrote: I'd like to offer my customers an option to resolve a ticket themselves, however I don't want them to have the Modify Ticket permission (which I think is the one you need for changing status). Does anyone know of a way round this? I'm running 3.6.3 Sure. Create $RTHOME/local/html/Callbacks/orbisuk/SelfService/Elements/Tabs/ Default that looks something like this: ==CUT HERE== %ARGS $actions $Ticket /%ARGS %INIT if ($Ticket) { my $id = $Ticket-id(); if ( $Ticket-OwnerObj-Name eq $session{'CurrentUser'}-Name ) { $actions-{'D'} = { path = SelfService/Resolve.html?id= . $id, title = loc('Owner Resolve') }; }; } /%INIT %# Local Variables: %# mode:perl %# End: ==CUT HERE== Then you'll need to create $RTHOME/local/html/SelfService/Resolve.html ==CUT HERE== %ARGS $Ticket /%ARGS %INIT if ($Ticket) { my $id = $Ticket-id(); if ( $Ticket-OwnerObj-Name eq $session{'CurrentUser'}-Name ) { $Ticket-SetStatus('resolved'); }; } RT::Interface::Web::Redirect($RT::WebURL.SelfService/); /%INIT %# Local Variables: %# mode:perl %# End: ==CUT HERE== Caveat emptor: I've modeled the code after something I have running which is similary, but I've never actually run it, so you may have to experiment. Especially since I don't actually do Perl. 8) Also, notice that this marks the ticket resolved without actually allowing the opportunity to add any commentary. ___ http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users Community help: http://wiki.bestpractical.com Commercial support: [EMAIL PROTECTED] Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com