[rt-users] performance slow
Hi, We are using RT3.6.3 as one of our internal project for request tracking. However we are facing many many many performance issues. We have our Rt database filled like any thing with live request. So the result is our ticket page opens very slow. Sometimes it takes about 3-4 min in rendering that ticket page. So is there anything from database side or apache configuration side that can help us to improve our rt system performance. Please try to solve our issue asap as we are in a big mess due to performance. Regards, Jyoti Sahu RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
[rt-users] New between Open
Hello, can someone explain the different between the Ticket State new and open ? Thank you Greetings from Germany Ingo von Itter ___ Geschäftsführer: Rolf Herzog, Thomas Benk AGES Maut System GmbH Co. KG AG Düsseldorf HRA 14045, USt-IdNr.: DE 202525868 AGES International GmbH Co. KG AG Düsseldorf HRA 16636, USt-IdNr.: DE 813749831 AGES ETS GmbH AG Düsseldorf HRB 55580, USt-IdNr.: DE 814789134 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] performance slow
On Tue, Sep 07, 2010 at 11:37:25AM +0530, Jyoti Sahu wrote: Hi, Hi, We are using RT3.6.3 as one of our internal project for request tracking. Upgrading this to latest version will give you better performances. Especially if you have many queues. FYI latest RT is 3.8.8. However we are facing many many many performance issues. We have our Rt database filled like any thing with live request. So the result is our ticket page opens very slow. Sometimes it takes about 3-4 min in rendering that ticket page. So is there anything from database side or apache configuration side that can help us to improve our rt system performance. Most RT performance problems commes from poorly tuned database. You should enable slow query logging on your DB and check indexes, you should also make sur that your DB is configured to use your server memory plenty. Please try to solve our issue asap as we are in a big mess due to performance. you're asking here for community support so don't espect any SLA ;) RT users will do their best in helping you if you give as much details as possible. Else, you may also try to find some commercial support. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Jeffrey. We're already using UseSQLForACLChecks and don't use web external auth. Tried the combination you've given though and unfortunately no difference :( - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 00:44, Jeffrey Fearn wrote: Hi Justin, I've recently been using siege to bash on RT, and have been testing the following two settings in our RT_SiteConfig.pm Set($UseSQLForACLChecks, 1); Set($WebExternalAuthContinuous, 0); The combined effect has been a serious reduction in rendering speed in general, and particularly so for long tickets. Cheers, Jeff. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it first.
I'm pretty much positive it's not a DB issue, as I've tried different sizes
of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried
the built in webserver with RT and no change there either.
Currently trying to install RT on Centos given that Roy (who has kindly been
helping me with details of his own setup) appears to have none of the same
problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've
currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part of
RT 3.10. Code at the moment is unstable, but eventually it wil be faster
then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com
wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on these
2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1 Jul 2010, at 11:51, Raed El-Hames wrote:
Justin,
Do you use Transaction custom fields, if you do n’t ; try and comment out
lines 70,71,72 from html/Ticket/Elements/ShowTransaction
% if ( $Transaction-CustomFieldValues-Count ) {
/Elements/ShowCustomFields, Object = $Transaction
% }
See if that improves things for you.
Some of our monitoring tickets can have up to 500 updates, such tickets use
to take up to 20s to load, once I commented out the above lines, load time
is now down to less than 5 seconds.
Regards;
Roy
From: rt-users-boun...@lists.bestpractical.com
[mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes
Sent: 01 July 2010 11:39
To: Kenneth Crocker
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] Slow Ticket History 3.8.8
We do Kenneth, but most tickets don't have many file attachments, so I
assume that's not an issue?
Cheers,
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 29 Jun 2010, at 17:54, Kenneth Crocker wrote:
Justin,
I didn't see this mentioned and may have missed it, but are you displaying
attachements inline? That might cut back on the I/O for History. Just a
thought.
Kenn
LBNL
On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com
wrote:
As a test we've just created a long ticket in an empty RT DB and it's very
fast. So does look to be DB related - contrary to our earlier
investigations.
I guess it must still access the DB resultset during the ticket rendering
(which isn't how we thought it would work).
Time to tune the hell out of mysql then...
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 29 Jun 2010, at 15:53, Justin Hayes wrote:
Seem to be quite a few things to look at Jason. Need to figure out what
they all mean first.
Justin
General Statistics
--
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.37-1ubuntu5.4-log
[OK] Operating on 64-bit architecture
Storage Engine
Re: [rt-users] Slow Ticket History 3.8.8
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should drop
our times. Guess we just wanted to explore all avenues before throwing hardware
at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried different
sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've
tried the built in webserver with RT and no change there either.
Currently trying to install RT on Centos given that Roy (who has kindly been
helping me with details of his own setup) appears to have none of the same
problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've
currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part of
RT 3.10. Code at the moment is unstable, but eventually it wil be faster
then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com
wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1 Jul 2010, at 11:51, Raed El-Hames wrote:
Justin,
Do you use Transaction custom fields, if you do n’t ; try and comment out
lines 70,71,72 from html/Ticket/Elements/ShowTransaction
% if ( $Transaction-CustomFieldValues-Count ) {
/Elements/ShowCustomFields, Object = $Transaction
% }
See if that improves things for you.
Some of our monitoring tickets can have up to 500 updates, such tickets
use to take up to 20s to load, once I commented out the above lines, load
time is now down to less than 5 seconds.
Regards;
Roy
From: rt-users-boun...@lists.bestpractical.com
[mailto:rt-users-boun...@lists.bestpractical.com] On Behalf Of Justin Hayes
Sent: 01 July 2010 11:39
To: Kenneth Crocker
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] Slow Ticket History 3.8.8
We do Kenneth, but most tickets don't have many file attachments, so I
assume that's not an issue?
Cheers,
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 29 Jun 2010, at 17:54, Kenneth Crocker wrote:
Justin,
I didn't see this mentioned and may have missed it, but are you displaying
attachements inline? That might cut back on the I/O for History. Just a
thought.
Kenn
LBNL
On Tue, Jun 29, 2010 at 8:04 AM, Justin Hayes justin.ha...@openbet.com
wrote:
As a test we've just created a long ticket in an empty RT DB and it's very
fast. So does look to be DB related - contrary to our earlier
investigations.
I guess it must still access the DB resultset during the ticket rendering
(which isn't how we thought it would work).
Time to tune the hell out of mysql then...
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On
Re: [rt-users] 10 newest unowned tickets
Peter Murfitt wrote / napísal(a): I think this is very similar to what Justin was saying a couple of weeks ago: http://lists.bestpractical.com/pipermail/rt-users/2010-July/065538.html Though it looked like that bug was fixed by 3.8.8 I have same problem, and I am on 3.8.8. Any idea how it solve? On 21 Jul 2010, at 19:04, James Berwick wrote: On 7/21/2010 1:46 PM, Kenneth Crocker wrote: Jeff, Not if I'm not supposed to. So what if there aren't 10 unowned tickets in the list? As a matter of security, I sure wouldn't want someone looking at sensitive payroll info on requests in the Payroll Queue if they aren't even allowed to have access to the payroll Queue. Kenn LBNL If I understand correctly: There are 100 open tickets. The 10 newest unowned tickets belong to a queue that a user can't see. The user logs in and the 10 newest unowned tickets box is empty. There are 90 other tickets that are new and unowned that the user should be allowed to see but are not displayed. What I believe Mark and Jeff are saying (and that UseSQLForACLChecks does) is have the box be filled with the 10 newest unowned tickets that a user is allowed to see, ie, filter the tickets during the search as opposed to finding the top 10 and then filtering out what shouldn't be visible. Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com Discover RT's hidden secrets with RT Essentials from O'Reilly Media. Buy a copy at http://rtbook.bestpractical.com RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
Hi Justin,
just found this threat, sounds interessting.
What i read so far: You have 1 quad core system with 8GB RAM, running both
WEB and DB, correct?
Think you should follow Raed's hints first to log the queries generated with
RT
In terms of debug; if you have not done this yet enable DBIx-SearchBuilder
StatementLog
Set($StatementLog,’debug’); in your etc/RT_SiteConfig.
I'm sure you will find some funny queries. Normally the Query Log of default
MySQL can only log queries taking longer than a second, but in your case i
think, you will have several much faster queries but in summary they take
longer - but you can't find in mysql-slow log.
Some more question regarding your hardware and setup.
1. One Server / quad core (hyper threating) - how many threats for
Mysql/Postgresql? / 8 GB Ram
2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O
Problems?)
3. RT Rights Setup, does the user performance is faster or slower than the
performance with root user?
Some more information?
We're running also a larger RT Instance with dedicated hardware for DB and
Webservers with no huge perferomance bottlenacks.
Tob
2010/9/7 Justin Hayes justin.ha...@openbet.com
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should
drop our times. Guess we just wanted to explore all avenues before throwing
hardware at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried different
sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've
tried the built in webserver with RT and no change there either.
Currently trying to install RT on Centos given that Roy (who has kindly
been helping me with details of his own setup) appears to have none of the
same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu
we've currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part of
RT 3.10. Code at the moment is unstable, but eventually it wil be faster
then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.comwrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1 Jul 2010, at 11:51, Raed El-Hames wrote:
Justin,
Do you use Transaction custom fields, if you do n’t ; try and comment out
lines 70,71,72 from html/Ticket/Elements/ShowTransaction
% if ( $Transaction-CustomFieldValues-Count ) {
/Elements/ShowCustomFields, Object = $Transaction
% }
See if that improves things for you.
Some of our monitoring tickets can have up to 500 updates, such tickets
use to take up to 20s to load, once I commented out the above lines, load
time is now down to less than 5 seconds.
Regards;
Roy
*From:* rt-users-boun...@lists.bestpractical.com [mailto:
rt-users-boun...@lists.bestpractical.com] *On Behalf Of *Justin Hayes
*Sent:* 01 July 2010 11:39
Re: [rt-users] Slow Ticket History 3.8.8
Thanks Torsten. Will have a look at some of those suggestions. We'd ruled the
DB out as we couldn't find an slow ones in the logs, and also because we tried
a totally fresh DB with only 1 ticket in it, and it was still slow. But it
might just be lots of tiny queries so we'll have another look at that.
We've also tried running the web services on a separate server to the DB. This
also made no difference.
Cheers,
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 11:13, Torsten Brumm wrote:
Hi Justin,
just found this threat, sounds interessting.
What i read so far: You have 1 quad core system with 8GB RAM, running both
WEB and DB, correct?
Think you should follow Raed's hints first to log the queries generated with
RT
In terms of debug; if you have not done this yet enable DBIx-SearchBuilder
StatementLog
Set($StatementLog,’debug’); in your etc/RT_SiteConfig.
I'm sure you will find some funny queries. Normally the Query Log of default
MySQL can only log queries taking longer than a second, but in your case i
think, you will have several much faster queries but in summary they take
longer - but you can't find in mysql-slow log.
Some more question regarding your hardware and setup.
1. One Server / quad core (hyper threating) - how many threats for
Mysql/Postgresql? / 8 GB Ram
2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O
Problems?)
3. RT Rights Setup, does the user performance is faster or slower than the
performance with root user?
Some more information?
We're running also a larger RT Instance with dedicated hardware for DB and
Webservers with no huge perferomance bottlenacks.
Tob
2010/9/7 Justin Hayes justin.ha...@openbet.com
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should
drop our times. Guess we just wanted to explore all avenues before throwing
hardware at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried
different sizes of DB, tried postgres AND mysql etc. I don't think it's
apache as I've tried the built in webserver with RT and no change there
either.
Currently trying to install RT on Centos given that Roy (who has kindly
been helping me with details of his own setup) appears to have none of the
same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu
we've currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part
of RT 3.10. Code at the moment is unstable, but eventually it wil be
faster then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com
wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1 Jul 2010, at
Re: [rt-users] Slow Ticket History 3.8.8
Are you running mod_perl2, FastCGI or FCGID?
2010/9/7 Justin Hayes justin.ha...@openbet.com
Thanks Torsten. Will have a look at some of those suggestions. We'd ruled
the DB out as we couldn't find an slow ones in the logs, and also because we
tried a totally fresh DB with only 1 ticket in it, and it was still slow.
But it might just be lots of tiny queries so we'll have another look at
that.
We've also tried running the web services on a separate server to the DB.
This also made no difference.
Cheers,
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 11:13, Torsten Brumm wrote:
Hi Justin,
just found this threat, sounds interessting.
What i read so far: You have 1 quad core system with 8GB RAM, running both
WEB and DB, correct?
Think you should follow Raed's hints first to log the queries generated
with RT
In terms of debug; if you have not done this yet enable DBIx-SearchBuilder
StatementLog
Set($StatementLog,’debug’); in your etc/RT_SiteConfig.
I'm sure you will find some funny queries. Normally the Query Log of
default MySQL can only log queries taking longer than a second, but in your
case i think, you will have several much faster queries but in summary they
take longer - but you can't find in mysql-slow log.
Some more question regarding your hardware and setup.
1. One Server / quad core (hyper threating) - how many threats for
Mysql/Postgresql? / 8 GB Ram
2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O
Problems?)
3. RT Rights Setup, does the user performance is faster or slower than the
performance with root user?
Some more information?
We're running also a larger RT Instance with dedicated hardware for DB and
Webservers with no huge perferomance bottlenacks.
Tob
2010/9/7 Justin Hayes justin.ha...@openbet.com
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should
drop our times. Guess we just wanted to explore all avenues before throwing
hardware at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried different
sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've
tried the built in webserver with RT and no change there either.
Currently trying to install RT on Centos given that Roy (who has kindly
been helping me with details of his own setup) appears to have none of the
same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu
we've currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part of
RT 3.10. Code at the moment is unstable, but eventually it wil be faster
then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.comwrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with
this graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
Re: [rt-users] Slow Ticket History 3.8.8
Well we've captured the time for all the queries run for our long ticket (which
takes ~20secs to generate).
Total query time is 0.871493s
So it's not the DB.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 11:13, Torsten Brumm wrote:
Hi Justin,
just found this threat, sounds interessting.
What i read so far: You have 1 quad core system with 8GB RAM, running both
WEB and DB, correct?
Think you should follow Raed's hints first to log the queries generated with
RT
In terms of debug; if you have not done this yet enable DBIx-SearchBuilder
StatementLog
Set($StatementLog,’debug’); in your etc/RT_SiteConfig.
I'm sure you will find some funny queries. Normally the Query Log of default
MySQL can only log queries taking longer than a second, but in your case i
think, you will have several much faster queries but in summary they take
longer - but you can't find in mysql-slow log.
Some more question regarding your hardware and setup.
1. One Server / quad core (hyper threating) - how many threats for
Mysql/Postgresql? / 8 GB Ram
2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O
Problems?)
3. RT Rights Setup, does the user performance is faster or slower than the
performance with root user?
Some more information?
We're running also a larger RT Instance with dedicated hardware for DB and
Webservers with no huge perferomance bottlenacks.
Tob
2010/9/7 Justin Hayes justin.ha...@openbet.com
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should
drop our times. Guess we just wanted to explore all avenues before throwing
hardware at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried
different sizes of DB, tried postgres AND mysql etc. I don't think it's
apache as I've tried the built in webserver with RT and no change there
either.
Currently trying to install RT on Centos given that Roy (who has kindly
been helping me with details of his own setup) appears to have none of the
same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu
we've currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part
of RT 3.10. Code at the moment is unstable, but eventually it wil be
faster then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.com
wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1 Jul 2010, at 11:51, Raed El-Hames wrote:
Justin,
Do you use Transaction custom fields, if you do n’t ; try and comment out
lines 70,71,72 from html/Ticket/Elements/ShowTransaction
% if ( $Transaction-CustomFieldValues-Count ) {
/Elements/ShowCustomFields,
Re: [rt-users] Slow Ticket History 3.8.8
OK, now we know, it is not the DB, have you used firebug to check which part
of the page takes long?
There is a nice addon from BPS to track the page memory used for each
request inside github, possible a good point to look next.
The time you messured are taken with firefox, Internet explorer?
2010/9/7 Justin Hayes justin.ha...@openbet.com
Well we've captured the time for all the queries run for our long ticket
(which takes ~20secs to generate).
Total query time is 0.871493s
So it's not the DB.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 11:13, Torsten Brumm wrote:
Hi Justin,
just found this threat, sounds interessting.
What i read so far: You have 1 quad core system with 8GB RAM, running both
WEB and DB, correct?
Think you should follow Raed's hints first to log the queries generated
with RT
In terms of debug; if you have not done this yet enable DBIx-SearchBuilder
StatementLog
Set($StatementLog,’debug’); in your etc/RT_SiteConfig.
I'm sure you will find some funny queries. Normally the Query Log of
default MySQL can only log queries taking longer than a second, but in your
case i think, you will have several much faster queries but in summary they
take longer - but you can't find in mysql-slow log.
Some more question regarding your hardware and setup.
1. One Server / quad core (hyper threating) - how many threats for
Mysql/Postgresql? / 8 GB Ram
2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O
Problems?)
3. RT Rights Setup, does the user performance is faster or slower than the
performance with root user?
Some more information?
We're running also a larger RT Instance with dedicated hardware for DB and
Webservers with no huge perferomance bottlenacks.
Tob
2010/9/7 Justin Hayes justin.ha...@openbet.com
I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit
faster than ours. We're going to try it on a faster server and that should
drop our times. Guess we just wanted to explore all avenues before throwing
hardware at the problem.
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 7 Sep 2010, at 10:30, Justin Hayes wrote:
Tried Centos last night, and no difference at all.
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 20:49, Justin Hayes wrote:
Hi Ruslan,
Sorry looks like I shrunk the image too much. The thing I find odd is that
there are others with similar hardware who don't get the problem. It'll be
great if 3.10 fixes it for me, but I'd love to get to the bottom of it
first. I'm pretty much positive it's not a DB issue, as I've tried different
sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've
tried the built in webserver with RT and no change there either.
Currently trying to install RT on Centos given that Roy (who has kindly
been helping me with details of his own setup) appears to have none of the
same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu
we've currently got live.
No idea if it's going to have any effect though :(
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote:
Justin.
First of all, I can not read from the chart, but anyway history rendering
has been worked on in a new code branch. Probably this code will be part of
RT 3.10. Code at the moment is unstable, but eventually it wil be faster
then the current version.
On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.comwrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with
this graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on
these 2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
Both are greps. Does this imply that perl itself is just slow?
IF so why would our perl be slow compared to other people's? We've tried
compiling it from source and that made no difference.
ATM we're at a bit of a loss
Justin
-
Justin Hayes
OpenBet Support Manager
justin.ha...@openbet.com
On 1
Re: [rt-users] Slow Ticket History
On 6 Sep 2010, at 5:54 pm, rt-users-requ...@lists.bestpractical.com wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on these
2 lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
From what I can tell, the real problem here is repeated scanning of both the
@attachments and @attachment_content arrays, which makes the execution speed
of the ShowHistory element O(N^2) with respect to the number of transactions;
painful, to say the least.
1) The %$trans_content hash can be made up front in a single pass, for all the
attachments in the ticket, turning this into an O(N) operation, rather than
O(N^2)
2) The $trans_content variable is only used in one place; it's passed to
ShowTransaction, where it is then passed on to ShowTransactionAttachments.
3) In there, there are some errors which cause some autovivification of hash
members which needn't happen.
4) You can do much the same up-front calculation with $trans_attachments as
well, so you don't have to keep grepping through it
I've now made those changes, and on a reasonably large ticket (216
transactions) it reduced the ticket rendering time on my system from 2.5
minutes to 34 seconds, which is a pretty good improvement, I think.
On a more extreme ticket, with 417 transactions, the 3.8.8 release code takes
over 20 minutes to render the ticket (I gave up waiting), so in fact it's
considerably worse order execution time than I thought. My patched code takes
2.2 minutes - still not brilliant but hey, this ticket is now renderable, which
it was not before.
Just for some background, the hardware I'm running on:
RT database: 2 CPU virtual machine, 8GB RAM, MySQL, running on vSphere 4.1 on
a 2.0 GHz E5504 Nehalem system
RT web server: 2 CPU virtual machine, 2GB RAM, on the same type of physical
hardware as the database server
As far as I can tell, I have not semantically altered the code, but others may
want to test more thoroughly. I have not yet put this on my production web
server - I cloned my web server VM and made my changes to the clone (God, I
love VMs for this sort of thing!)
Regards,
Tim
Here are my patches to Ticket/Elements/ShowHistory:
--- /opt/rt3/share/html/Ticket/Elements/ShowHistory 2010-05-14
13:58:15.0 +0100
+++ /opt/rt3/local/html/Ticket/Elements/ShowHistory 2010-09-07
11:59:30.0 +0100
@@ -84,6 +84,10 @@
%perl
my @attachments = @{$Attachments-ItemsArrayRef()};
my @attachment_content = @{$AttachmentContent-ItemsArrayRef()};
+my $trans_content = {};
+map { $trans_content-{$_-TransactionId}-{$_-Id} = $_ }
@attachment_content;
+my $trans_attachments = {};
+map { push (@{$trans_attachments-{$_-TransactionId}}, $_) } @attachments;
while ( my $Transaction = $Transactions-Next ) {
my $skip = 0;
@@ -97,12 +101,6 @@
$i++;
-my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
-
-my $trans_content = {};
-grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
-
-
my $IsLastTransaction = 0;
if ( $OldestFirst ) {
$IsLastTransaction = $Transactions-IsLast;
@@ -118,7 +116,7 @@
Transaction = $Transaction,
ShowHeaders = $ShowHeaders,
RowNum = $i,
- Attachments = \...@trans_attachments,
+ Attachments = $trans_attachments-{$Transaction-id},
AttachmentContent= $trans_content,
LastTransaction = $IsLastTransaction
);
and here's the patch to ShowTransactionAttachments (both files need to be
patched):
--- /opt/rt3/share/html/Ticket/Elements/ShowTransactionAttachments
2010-05-14 13:58:15.0 +0100
+++ /opt/rt3/local/html/Ticket/Elements/ShowTransactionAttachments
2010-09-07 11:04:53.0 +0100
@@ -189,8 +189,10 @@
{
my $content;
-if ( $AttachmentContent-{ $message-id } ) {
-$content = $AttachmentContent-{ $message-id }-Content;
+ my ($transaction_id, $message_id) = ($Transaction-id,
$message-id);
+if ( exists($AttachmentContent-{ $transaction_id })
+exists($AttachmentContent-{ $transaction_id
[rt-users] designing not proper
Hi, We have just installed 3.8.8 version of RT. However we are facing some problem related to designing. The page looks proper but without proper css Can anybody suggest how this can be solved? Is some perl module is required to be installed? Regards, Jyoti Sahu RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History 3.8.8
Hi Justin, just created inside a RT Test VM (slow one with 500mb ram) a single ticket with around 60 replies and some comments. Tested the speed with different users 1. root user to open this ticket: around 26 sec - 870 single sql queries in around 4 sec! (Queries: http://pastebin.com/7Yekfx2Y) 2. user with full access (take, own, modify etc): around same time and queries like root (Queries: http://pastebin.com/U0HnPcJL) 3. user with less rights (no take, no own, only showticket, seequeue): time around 15 sec and 600 sql queries in around 2 sec! (Queries: http://pastebin.com/fXDHu6im) After this the apache starts to render the page from the results and push them to the browser. The page is for my few comments/replies already 206KB without any apache optimizations After adding: SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary ExpiresActive On ExpiresByType text/css A604800 ExpiresByType image/x-icon A31536000 ExpiresByType image/gif A604800 ExpiresByType image/jpg A604800 ExpiresByType image/jpeg A604800 ExpiresByType image/png A604800 ExpiresByType application/x-javascript A3600 Header set Cache-Control must-revalidate to the rt vhost, the page load time goes down from 26 sec to 8 sec and from 206 kb to 10kb you should try. Torsten 2010/9/7 Justin Hayes justin.ha...@openbet.com Well we've captured the time for all the queries run for our long ticket (which takes ~20secs to generate). Total query time is 0.871493s So it's not the DB. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 11:13, Torsten Brumm wrote: Hi Justin, just found this threat, sounds interessting. What i read so far: You have 1 quad core system with 8GB RAM, running both WEB and DB, correct? Think you should follow Raed's hints first to log the queries generated with RT In terms of debug; if you have not done this yet enable DBIx-SearchBuilder StatementLog Set($StatementLog,’debug’); in your etc/RT_SiteConfig. I'm sure you will find some funny queries. Normally the Query Log of default MySQL can only log queries taking longer than a second, but in your case i think, you will have several much faster queries but in summary they take longer - but you can't find in mysql-slow log. Some more question regarding your hardware and setup. 1. One Server / quad core (hyper threating) - how many threats for Mysql/Postgresql? / 8 GB Ram 2. Hard Disk Setup? (logfiles and db storred on different HDD's? Any I/O Problems?) 3. RT Rights Setup, does the user performance is faster or slower than the performance with root user? Some more information? We're running also a larger RT Instance with dedicated hardware for DB and Webservers with no huge perferomance bottlenacks. Tob 2010/9/7 Justin Hayes justin.ha...@openbet.com I *think* we're just CPU bound. Roy's webservers are 3.6ghz so quite a bit faster than ours. We're going to try it on a faster server and that should drop our times. Guess we just wanted to explore all avenues before throwing hardware at the problem. Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 7 Sep 2010, at 10:30, Justin Hayes wrote: Tried Centos last night, and no difference at all. - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 20:49, Justin Hayes wrote: Hi Ruslan, Sorry looks like I shrunk the image too much. The thing I find odd is that there are others with similar hardware who don't get the problem. It'll be great if 3.10 fixes it for me, but I'd love to get to the bottom of it first. I'm pretty much positive it's not a DB issue, as I've tried different sizes of DB, tried postgres AND mysql etc. I don't think it's apache as I've tried the built in webserver with RT and no change there either. Currently trying to install RT on Centos given that Roy (who has kindly been helping me with details of his own setup) appears to have none of the same problems on that OS. Perhaps perl is just slow on the 64bit ubuntu we've currently got live. No idea if it's going to have any effect though :( Justin - Justin Hayes OpenBet Support Manager justin.ha...@openbet.com On 6 Sep 2010, at 18:37, Ruslan Zakirov wrote: Justin. First of all, I can not read from the chart, but anyway history rendering has been worked on in a new code branch. Probably this code will be part of RT 3.10. Code at the moment is unstable, but eventually it wil be faster then the current version. On Mon, Sep 6, 2010 at 8:56 PM, Justin Hayes justin.ha...@openbet.comwrote: So far we've
Re: [rt-users] designing not proper
On Tue, Sep 07, 2010 at 05:02:49PM +0530, Jyoti Sahu wrote: Hi, We have just installed 3.8.8 version of RT. However we are facing some problem related to designing. The page looks proper but without proper css Can anybody suggest how this can be solved? Is some perl module is required to be installed? you can check that you meet requirement by running make testdeps in your RT source. but I suspect some kind of configuration error in webpath or in your vhost conf. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Report on time worked per user
Yes, the TimeWorkedReport does have the option to show time worked per user per ticket. I can't remember if I coded it to be the default presentation or not, but it's documented on that wiki page IIRC. -Fran -Original Message- From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users- boun...@lists.bestpractical.com] On Behalf Of Johan Sjöberg Sent: Thursday, September 02, 2010 3:57 PM To: rt-users@lists.bestpractical.com Subject: Re: [rt-users] Report on time worked per user I think that this is what you want. http://wiki.bestpractical.com/view/TimeWorkedReport /Johan -Original Message- From: rt-users-boun...@lists.bestpractical.com [mailto:rt-users- boun...@lists.bestpractical.com] On Behalf Of Ian Rowland Sent: den 2 september 2010 22:45 To: rt-users@lists.bestpractical.com Subject: [rt-users] Report on time worked per user Hi folks I'd like to pull a report on time worked per user on a ticket, at present I am pulling a report on resolved, then go into each ticket to see how much time was spent per user on a ticket. Is there a way to report on time spent per user on a ticket? Example: Ticket ID Subject Total Time SpentTime User A Time User B etc 2223 Ian cannot print 30 15 15 etc RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Copy Queue Permissions
Not that I'm aware of but I did write an RT extension that makes it easy to compare queue rights side-by-side and assign them to specific users and groups. http://search.cpan.org/~htchapman/RTx-RightsMatrix-0.03.00/lib/RTx/RightsMatrix.pm On Sun, Sep 5, 2010 at 12:28 PM, John Alberts john.albe...@exlibrisgroup.com wrote: I have a bunch of queues that I would like to all have the same permissions. Is there any way to copy permissions from one queue to another so I don't have to configure all of the queues individually? Thanks -- John Alberts Hosted Services Exlibris USA john.albe...@exlibrisgroup.com cell: 1-508-878-2197 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Copy Queue Permissions
Thank you. That helps. On 09/07/2010 10:21 AM, Todd Chapman wrote: Not that I'm aware of but I did write an RT extension that makes it easy to compare queue rights side-by-side and assign them to specific users and groups. http://search.cpan.org/~htchapman/RTx-RightsMatrix-0.03.00/lib/RTx/RightsMatrix.pm http://search.cpan.org/%7Ehtchapman/RTx-RightsMatrix-0.03.00/lib/RTx/RightsMatrix.pm On Sun, Sep 5, 2010 at 12:28 PM, John Alberts john.albe...@exlibrisgroup.com mailto:john.albe...@exlibrisgroup.com wrote: I have a bunch of queues that I would like to all have the same permissions. Is there any way to copy permissions from one queue to another so I don't have to configure all of the queues individually? Thanks -- John Alberts Hosted Services Exlibris USA john.albe...@exlibrisgroup.com cell: 1-508-878-2197 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! -- John Alberts Hosted Services Exlibris USA john.albe...@exlibrisgroup.com cell: 1-508-878-2197 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Slow Ticket History
Hi Tim,
You might want to send your patch to the RT devel list. I would be
interested to see what others, including the BP people have to say about it.
~Jason
On 09/07/2010 07:24 AM, Tim Cutts wrote:
On 6 Sep 2010, at 5:54 pm, rt-users-requ...@lists.bestpractical.com wrote:
So far we've tried installing RT on different hardware, both 32 and 64bit
versions of linux. RT is still very slow for long tickets. All the time is
taken up by the perl/apache process maxing out a core of CPU.
We've even gone as far as trying to profile the code. We came up with this
graph of where the time was going:
TIMING.png
We then tried to go further into those functions but can't find a single
smoking gun call that is taking all the time.
For example in a ticket that takes 22s to render approx 5 secs goes on these 2
lines:
File: Ticket/Elements/ShowHistory line: 100-103 version 3.8.8
my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
From what I can tell, the real problem here is repeated scanning of both the
@attachments and @attachment_content arrays, which makes the execution speed of
the ShowHistory element O(N^2) with respect to the number of transactions;
painful, to say the least.
1) The %$trans_content hash can be made up front in a single pass, for all the
attachments in the ticket, turning this into an O(N) operation, rather than
O(N^2)
2) The $trans_content variable is only used in one place; it's passed to
ShowTransaction, where it is then passed on to ShowTransactionAttachments.
3) In there, there are some errors which cause some autovivification of hash
members which needn't happen.
4) You can do much the same up-front calculation with $trans_attachments as
well, so you don't have to keep grepping through it
I've now made those changes, and on a reasonably large ticket (216
transactions) it reduced the ticket rendering time on my system from 2.5
minutes to 34 seconds, which is a pretty good improvement, I think.
On a more extreme ticket, with 417 transactions, the 3.8.8 release code takes
over 20 minutes to render the ticket (I gave up waiting), so in fact it's
considerably worse order execution time than I thought. My patched code takes
2.2 minutes - still not brilliant but hey, this ticket is now renderable, which
it was not before.
Just for some background, the hardware I'm running on:
RT database: 2 CPU virtual machine, 8GB RAM, MySQL, running on vSphere 4.1 on
a 2.0 GHz E5504 Nehalem system
RT web server: 2 CPU virtual machine, 2GB RAM, on the same type of physical
hardware as the database server
As far as I can tell, I have not semantically altered the code, but others may
want to test more thoroughly. I have not yet put this on my production web
server - I cloned my web server VM and made my changes to the clone (God, I
love VMs for this sort of thing!)
Regards,
Tim
Here are my patches to Ticket/Elements/ShowHistory:
--- /opt/rt3/share/html/Ticket/Elements/ShowHistory 2010-05-14
13:58:15.0 +0100
+++ /opt/rt3/local/html/Ticket/Elements/ShowHistory 2010-09-07
11:59:30.0 +0100
@@ -84,6 +84,10 @@
%perl
my @attachments = @{$Attachments-ItemsArrayRef()};
my @attachment_content = @{$AttachmentContent-ItemsArrayRef()};
+my $trans_content = {};
+map { $trans_content-{$_-TransactionId}-{$_-Id} = $_ }
@attachment_content;
+my $trans_attachments = {};
+map { push (@{$trans_attachments-{$_-TransactionId}}, $_) } @attachments;
while ( my $Transaction = $Transactions-Next ) {
my $skip = 0;
@@ -97,12 +101,6 @@
$i++;
-my @trans_attachments = grep { $_-TransactionId == $Transaction-Id }
@attachments;
-
-my $trans_content = {};
-grep { ($_-TransactionId == $Transaction-Id )
($trans_content-{$_-Id} = $_) } @attachment_content;
-
-
my $IsLastTransaction = 0;
if ( $OldestFirst ) {
$IsLastTransaction = $Transactions-IsLast;
@@ -118,7 +116,7 @@
Transaction = $Transaction,
ShowHeaders = $ShowHeaders,
RowNum = $i,
- Attachments = \...@trans_attachments,
+ Attachments = $trans_attachments-{$Transaction-id},
AttachmentContent= $trans_content,
LastTransaction = $IsLastTransaction
);
and here's the patch to ShowTransactionAttachments (both files need to be
patched):
--- /opt/rt3/share/html/Ticket/Elements/ShowTransactionAttachments
2010-05-14 13:58:15.0 +0100
+++ /opt/rt3/local/html/Ticket/Elements/ShowTransactionAttachments
2010-09-07 11:04:53.0 +0100
@@ -189,8 +189,10 @@
{
my $content;
-if ( $AttachmentContent-{ $message-id } ) {
-$content = $AttachmentContent-{ $message-id
Re: [rt-users] Chose Scrip order?
On 9/1/2010 4:40 PM, Jeff Blaine wrote: Is there any way to prioritize Scrips? Or perhaps someone can suggest something else based on the following? So, no? :) I'm doing something in Scrip 1 that may alter custom field X (employee numbers). Scrip 2 which is in place and working fine right now has the job of looking up some employee details for every employee number listed in field X, and storing those details in field Y. It runs only when: Transaction type is Create OR Transaction field is field X (field X was changed) OR Someone modified field Y by hand (nobody is ever supposed to do this) If I know Scrip 2 runs after Scrip 1, then I know custom field Y will have the correct data because field X was up to date. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
[rt-users] Get the selected value of a CustomField with a Script
Hello, I have a ticket with some Custom Fields. I wrote a scrip and i need to read the Custom Fields values, but i don't find the command. For example, if i have a CF myCF(combobox) with values ( red, orange, green), and user select a value ( red ) What command on my script i can use to get the selected value ? I look for something like that: $myColor = $myCF-getSelectedValue; now $myColor has 'red' Tanks best regards. Marco RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Chose Scrip order?
Jeff, Alphabetically. For those with the same name/function, I put numbers in front, like 1a or 1b. Also, be sure to set the stage to TransactionBatch. Kenn LBNL On Tue, Sep 7, 2010 at 8:38 AM, Jeff Blaine jbla...@kickflop.net wrote: On 9/1/2010 4:40 PM, Jeff Blaine wrote: Is there any way to prioritize Scrips? Or perhaps someone can suggest something else based on the following? So, no? :) I'm doing something in Scrip 1 that may alter custom field X (employee numbers). Scrip 2 which is in place and working fine right now has the job of looking up some employee details for every employee number listed in field X, and storing those details in field Y. It runs only when: Transaction type is Create OR Transaction field is field X (field X was changed) OR Someone modified field Y by hand (nobody is ever supposed to do this) If I know Scrip 2 runs after Scrip 1, then I know custom field Y will have the correct data because field X was up to date. RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT! RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] Get the selected value of a CustomField with a Script
Marco,
Try $Ticket-FirstCustomFieldValue('Name of CF') for templates and
$self-TicketObj-FirstCustomFieldValue('Name of CF') for scrips.
I believe there is an example in the book.
Kenn
LBNL
On Tue, Sep 7, 2010 at 8:51 AM, marco.deso...@billag.com wrote:
Hello,
I have a ticket with some Custom Fields.
I wrote a scrip and i need to read the Custom Fields values,
but i don't find the command.
For example, if i have a CF myCF(combobox) with values ( red, orange,
green),
and user select a value ( red )
What command on my script i can use to get the selected value ?
I look for something like that:
$myColor = $myCF-getSelectedValue;
now $myColor has 'red'
Tanks best regards.
Marco
RT Training in Washington DC, USA on Oct 25 26 2010
Last one this year -- Learn how to get the most out of RT!
RT Training in Washington DC, USA on Oct 25 26 2010
Last one this year -- Learn how to get the most out of RT!
Re: [rt-users] RT::Authen::ExternalAuth AutoCreate [Un]Privileged Users
Gabriel,
Try removing the group 'rt'. Then use filter to accept a broader range of
LDAP users (we use division codes). Then you can use the autocreate
Privileged setting. That way anyone who passes the LDAP test will be
autocreated as Privileged users. that's my best guess.
Kenn
LBNL
On Mon, Sep 6, 2010 at 4:25 AM, Robert Gabriel rgabr...@fnb.co.za wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hello all,
I've done some initial investigation but this doesn't seem to be so
simple for me to do.
Please can someone assist?
I'm using RT::Authen::ExternalAuth and have the following working:
External auth with LDAP and auto create privileged users if they are
in 'rt' group in LDAP.
How can unprivileged users be auto created if they are in LDAP but not
in the 'rt' group when they send a mail ticket request so they can login
through self service access?
PS What should the ExternalInfoPriority be set to if no LDAP
lookups for creating new users via RT?
Thanks.
Set( $rtname, '***.***.**.**');
Set($Organization , '.***.**.**');
Set($Timezone , 'Africa/Johannesburg');
Set(@Plugins,(qw(Extension::QuickDelete RT::FM RT::Authen::ExternalAuth)));
Set( @Plugins, qw(RT::Authen::ExternalAuth) );
Set($RTAddressRegexp , '^(-***)?...@***\.**\.**$');
Set($LogToSyslog , 'debug');
Set($LogToScreen, 'debug');
Set($DatabaseType , 'mysql');
Set($DatabaseHost , '');
Set($DatabaseRTHost , '');
Set($DatabasePort , '');
Set($DatabaseUser , '');
Set($DatabasePassword , '*');
Set($DatabaseName , '');
Set($DatabaseRequireSSL , undef);
Set($OwnerEmail , 'root');
Set($MaxAttachmentSize , 1000);
Set($CanonicalizeOnCreate, 0);
Set($AutoCreate, {Privileged = 1});
require
/opt/rt3/local/plugins/RT-Authen-ExternalAuth/etc/RT_SiteConfig.pm;
Set($ExternalAuthPriority, ['My_LDAP']);
Set($ExternalInfoPriority, ['My_LDAP']);
Set($ExternalServiceUsesSSLorTLS, 0);
Set($AutoCreateNonExternalUsers, 0);
Set($ExternalSettings, {
'My_LDAP' = {
'type'= 'ldap',
'server' = '**',
'user'= '',
'pass'= '',
'base'= 'dc=,dc=***,dc=**,dc=**',
'filter' = '(objectClass=*)',
'd_filter'= '(objectClass=FooBarBaz)',
'tls' = 0,
'ssl_version' = 3,
'net_ldap_args' = [version = 3],
'group' = 'cn=rt,ou=groups,dc=,dc=,dc=**,dc=***',
'group_attr' = 'member',
'attr_match_list' = ['Name', 'EmailAddress'],
'attr_map'= {'Name' = 'uid', 'RealName' = 'cn',
'ExternalAuthId' = 'uid', 'Gecos' = 'cn', 'EmailAddress' = 'mail'}
}
}
);
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJMhM+hAAoJEBMzHChmstlqrfsH/3UFar4PQFUBjN3o7pc4iBce
8oOGftGf75+0/CZkVVt3ogOo+JCFWlfpSb21Kh4YKYMUZ2NXRQVWQO6O25iO8u0x
8aL/rkzei98mKCNlkWP6O/lVIiXeTzAHMJgHJpbC207mEcqRFCKToJ61nOnmtU8I
PBZntO+SRK5V/i+WPFk75/ZmAayJ30wZxVZmThjKPPpINSMkP/y5naUAH1aFwuk0
LMg5CcxloOxq0pEFA6PfQGjetk8NEeF6T01ypS8R8+ArQBrBBJYUJkhuPrRjge3o
Dyl9Eb0wE/HwubZBVixSvLoTMFj4tPo+mYHth+cexMyRZf7br6ieWMSSOwYFNzA=
=dkSU
-END PGP SIGNATURE-
To read FirstRand Bank's Disclaimer for this email click on the following
address or copy into your Internet browser:
https://www.fnb.co.za/disclaimer.html
If you are unable to access the Disclaimer, send a blank e-mail to
firstrandbankdisclai...@fnb.co.za and we will send you a copy of the
Disclaimer.
RT Training in Washington DC, USA on Oct 25 26 2010
Last one this year -- Learn how to get the most out of RT!
RT Training in Washington DC, USA on Oct 25 26 2010
Last one this year -- Learn how to get the most out of RT!
[rt-users] FW: What formatting is available for RTFM Wiki Format?
Bump From: Mike Brady Sent: Friday, 3 September 2010 11:21 a.m. To: 'rt-users@lists.bestpractical.com' Subject: What formatting is available for RTFM Wiki Format? I have just been setting up RTFM in our test environment and am confused about the formatting of articles. The installation all seems to be fine and I have added a custom field with type file in one wiki text area and it shows up when creating an article. There are a couple of contradictory Wiki articles about what formatting is available, neither of which seem to be correct based on my testing. I had been expecting (with no good reason) to see the FCKeditor level of formatting available as per what is available in RT itself. In the middle of one of the Wiki pages a comment is made that only the formatting available in Text::WikiFormat http://wiki.bestpractical.com/view/WikiFormat is what is supported. From my testing this does seem to be the case. Am I missing something here or is this the current state of RTFM? My installed versions are RT 3.8.8 and RTFM 2.4.2 on Centos 5.5. Thanks Mike Connect 2 Ltd. Phone: +64 9 5249596 Mobile: + 64 21 645087 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
[rt-users] Versions of PostgreSQL supported with RT 3.6.7
Hi all, We're using RT 3.6.7 and would like to know what is the latest version of PostgreSQL this is known to work with? This wasn't a detail I could find on the wiki. Thanks in advance you for your responses. Regards, -- Cliff Flood System Administrator +1 416 673 4151 RT Training in Washington DC, USA on Oct 25 26 2010 Last one this year -- Learn how to get the most out of RT!
