Re: [rt-users] RT::User::ExternalAuthId Unimplemented in RT::Record
On Thu, 2 Feb 2017 19:59:47 +
Daniel Burchfield wrote:
> I am trying to get RT to pull in users from my local active directory
> and use AD for auth. Meaning when I change a user's password in AD it
> should reflect the change in RT. I'm running RT 4.4.1. Currently,
> when I run the import I get the following error:
>
> Set($LDAPMapping, {
> Name=> 'sAMAccountName',
> EmailAddress=> 'mail',
> Organization=> 'department',
> RealName=> 'cn',
> NickName=> 'givenName',
> ExternalAuthId => 'sAMAccountName',
This is the culprit line -- this column was removed in RT 4.4. Remove
this line from your configuration, and it should resolve the issue.
- Alex
Re: [rt-users] Problem with index
On Wed, 11 Jan 2017 15:17:53 -0500 François Meehan wrote: > We are using RT 4.2.12. We noticed that the indexation stopped working. > When trying to run the indexer manually, I would get: > > [11238] [Wed Jan 11 19:53:56 2017] [warning]: DBD::mysql::st execute > failed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer > line 216. (/opt/rt4/sbin/rt-fulltext-indexer:216) > [11238] [Wed Jan 11 19:53:56 2017] [warning]: DBD::mysql::st execute > failed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer > line 222. (/opt/rt4/sbin/rt-fulltext-indexer:222) > [11238] [Wed Jan 11 19:53:56 2017] [critical]: Attachment 32267 cannot be > indexed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer > line 254. (/opt/rt4/sbin/../lib/RT.pm:389) > Attachment 32267 cannot be indexed: MySQL server has gone away at > /opt/rt4/sbin/rt-fulltext-indexer line 254. This is likely due to a too-low `max_allowed_packet` in your MySQL configuration. Try bumping it higher and re-running the indexer. - Alex
Re: [rt-users] Queue still being CC'ed with $ParseNewMessageForTicketCcs and $RTAddressRegexp
On Tue, 10 Jan 2017 20:26:05 + "Cena, Stephen (ext. 300)" wrote: > Set($RTAddressRegexp, '(local-part1@domain1\.tld)| > (local-part2@domain2\.tld)| > (local-part3@domain3\.tld)| > : > : > (local-partN@domainN\.tld)/xi'); If you want to use the /x modifier, provide a regular expression object, not a string: Set($RTAddressRegexp, qr/ (local-part1@domain1\.tld)| (local-part2@domain2\.tld)| (local-part3@domain3\.tld)| : : (local-partN@domainN\.tld) /xi ); Tangentially, the inner grouping ()s are not necessary here, but you will want to anchor the regular expression to the start and end of what it's matching. And you'll need to escape the "@" signs: Set($RTAddressRegexp, qr/ ^ ( local-part1\@domain1\.tld | local-part2\@domain2\.tld | local-part3\@domain3\.tld | : | : | local-partN\@domainN\.tld ) $ /xi ); - Alex
Re: [rt-users] Where to put crontool scripts?
On Wed, 4 Jan 2017 11:13:38 -0500 Alex Hall wrote: > I'm considering putting them in /opt/rt4/etc, maybe in a "crontool-scripts" > folder, but I don't know what RT upgrades might do to that. RT upgrades won't remove any extra files you have lying around. - Alex
Re: [rt-users] RT 4.4.1 and transaction isolation level on Postgres
On Tue, 3 Jan 2017 17:06:47 +0100 Václav Ovsík wrote: > How about the Mysql don't have this problem - is this caused by > the different default transaction isolation level or not? MySQL suffers from the exact same problem -- but, as it happens, both more silently and more catastrophically. See https://github.com/bestpractical/rt/commit/e36364c5 > I can change isolation level in postgresql.conf to 'repeatable read' > and things are different. I advise against doing that. Upon inspection, RT is not prepared to deal with the "could not serialize access due to concurrent update" errors that arise from updates to rows in multiple transactions in Postgres' repeatable-read isolation. Repeatable-read is only possible in MySQL because it has a fascinating definition of "repeatable": - Process 1 mysql> set transaction isolation level repeatable read; Query OK, 0 rows affected (0.00 sec) mysql> start transaction with consistent snapshot; Query OK, 0 rows affected (0.00 sec) mysql> select id, Subject from Tickets where id = 1; ++-+ | id | Subject | ++-+ | 1 | foo | ++-+ 1 row in set (0.00 sec) - Process 2 mysql> set transaction isolation level repeatable read; Query OK, 0 rows affected (0.00 sec) mysql> start transaction with consistent snapshot; Query OK, 0 rows affected (0.00 sec) mysql> update Tickets set Subject = 'bar' where id = 1; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> commit; Query OK, 0 rows affected (0.00 sec) - Process 1 mysql> select id, Subject from Tickets where id = 1; ++-+ | id | Subject | ++-+ | 1 | foo | ++-+ 1 row in set (0.00 sec) mysql> select id, Subject from Tickets where id = 1 FOR UPDATE; ++-+ | id | Subject | ++-+ | 1 | bar | ++-+ 1 row in set (0.00 sec) Contrast this with PostgreSQL, whose definition of repeatable read acknowledges that fully consistent updates are not possible in all cases: - Process 1 rt4=# start transaction; START TRANSACTION rt4=# set transaction isolation level repeatable read; SET rt4=# select id, Subject from Tickets where id = 1; id | subject +- 1 | foo (1 row) - Process 2 rt4=# start transaction; START TRANSACTION rt4=# set transaction isolation level repeatable read; SET rt4=# update Tickets set Subject = 'bar' where id = 1; UPDATE 1 rt4=# commit; COMMIT - Process 1 rt4=# select id, Subject from Tickets where id = 1; id | subject +- 1 | foo (1 row) rt4=# select id, Subject from Tickets where id = 1 FOR UPDATE; ERROR: could not serialize access due to concurrent update ( Yes, MySQL requires SET TRANSACTION ISOLATION _outside_ the transaction, and PostgreSQL requires it to be _inside_. See https://dev.mysql.com/doc/refman/5.7/en/set-transaction.html https://www.postgresql.org/docs/9.1/static/sql-set-transaction.html ) > Should I change the default isolation level on Postgres for RT to > 'repeatable read'? No. You should try the 4.4/previewscrips-race branch, which I've just pushed: https://github.com/bestpractical/rt/compare/4.4-trunk...4.4/previewscrips-race The gory details are contained in the commits therein. - Alex
Re: [rt-users] RT 4.4.1 and transaction isolation level on Postgres
On Mon, 2 Jan 2017 17:12:29 +0100 Václav Ovsík wrote: > Can anybody confirm on different system? Thanks for the detailed replication instructions. I can replicate, and have tracked down a minimal replication case. I'll drop my findings and suggestion on your ticket. The short form is that this is due to 4.4's new PreviewScrips functionality, which simulates (then rolls back) all of the changes, which is racing with the actual change. The bad news is that it's the real change, not the dry run, which gets killed in the deadlock detector, meaning that the owner does go unchanged. At least this is bubbled up to the user in the ticket display page, but this is still quite unfortunate. Amusingly, there are some strong parallels to the canonical Therac-25 case[1] -- the race here requires that one trigger a PreviewScrips very quickly before submitting, which is rare except with users quite accustomed to the UI. In the Therac-25 case, only skilled users could navigate to the bottom of the form within 8 seconds and thus deliver lethal doses of radiation to their patients. ...RT is designed to be robust, but there's a reason it doesn't rate itself as meant to be used for safety- or life-critical applications. - Alex [1] https://en.wikipedia.org/wiki/Therac-25
Re: [rt-users] Ticket SQL for dates?
On Sat, 31 Dec 2016 12:41:21 -0500 Alex Hall wrote: > In a thread a week or two ago, I was asking about the syntax for finding > tickets by relative dates, like "3 days ago". It wasn't working, and at > least one other list member was able to confirm that it wasn't. You're getting tripped up by the "ago", which in English inverts the meaning of less-than and greater-than. A search for "LastUpdated > '3 days ago'" finds tickets whose LastUpdated stamp is after the point in time 3 days ago -- that is, within the last three days. Here's that search on issues.bestpractical.com: https://issues.bestpractical.com/Search/Results.html?Query=Queue+%3D+%27RT%27+AND+Status+%3D+%27__Active__%27+AND+LastUpdated+%3E+%273+days+ago%27 TicketSQL isn't SQL because we don't want to allow SQL injection attacks. - Alex
Re: [rt-users] mysql DB engine ndbdcluster
On Wed, 14 Dec 2016 09:42:42 -0500 Mike Diehl wrote: > At the risk of picking a fight, I'd like to understand this a bit better. Happy to explain more -- and my instinct may have been wrong on one count; see below. > As long as the database supports minimum functions, such as transactions, > joins, datatypes, etc., why should an application care about the underlying > storage engine? Because distributed databases have different properties around atomicity and data locality than single-host databases. Applications running atop such databases need to be built to accommodate these correctness and performance properties. The biggest issue is that of transaction isolation[1] -- not all transactions are equal. RT assumes that a database transaction gives it "repeatable read" isolation from other transactions. This isolation level, the default for InnoDB tables[2], means that essentially, upon the first read, a snapshot of the state of the database is taken, and it provides strong guarantees that regardless how long the transaction is open, all queries within it will return consistent data[3]. I believe it likely (though I cannot prove, offhand) that RT assumes repeatable read isolation semantics -- and NDB only offers "read committed" isolation, which admits the possibility of different results for the same query run twice within the same transaction. However, upon writing this, it occurs to me that Postgres' default isolation level is _also_ "read committed."[4] Thus any possible race conditions that might show up under NDB are also possible under Postgres. I'd need to do some close analysis to determine if this means that Postgres is open to data corruption, or if both are safe because the queries RT runs do not care about repeatable-read semantics. So NDB may actually be fine on this front. The other property concerns data locality, and is purely a performance constraint. NDB stores data across a cluster of data notes, optionally with replication, which are queried by other hosts that serve as SQL nodes[5]. This means that joining data across tables cannot be done in-memory, but instead may incur network-level latencies to match up the data between data nodes -- meaning poor query performance. MySQL Cluster 7.2 (equivalent to MySQL 5.5) does provide some tricks to prevent this performance hit[6], but it's not clear that those optimizations will be able to be applied to RT's queries, as not all of the column types match between tables. It also doesn't get you all of the way to InnoDB join performance. Finally, the tables may also need explicit hinting in order to partition the data to give any sort of locality across the hosts. On the other hand, if you're deploying an NDB cluster, you may already have the MySQL DBAs on-hand to attend to those. I've never heard of an NDB deploy, discovering the correct partitioning scheme would be all uncharted territory. NDB clusters also don't support FULLTEXT indexes[7], though that's clearly only an optional feature for RT. Pescoller, consider me curious to hear back if you actually deploy RT against and NDB cluster in production, and the performance characteristics you see compared to single-host InnoDB. - Alex [1] https://en.wikipedia.org/wiki/Isolation_(database_systems) [2] https://dev.mysql.com/doc/refman/5.7/en/innodb-transaction-isolation-levels.html [3] Repeatable read nominally opens up the possibility of "phantom reads" where range queries can return inconsistent data from one query to another; however, InnoDB uses some clever locking tricks to prevent them. [4] https://www.postgresql.org/docs/9.1/static/transaction-iso.html#XACT-READ-COMMITTED [5] http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-overview.html [6] http://mysqlhighavailability.com/70x-faster-joins-with-aql-in-mysql-cluster-7-2/ [7] https://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-limitations-syntax.html - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Los Angeles - January 9-11 2017
Re: [rt-users] mysql DB engine ndbdcluster
On Tue, 13 Dec 2016 12:25:37 +0100 Pescoller Reinhold wrote: > Thanks for your informations. > > I tried to do so but rt give me an error that innodb is required and > that I should upgrade my tables. > Have I to change this direct in the code? lib/RT/Handle.pm: https://github.com/bestpractical/rt/blob/095caac2a4b4fc7baba0d7878a79f8b486579854/lib/RT/Handle.pm#L291 I'll reiterate that while RT may appear to work in trivial conditions, you're setting yourself up for a world of both poor performance and nasty race condition bugs. You get to keep all of the pieces when it bursts into flames in production -- NDB is in no way a supported, suggested, or sane backing engine for RT. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Los Angeles - January 9-11 2017
Re: [rt-users] mysql DB engine ndbdcluster
On Tue, 13 Dec 2016 10:28:41 +0100 Reinhold Pescoller wrote: > Is there some possibilty to change the default db engine from innodb to > ndbdcluster in rt4? RT assumes REPEATABLE READ isolation; you may encounter subtle and difficult to diagnose bugs under READ COMMITTED isolation, which is all that NDB supports. Performance of joins is generally not great under NDB, and RT assumes that joins do not incur a significant cost penalty. In short, you _might_ be able to get it to run simply by adjusting the table types, but I only expect it to get you into trouble down the road. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Los Angeles - January 9-11 2017
Re: [rt-users] supplying database credentials to rt-fulltext-indexer
On Fri, 9 Dec 2016 17:30:31 -0500 Alex Hall wrote: > What this file never says is how to tell the indexer tool how to connect to > the database. It clearly isn't pulling from the RT configuration, nor from > /home/www-data/rtrc. The indexer reads and uses the database configuration from your /opt/rt4/etc/RT_Config.pm and /opt/rt4/tec/RT_SiteConfig.pm files. rtrc files are _only_ used by the "bin/rt" tool, which is meant to be run from other machines than your RT host. The rt-setup-fulltext-indexer wants to know your DBA username and password because it needs to create new tables and indexes, which the standard RT database user does not have permissions to do. Once the index is set up, updates to the index are done as RT's standard database user. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Los Angeles - January 9-11 2017
Re: [rt-users] FTS enabled in SiteConfig, but not being enabled
On Tue, 15 Nov 2016 11:36:53 -0500 Alex Hall wrote: > I found the problem. I've seen "Table" and "TableName" both used in sample > configurations, but apparently only one is correct. As soon as I switched > to "Table" in my config file, FTS enabled itself. If any of the configurations with "TableName" that you found were on BPS' site, please let them know where, so it can be fixed. If they were on the wiki, please update them. When I start RT 4.4.1 with "TableName" instead of "Table", I see the following in my error log: [error]: No Table set for full-text index; disabling It surprises me that you didn't see such an error. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Los Angeles - January 9-11 2017
Re: [rt-users] rt-setup-fulltext-index dropped DB connections with MySQL
On Thu, 13 Oct 2016 20:32:36 + James Zuelow wrote: > Package versions at the moment are MySQL 5.6.30, Perl 5.2.22, and > request-tracker 4.2.13. There is a Perl 5.4 upgrade available, but > that breaks my rt4 installation. For reference, your perl versions are not comprehensible. Wheezy ships perl 5.14.2, per https://packages.debian.org/wheezy/perl, and there has never been a perl version 5.2 or 5.4 -- 5.002 was released in 1996, 5.004 in 1997, and 5.6 in 2000. > Indexing existing data... > Going to run the following in the DB: > CREATE FULLTEXT INDEX AttachmentsIndex ON AttachmentsIndex(Content) > > [58876] [Thu Oct 13 02:23:42 2016] [warning]: DBD::mysql::db do failed: > Lost connection to MySQL server during query at > ./rt-setup-fulltext-index line 736, line 2. Check your "max_allowed_packet" setting in mysqld. The most common cause is the MySQL client (here, rt-fulltext-indexer) sending something in the MySQL wire protocol which is larger than that limit, which causes the server to unceremoniously drop the connection. Raise that limit and restart mysql, and it should resolve the issue. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] Customizing with Overlays
On Mon, 3 Oct 2016 08:58:25 +0530 Nilesh wrote: > Can I put my Overlay files in local/lib? Yup; that's what's generally suggested. > Do I have to maintain the same directory structure? Yes. > Let's say I have to customize RT::Action::SendEmail, so my Overlay should > be in local/lib/RT/Action/SendEmail_Overlay.pm or just > local/lib/SendEmail_Overlay.pm ?? The former. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] Bug about subject in utf-8
On Thu, 1 Sep 2016 09:42:59 +0200
Albert Shih wrote:
> > First, https://tools.ietf.org/html/rfc2047#page-5
> > unencoded white space characters (such as SPACE and HTAB) are
> > FORBIDDEN within an 'encoded-word'
> >
> > As such, "=?utf-8?q? #NUMBER=5D?=" is not a valid encoded-word.
>
> Well I think that's my bad, I change a little the subject to fit my first
> email about the tag. The real subject is
>
> =?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour
> =?utf-8?q?=C3=A0?= vous
OK, that's a little different. Rather better. It still violates:
> However, an 'encoded-word' that appears in a header field defined as
> '*text' MUST be separated from any adjacent 'encoded-word' or 'text'
> by 'linear-white-space'.
But:
> I'm a not very good with perl, but when I try using ruby to decode this
> line
>
> irb(main):008:0>
> Mail::Encodings.unquote_and_convert_to('=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?=
> Obspm =?utf-8?q?#31684=5D?= Bonjour =?utf-8?q?=C3=A0?= vous','utf-8')
> => "Re: [Info Obspm #31684] Bonjour à vous"
>
> the result seem correct.
For decoders that are lenient to encoded-words that aren't
space-separated, that's correct. The difference between this and what
you had previously is the non-encoded word between the two
encoded-words, which makes the space significant.
And indeed, this does point to an RT bug. Namely, for historical and
bad reasons, RT doesn't use the standard MIME-words decoding library,
which would produce:
> perl -MEncode -lE 'print Encode::encode("utf8",
> Encode::decode("MIME-header",
> "=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour
> =?utf-8?q?=C3=A0?= vous"))'
>
> Re: [Info Obspm #31684] Bonjour à vous
Instead, it rolls its own, and gets it wrong:
> perl -Ilib -MRT=-init -le 'print RT::I18N::DecodeMIMEWordsToUTF8(
> "=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour
> =?utf-8?q?=C3=A0?= vous","Subject")'
>
> Re: [Info Obspm#31684] Bonjourà vous
Specifically, it removes spaces before the second and later
encoded-words, due to
https://github.com/bestpractical/rt/blob/stable/lib/RT/I18N.pm#L445
This looks to be a bug. I've pushed 4.2/encoded-word-spaces to
address it; if you'd like to test the fix locally, you can apply
https://github.com/bestpractical/rt/commit/bdd6bd96 .
Thanks for the more complete bug report.
- Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day!
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017
Re: [rt-users] Bug about subject in utf-8
On Wed, 31 Aug 2016 23:12:28 +0200 Albert Shih wrote: > So until known everything is correct. The problem is when the person who > answer this ticket encode the subject like this > > =?utf-8?q?Re=3A?==?utf-8?q?_=5BRTTAG =?utf-8?q? #NUMBER=5D?= Bonjour > =?utf-8?q?=C3=A0?= vous > > because in that case RT drop the space between the RTTAG and the #NUMBER. What mail client is generating that? Whatever it is, it is violating RFC 2047 spec in _multiple_ ways. First, https://tools.ietf.org/html/rfc2047#page-5 unencoded white space characters (such as SPACE and HTAB) are FORBIDDEN within an 'encoded-word' As such, "=?utf-8?q? #NUMBER=5D?=" is not a valid encoded-word. Secondly, https://tools.ietf.org/html/rfc2047#page-7 However, an 'encoded-word' that appears in a header field defined as '*text' MUST be separated from any adjacent 'encoded-word' or 'text' by 'linear-white-space'. As such, "=?utf-8?q?Re=3A?==?utf-8?" is not valid, as the two "encoded-word"s are not separated by spaces. Even ignoring those errors, the example you gave still isn't parsable. My best attempt splits it into the following tokens: =?utf-8?q?Re=3A?= # "Re: =?utf-8?q?_=5BRTTAG # " [RTTAG", but no closing "?=" ?! =?utf-8?q?#NUMBER=5D?=# "#NUMBER]" Bonjour # "bonjour" =?utf-8?q?=C3=A0?=# "à vous # "vous" Were it somehow parsed as the above, RT would _still_ be correct in omitting the space before the number, because space between encoded-words is removed, https://tools.ietf.org/html/rfc2047#page-10 : When displaying a particular header field that contains multiple 'encoded-word's, any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored. In short, fix the mail client. Failing that, set $ExtractSubjectTagMatch, as this is not a bug in RT. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!
On Tue, 23 Aug 2016 13:21:20 +0200 Claes Merin wrote: > I beg to differ about the attachments table, in our setup we rely on > customfields and there parameters. We have quite a few tickets that does > not have any attachments. > If you do not write anything in the message body when you create a > ticket (through web-ui or REST), there will be no attachment connected > to that ticket. Creating tickets via the web UI creates empty Attachment rows if the user enters no content, from my testing on 4.4. I thought we'd resolved that REST didn't do the same, but I concur that still isn't the case. > Also i did a comparison between mysql/mariadb FULLTEXT index and sphinx, > and it seems that sphinxsearch is able to extract more info than the > mysql FTS. I guess it's about the data processing that sphinx does with > encoding and other things. I'd be curious to hear more about this. What do you mean by "extract more info"? Do you mean better stemming, or what? > Our goal is to include CF's in the fulltext index as well. As far as I > can see right now, there are two ways to do this. Be aware that this can have security implications, as RT has no way to apply ACLs if all of the different content sources are in the same index; it becomes possible to find tickets based on custom field values you cannot actually see. > a) Modify the RT simplesearch use an "TicketsIndex" instead of the > "AttachmentsIndex". > > Use sphinx to create the TicketsIndex table and modify the collector > query to include both attachment, ticket.subject and customfields in the > the database. (Simple join statements) > > b) Create "null" attachments and attach these to tickets that does not > have any current attachments, just to create and relation between ticket > and attachment. The advantage with this is that we do not have to modify > any of the RT code, but of course it adds some rows to the database. (b) makes more sense to me. You currently can limit on transaction date and creator, and those limits apply in conjunction with content limits. If you effectively lump all attachment content to be cross-referenced by ticket-id instead of attachment-id, you lose that functionality. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] Full text indexing error with MySQL
On Thu, 25 Aug 2016 20:49:48 -0700 Omen Wild wrote: > I activated the full text indexing under Ubuntu 16.04 (request-tracker4 > 4.2.12-5) and (mysql-server 5.7.13-0ubuntu0.16.04.2). Both the initial > indexing and subsequent calls to rt-fulltext-indexer have the same error: > > - Begin quote - > [3159] [Fri Aug 26 03:03:08 2016] [warning]: DBD::mysql::st execute failed: > MySQL server has gone away That error is almost always that RT tried to do an insert or update of data larger that the MySQL server's max_allowed_packet. Try increasing that setting, restarting MySQL, and indexing again. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!
On Fri, 19 Aug 2016 16:35:07 +0200 Claes Merin wrote: > We are using sphinx (since last night) for FTS searches. > > Setup is as described in this excellent guide: > > http://blog.christosoft.de/2016/07/rt4-request-tracker4-fulltext-search-sphinx-debian-jessie/ Is there a reason you're not using MySQL's built-in search, available since 4.2.10? It's more performant, and has none of the caveats: https://docs.bestpractical.com/rt/4.2/full_text_indexing.html#Caveats1 Especially since it allows for incremental indexing of new attachment rows, I know of no reason to prefer Sphinx. > We also ran in to the problem described here, and I was thinking if one > could resolve it by including the subject in the sphinx database... I expect that should work fine. > Modifying the query in the guide to simply include the t.Subject like > this... > > ## /etc/sphinxsearch/sphinx.conf > sql_query = \ > SELECT a.id id, t.Subject, a.content FROM Attachments a \ > JOIN Transactions txn ON a.TransactionId = txn.id AND txn.ObjectType = > 'RT::Ticket' \ > JOIN Tickets t ON txn.ObjectId = t.id \ > WHERE a.ContentType LIKE '%text%' AND t.Status != 'deleted' > ## > > The problem with this is, if there is a ticket that does not have an > attachment, it will not get indexed and is not searchable through FTS... The "attachments" table is used to index all content in RT. This includes textual content, such as the bodies of messages. As such, it is impossible for a ticket to not have at least one Attachment record. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] manipulate correspond content in history
On Fri, 19 Aug 2016 15:26:12 +0300 Woody - Wild Thing Safaris wrote: > My correspond scrip adds a table of CF values to the correspond email > from the template, and sends to the client. I would like those same > changes appended to the transaction content that appears in the history, > so it matches what was sent to the client and we know easily what was sent. This is what the "Show" link next to the "Outgoing email recorded" button is for. > $self->TransactionObj->ContentObj->SetContent($content); > > no errors, but the content is not changed All ->Set... methods return a tuple of ($success, $msg). You'll find that $success is false, and $msg is telling you that the field is read-only. RT views attachments as effectively immutable. Changing this invariant is complex, and not generally recommended. - Alex - RT 4.4 and RTIR training sessions, and a new workshop day! https://bestpractical.com/training * Boston - October 24-26 * Los Angeles - Q1 2017
Re: [rt-users] RT adding words on the beginning of the emails
On Thu, 4 Aug 2016 14:26:58 + Renos Nikolaou wrote: > We are using RT 4.4.1 and we would like to know why our RT adding the > word 'Capture' (When we forward screenshot from Snipping Tool to > outlook and then to open a ticket) and some other word like our > organization name on the beginning of each emails ? > > Is there any way to disable this option ? This is not an "option" from RT, merely a property of RT's HTML rendering of the mail that Snipping Tool creates. If you click the "Download (untitled)" link on the right edge, you can view the HTML that is being generated, which may make clear where the "Capture" word is coming from. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Los Angeles - September, 2016
Re: [rt-users] How to sync translations?
On Tue, 26 Jul 2016 10:53:04 -0700 (MST) fleon wrote: > I have my own spanish translation (which i uploaded today in launchpad). It's > about 99% complete. But today i upgraded to 4.4.1 and lost my changes (i did > backup it first). Looks like there were failures importing your changes: https://translations.launchpad.net/rt/4.4/+imports Namely, line 5997 has an extra " at end of line. Fix those up and re-upload. Once that imports, you can export the translations (see below) and look for new untranslated strings. > What method should i use to put my file back and know which new lines need > translating, if any? Are all new strings at the bottom of the file? The .po file is sorted alphabetically. > I tried using diff but since the line numbers in the comments changed it > shows up too many changes. Perhaps some script-fu to ignore all lines with > comments? If you export the translations from Launchpad as a tarball from https://translations.launchpad.net/rt/4.4/+export , it will email you a link. Pass that link to: devel/tools/rt-message-catalog rosetta the-url-from-email-here ...and it will merge in the translations from Launchpad into your local share/po/*.po files, preferring translations from launchpad but leaving any new local translations. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Los Angeles - September, 2016
Re: [rt-users] Bizarre errors with shredder
On Tue, 12 Apr 2016 10:41:24 +0200 (CEST) "Sternberger, Sven" wrote: > root@rt:~# /opt/rt4/sbin/rt-shredder --force --plugin > "Users=no_tickets,1;status,any;replace_relations,Nobody;limit,3;member_of,unprivileged" > SQL dump file is '/root/20160411T134517-0001.sql' > [2329] [Mon Apr 11 13:45:50 2016] [critical]: Couldn't wipeout object: > Bizarre copy of CODE in list assignment at > /usr/local/share/perl/5.18.2/Devel/StackTrace.pm line 61. > (/opt/rt4/sbin/../lib/RT.pm:388) "Bizarre copy of ..." is always a sign of an internal error in perl. In this case, it's caused by the fact that objects on the perl stack aren't refcounted, which means that in some cases they can already have been garbage-collected by the time the object is examined. Specifically, when RT produces an error, Devel::StackTrace tries to suss out the values that were passed to each function in the call stack -- and in some cases, those values may no longer exist. So this is a sign of some other error, which Devel::StackTrace is trying to build the stack trace for, which itself triggers an internal perl error. RT catches _that_ error and gives the stack trace for _that_, which is why what you're seeing is a stack trace of the log infrastructure. You can likely work around this by disabling LogStackTraces, which I believe you've turned on. You'll then see the underlying RT error that it was trying to report the stack trace of. Perl 5.18 is technically out of support, but "stack isn't refcounted" isn't exactly fixed in any recent Perls, so upgrading might only fix this if it happened to shift the global destruction order. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Washington DC - May 23 & 24, 2016
Re: [rt-users] Full text indexer issue with img html tag
On Thu, 31 Mar 2016 09:47:06 +0200 Peter Viskup wrote: > Just discovered we didn't ran rt-fulltext-indexer enough many times. > It proceeded with only 200 attachments at once. Is that some > limitation of PgSQL or rt-fulltext-indexer? Didn't read that in the > documentation. See https://docs.bestpractical.com/rt/4.4.0/full_text_indexing.html#Updating-the-index You can pass --all to index all attachments, which looks to not be documented. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Washington DC - May 23 & 24, 2016
Re: [rt-users] Full text indexer issue with img html tag
On Wed, 30 Mar 2016 21:32:37 -0700 Alex Vandiver wrote: > What version of RT? That's a symptom of RT 4.0.9 or earlier; 4.0.10 > contains a fix that makes RT simply skip that attachment. Actually, my mistake -- it was fixed in 4.0.3: https://github.com/bestpractical/rt/commit/692b5bcb If you're running pre-4.0.3, you _certainly_ want to upgrade. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Washington DC - May 23 & 24, 2016
Re: [rt-users] Full text indexer issue with img html tag
On Wed, 30 Mar 2016 17:41:02 +0200 Peter Viskup wrote: > we are experiencing issue with full text indexer What version of RT? That's a symptom of RT 4.0.9 or earlier; gradi4.0.10 contains a fix that makes RT simply skip that attachment. - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Washington DC - May 23 & 24, 2016
Re: [rt-users] PostgreSQL full-text index default type: GiST or GIN
On Thu, 17 Mar 2016 17:47:50 -0500 Matt Zagrabelny wrote: > > I am curious about the benchmarks you used to make this determination? > > In particular, the GIN fastscan option can dramatically improve search > > performance in versions 9.4 and above. Here is a nice discussion: > > > > http://blog.pgaddict.com/posts/performance-since-postgresql-7-4-to-9-4-fulltext > > > > In addition, GIN indexes are much smaller in PostgreSQL 9.4 and above. > > So I am curious about the data that motivated the change to GiST. You are quite correct; GIN indexes are superior in virtually every way for our use case. And the documentation is unfortunately in error; the default switched in the other direction, from GiST to GIN. See https://github.com/bestpractical/rt/commit/e103f6da for the actual behavior change. This mistake is entirely my fault, and my only excuse for getting it backwards is that the documentation commit was written several months after the code was. Apologies! > Here is a correspondence between myself and a BP engineer you may find > helpful: > [snip] That correspondence is out of date; at the time, GiST was the default, based on no particular research. The change to default to GIN was made late 2014/early 2015 after I conducted additional research on real-world data -- as well as the Postgres documentation, which is quite clear that static data should use GIN. You can see the results of that research at https://chmrr.net/fts-charts/pg.html and https://chmrr.net/fts-charts/query.html#pg Those are from performing speed analysis on rebuilding the index on a clone of Best Practical's own ticketing system, which was taken as being a representative real-life sample of data. It was run on Postgres 9.3, and resulted in the following branch: https://github.com/bestpractical/rt/compare/1d1ffe44...7c48294a The last ~7 commits are the most Postgres-specific. The end result is an indexing engine which, in its default configuration, indexes data an order of magnitude faster, as well as provides results an order of magnitude faster. Wins all around! I'm happy to supply a patch to fix the documentation, but I suspect Shawn or someone at BPS will be faster to simply fix it directly. :) - Alex - RT 4.4 and RTIR Training Sessions https://bestpractical.com/training * Washington DC - May 23 & 24, 2016
Re: [rt-users] GnuPG key management issues
On Thu, Oct 08, 2015 at 08:40:37AM +0200, Christopher Kunz wrote: > we are using the RT::Crypt::GnuPG module and have run into an issue that > I'm afraid we cannot solve. > [snip] > This is where the problems start. With our current setup (and a local > keyring on the RT server), RT will try handling the .asc attachment as > encrypted data (which it isn't), fail and not handle the rest of the > ticket. I believe the 4.2/skip-asc-keys branch[1] addresses this particular issue, of interpreting .asc as encrypted data. I'm sure that BPS would appreciate the feedback if it resolves the issue for you. > We are using RT 4.2.8. Is there any configuration setting that I might > have missed, or any other possibility to get RT to automatically import > keys that come into the system as a .asc attachment? I am aware of the > security implications of this (and, in fact, have set the trust settings > to "always", as authentication of user requests is always performed > separately). The security implications of such a flag almost certainly preclude its inclusion in core RT -- though you understand the security implications, many sites might not, and might enable it regardless of any warnings placed on it. Operating in such configurations is far worse than operating without GPG at all. If you wish to implement this yourself, there are a couple options for where to implement the behavior. The first two of them will require that the web user have write access to the keyring. 1. Implement it as a mail plugin, run before RT::Crypt::GnuPG. This will allow it to import the key before the decryption happens. Mail plugins in 4.2 are cumbersome, but an ApplyBeforeDecode plugin that is in @MailPlugins before Auth::Crypt should work fine. I would suggest using RT::Extension::FutureMailgate[2] backport of the 4.4/mailgate-refactor branch[3] to make writing the mail plugin entirely more friendly, but I just checked, and it looks like that backport doesn't include the Crypt changes from the branch[3], so running that may be unhelpful. 2. Write a scrip to parse out the keys after the mail comes in. This means that if the original mail is signed by the key it includes, the key will be added after it has been examined, so it will be marked as unverified. 3. Pre-process the keys in the mail gateway, or before. If the mail has a standard format, you can potentially do all of the work by having procmail or similar match the message and pipe it to a custom program that extracts and adds the key. This removes RT from the loop entirely. If this is central to your workflow, you may wish to consider contacting sa...@bestpractical.com to see if they can help you implement one of the above solutions. - Alex [1] https://github.com/bestpractical/rt/compare/4.2-trunk...4.2/skip-asc-keys [2] https://github.com/bestpractical/rt-extension-futuremailgate [3] https://github.com/bestpractical/rt/compare/master...4.4/gateway-refactor
Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!
On Wed, Oct 07, 2015 at 04:23:02PM -0500, k...@rice.edu wrote: > It looks like you should be able to change the following function in > lib/RT/Search/Simple.pm: [snip] RT 4.2.10 made the change to only search the Content field, and not also the Subject field, intentionally; see the commit the did so [1] for the full rationale. In short, it is impossible to have a performant full-text search if you search both Subject and Content. In most RT installs, the auto-reply from the ticket creation includes the subject of the ticket, which means it is indexed along with the ticket, so removing the Subject clause still results in finding the appropriate ticket. You're welcome to revert the change locally -- though I would suggest doing so via an overlay and not by editing the file directly, or your change will be lost when you next upgrade. If you do make the change, I expect you'll find the simple search to be noticeably (and perhaps unusably) slower. - Alex [1] https://github.com/bestpractical/rt/commit/8450f0a9f233d6a761ac22dbdf14926abc54d7fa
Re: [rt-users] 'Started' not being set when ticket status changes from 'New' -> 'xxx'
On Sat, 12 Sep 2015 13:37:51 -0700 Alex Vandiver wrote: > On Fri, 11 Sep 2015 14:55:47 -0500 "k...@rice.edu" > wrote: > > Just to follow-up. I made a global scrip to set the 'Started' date > > field when it is un-set and the transaction is a status change from > > 'new' to either 'open' or 'resolved'. I thought that this was > > handled by RT internally when the status changed from 'new' but I > > could not find any place in the code that handled it. > > This was in core code in RT 4.0, but was moved to a global scrip in RT > 4.2: [snip] Sorry, I mis-read your question; the scrip in question is for the opposite (opening the ticket if Started is set). The code that sets Started when a ticket is moved to a non-initial state is: https://github.com/bestpractical/rt/blob/stable/lib/RT/Ticket.pm#L2393-L2403 The only thing that could be causing that to not fire is if you have a custom lifecycle with all-Initial statuses. - Alex
Re: [rt-users] 'Started' not being set when ticket status changes from 'New' -> 'xxx'
On Fri, 11 Sep 2015 14:55:47 -0500 "k...@rice.edu" wrote: > Just to follow-up. I made a global scrip to set the 'Started' date > field when it is un-set and the transaction is a status change from > 'new' to either 'open' or 'resolved'. I thought that this was handled > by RT internally when the status changed from 'new' but I could not > find any place in the code that handled it. This was in core code in RT 4.0, but was moved to a global scrip in RT 4.2: https://bestpractical.com/docs/rt/4.2/UPGRADING-4.2.html (search for Started) If this was an RT instance that was upgraded from 4.0, the lack of this behavior my imply that not all upgrade steps were run. - Alex
Re: [rt-users] forwarding correspondence lacks important data in headers
On Tue, Sep 08, 2015 at 03:35:46AM -0400, Alex Vandiver wrote: > > I would expect to see proper From, To, Subject, Date (like in original > > correspondence) and possibly other headers, too. Also there is incorrect > > Content-Length in it. > > Hm -- the content-length problem doesn't seem to be replicated on > issues.bestpractical.com. Are you seeing it with a vanilla 4.2.12 > install? Wait, the content-length of a multipart/mixed part should indeed be 0 -- it itself has no content. It is the sub-parts that have content. Arguably Content-Length should be entirely omitted on multipart/* parts, but its presence is unlikely to cause interoperability concerns. But that does beg the question -- are you seeing the headers bug only on multipart/mixed messages? How are you creating them via the web UI, by adding attachments? - Alex
Re: [rt-users] forwarding correspondence lacks important data in headers
On Tue, Sep 08, 2015 at 08:56:53AM +0200, Arkadiusz Miśkiewicz wrote: > > Hi. > > rt 4.2.12, scenario - there is a ticket, user enters new correspondence via > web interface which is send via email to requestor. > > Then user clicks "forward" near that correspondence and forwards it to third > email address / third person. > > Unfortunately that email sent to third person is lacking important > information. It lacks From, To, Subject headers in attached forwarded message. This should have been fixed in 4.2.10: https://bestpractical.com/release-notes/rt/4.2.10 https://github.com/bestpractical/rt/commit/134a478be1b63ca418380fc355923ea849812bfa Are you still seeing a lack of such headers for tickets created since your upgrade to 4.2.12? Tickets created _prior_ to an upgrade to 4.2.10 will still lack the headers in their Create transactions, of course. > I would expect to see proper From, To, Subject, Date (like in original > correspondence) and possibly other headers, too. Also there is incorrect > Content-Length in it. Hm -- the content-length problem doesn't seem to be replicated on issues.bestpractical.com. Are you seeing it with a vanilla 4.2.12 install? - Alex
Re: [rt-users] rt-mailgate needs http for comment
On Sun, Sep 06, 2015 at 12:31:28AM -0700, Joseph D. Wagner wrote: > Here it is. I left everything intact except the url. > [snip] Those look fine. Double-check that you have not multiply-defined prc-staff elsewhere in aliases, and that you've run `newaliases`. Short of that, my only suggestion is to turn on bug logging in your MTA -- I can say with certainty that rt-mailgate doesn't deal differently with correspond vs comment and http/https connections. - Alex
Re: [rt-users] rt-mailgate needs http for comment
On Sat, Sep 05, 2015 at 11:22:48PM -0700, Joseph D. Wagner wrote: > I had apache set to allow rt over https only. Trying over http would fail. > > rt-mailgate was working perfectly fine over https when using > "--action correspond". However, when using "--action comment", it > fails saying it is unable to connect. > > When I changed my apache configuration to allow http on local > connections, it started working. > > I suspect there is some code in the comment path of rt-mailgate that > is forcing it over the http connection, rather than properly > deriving the connection from the url parameter. > > Being new to rt, I am open to the possibility I misconfigured > something. Is anyone else able to reproduce this? I strongly suspect misconfiguration in your /etc/aliases. The only difference between correspond and comment paths is the value of a query parameter that they POST: https://github.com/bestpractical/rt/blob/stable/bin/rt-mailgate.in#L168-L170 Check to make sure that you have https:// on all of your aliases, and that you've run newaliases (or equivalent) after updating them. - Alex
Re: [rt-users] Customize QuickSearch for RT 4.0.19 - Removing all
On Fri, Sep 04, 2015 at 09:45:56AM -0700, ewanchic wrote: > I'm not Looking to change the Lifecycles because that look fine to me, but I > would like to just remove all the extra criteria and just focus on the > criteria requested in the Quick Search. Is this possible? Thanks https://docs.bestpractical.com/RT_Config#OnlySearchActiveTicketsInSimpleSearch Set this to 0 in your RT_SiteConfig.pm: Set( $OnlySearchActiveTicketsInSimpleSearch, 0); - Alex
Re: [rt-users] RT and Disaster Recovery - problem
On Wed, Sep 02, 2015 at 03:11:52PM +, Guadagnino Cristiano wrote: > Our RT servers are virtualized on VMware. > [snip] > However, the problem is that - after reconfiguring the VMs - RT becomes > slow as a snail (tens of seconds for each page change/refresh). Do the VMs have any snapshots enabled? I know that historically, at least, the mere presence of a snapshot file caused all I/O to be COW, which caused order-of-magnitude I/O bandwidth degradation. - Alex
Re: [rt-users] scrip to delete a Cc adress in a ticket
On Mon, Aug 31, 2015 at 05:24:51PM +0200, Loïc Cadoret wrote: > We are running RT 3.8.11 (update to RT 4.2.x is currently not an option) Your RT instance contains arbitrary remote execution of code, session fixation and hijacking, XSS injection, SQL injection, and weak password hashing that allows trivial reconstruction of passwords from said SQL injection. Whatever your reasons are for 4.2 being "not an option," you should at _very_ least upgrade to 3.8.17, which resolves the worst of those. It will still, of course, be unsupported, and vulnerable to other vulnerabilities (including CVE-2014-9472, a denial-of-service via RT's email gateway, as well as CVE-2015-1165 and CVE-2015-1464, which allow for information disclosure and session hijacking via RT's RSS feeds) but will be slightly less exploitable. Please upgrade. - Alex
Re: [rt-users] Pg FTS query works, RT search returns 0 results ( was: '.' as delimiter/boundary breaks domain name searches)
On Mon, Aug 31, 2015 at 09:39:19AM -0400, Jeff Blaine wrote:
> I'm reviving this one time in case anyone has further ideas.
>
> * PostgreSQL 8.4.20 (RHEL 6.6) with FTS does the right
> thing when parsing an email address[1]
>
> * An RT 4.2.12 search for the same string returns 0 results[2].
> I'm 98% certain the ticket *is* indexed though as other queries
> return it[2].
>
> * The rt-fulltext-indexer script runs every 10 minutes and has
> no errors.
Is the ticket in question a merged ticket? You had a mail from the
same timeframe which dealt with merged ticket FTS, which is a known
bug: https://issues.bestpractical.com/Ticket/Display.html?id=9370
Sadly, there's not good solution to that bug at this time.
Otherwise, seeing the query RT is generating, and stripping clauses
out of it until it matches, is likely your best bet. You can see the
SQL generated from a TicketSQL query by running:
perl -MRT=-init -le
'$t=RT::Tickets->new(RT->SystemUser);$t->FromSQL("@ARGV");print
$t->BuildSelectQuery' \
"Content LIKE 'f...@domain.com'"
- Alex
Re: [rt-users] can't create ticket when "On Create Autoreply To Requestors" scrip is enabled
On Wed, 15 Jul 2015 11:46:08 -0700 Roman Massey wrote: > On a fairly fresh RT 4.2.9 install, having a weird problem. Unable to > create tickets, when creating through web interface it sits for about > a minute then gets an error. When creating through email, it gets > same error and fires a confirmation email every fetch mail. > > If I disable the scrip “On Create Autoreply to Requestors” then > everything works fine (other than the requestors don’t get auto > reply, which i require) Do other mail-sending scrips work as expected? Can you show your RT_SiteConfig.pm? What are you using as your local MTA? - Alex
Re: [rt-users] All Incoming SMIME Signed Messages Showing as No Trust
On Thu, 16 Jul 2015 15:52:34 -0400 "Zoey Schutt" wrote: > OS and RT4 Info: > > Debian GNU/Linux 7 (wheezy) > Apache/2.2.22 (Debian) > PHP 5.5.26-1~dotdeb+7.4 > Request Tracker 4.2.11 As a note, RT is written in Perl, not PHP. The other useful version to know is the version of openssl, which you can find by running: openssl version > I am attempting to configure S/MIME support in my RT4 instance, and I > have every piece working other than the verification of signatures on > incoming email. What software is generating your certificates, and sending the incoming mail? I suspect your certificates are weird in a way that is throwing openssl off. Can you send me a simple S/MIME signed message and your CA's PEM file, off-list, so I can inspect it? - Alex
Re: [rt-users] Question about pgp headers using GnuPG Inline option
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 14 Jul 2015 22:51:55 -0500 Mario Góngora wrote: > I am trying to implement GnuPG on my RT (version 4.2.10) with the > Inline option, but when I send a message people are receiving this > type of wrapping : Can you show your %Crypt, %GnuPG and %GnuPGOptions settings? What version of GPG does `gpg --version` say you're running? - Alex -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlWl4ZEACgkQMflWJZZAbqBi1gCdHYg/6/ZAbfv2a1DyCDggmJYY MPYAn1Hz3vr0JA4LUhJ4m2ckFC3QSXc2 =/E5T -END PGP SIGNATURE-
Re: [rt-users] Priority value
On Tue, 14 Jul 2015 20:57:19 -0400 wrote: > Hi, > is there a way i can change the priority field in a ticket to a pull > down menu with 4 options like : > 100 - urgent 75 - high, and so on? > > Or use a custom field which the user selects, when the ticket is > created update the priorty numeric value? > > Thanks in advanced. Try https://metacpan.org/pod/RT::Extension::PriorityAsString - Alex
Re: [rt-users] Notification sent to requestor even when squelched
On Mon, 22 Jun 2015 19:10:50 +0200 Marcos Orallo wrote: > I manage an RT 4.2.11 deployment using Debian Wheezy, Apache 2.2, > mod_fcgid > > Today we had a very concerning issue regarding a notification being > sent to a requestor even when it had been explicitly unchecked in the > recipients list. At first I didn't believe the user, but then I found > this in the log: > [snip] There's a 47-second interval between the first and the second of those. Can you show the full log of everything between them? My guess is that the first one was a "dry run" to list the potential recipients, and the second was the actual submission. This would explain the different process IDs, as well as the different behaviors. > You can see that the requestor address was unchecked, but he was > notified anyway. The notificatoin script uses a custom condition to > just send the notification when using the web interface, with the > possibility of squelching recipients manually. The default behaviour > (implicit notification to all requestors even for correspondence by > mail) was confusing for my team mates. Your wording is confusing. Is this jut the stock "Notify Requestor and Ccs" action, with a custom condition? - Alex
Re: [rt-users] Full-Text Search: RT-4.2.11 MySQL 5.6.25 vs MariaDB 10.0.19+Sphinx
On Wed, 10 Jun 2015 19:47:27 + Paul Hirose wrote: > In short, trying to find the best mechanism for FTS, whether I can > use MySQL 5.6.25 InnoDB, or MariaDB 10.0.19 + Sphinx on RHEL 7. My answers below assume RT 4.2.11 -- there were some order-of-magnitude speed improvements to the indexer since 4.2.10. I'd recommend pure MySQL FTS (either InnoDB or MyISAM), and not Sphinx, as there are a large number of caveats with the Sphinx search (no-result responses and false negatives, primarily). Interestingly the performance of InnoDB FTS is slower than MyISAM's: https://chmrr.net/fts-charts/query.html > I'd prefer to use MySQL, since that seems to support FTS by itself. > Otherwise, I opt for MariaDB, simply because that has the Sphinx > engine built-in to the RPM I can get (saving me the hassle of > build-from-source) although I'd still have to separately maintain the > actual Sphinx/searchd daemon package. > > Also, when running rt-fulltext-indexer for MySQL the notes say it > does 200 tickets at a time. I take it the script remembers the last > ticket it did, and continues where it left off? Does running this > place any locks on any tables/rows, or can it be run at any time? It indexes in batches, based on the high-water-mark of the last successfully index attachment. It does not use database-level locking, instead using taking a lock on the indexer script itself; there is no impact on availability of RT for other operations. It can be canceled and resumed at will. > And to be clear, for Sphinx, the rt-setup-fulltext-indexer & indexer > rt is a one-time thing, and subsequent is simply indexer rt > --rotate? And similarly, running indexer rt --rotate can be done at > any time even while RT is being used? Yes -- however, `indexer rt` re-indexes _all_ attachments, since Sphinx does not support incremental indexes. As such, having close-to-live FTS indexes with Sphinx requires either a small database, or a main + delta technique -- see http://www.sphinxconsultant.com/sphinx-search-delta-indexing/#delta-2 for an example of such. > Oh, I do intend to "DontSearchFileATtachments = true", although we > might try it undef for a while, and see. As a note, DontSearchFileAttachments is respected on search, not on index, so it won't impact indexing time. - Alex
Re: [rt-users] Configuring fetchmail to poll emails to respective queues.
On Thu, 4 Jun 2015 12:31:57 + Bob Shaker wrote: > You don’t need 2 separate emails for those fields. Our instance works > fine with the same email in both fields. This isn't a good idea, though it does _work_. The problem is when someone records a comment, and it gets sent out just to your company -- but "From" the one address you have set. When some well-meaning person replies to that mail, it will come into the address RT uses for correspondence -- and then the end-user will get the information that you intended to be a comment. This is at best confusing, and at worst horribly embarrassing. - Alex
Re: [rt-users] Autocomplete vs. Drop-down for 'Owner'
On Thu, 04 Jun 2015 15:20:09 -0400 Jeff Blaine wrote: > RT 4.2.11 > > I'm failing to find where in the code this "50 privileged users = use > autocompleter" logic is. You can adjust the limit via the ModifyDropdownLimit callback in share/html/Elements/SelectOwnerDropdown - Alex
Re: [rt-users] [SOLVED] Re: charset troubles
On Thu, 14 May 2015 16:50:23 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: > I fixed it. Replying to myself with the whole story for someone else's > future reference. Good to hear the full debugging story. > The problem was database encoding. RT can use PostgreSQL with encoding > "UTF-8" or the default "SQL_ASCII". That later encoding means PostgreSQL > does not care about encoding and just gives back the bytes it was given > without any check. The former enforces UTF-8 usage and is able to > automatically transcode if the client claims to use another encoding. > > My RT installation had been configured with the PostgreSQL database > using "UTF-8" encoding for a while. At some time I upgraded PostgreSQL > and I reloaded the data from a dump after reinitializing the database. > But since I did not check for it, it got "SQL_ASCII", a setup where the > application must take care of data encoding. So this is the first place where things went awry. How was the database created to reload the database dump, such that it got SQL_ASCII? By hand using 'createdb' from the command line? And is your template0 database marked as 'SQL_ASCII' ? For reference, https://docs.bestpractical.com/backups#Restoring-from-backups1 is the documented technique for loading in a Pg backup. - Alex
Re: [rt-users] ticket content search maybe doesn't work as excepted
On Tue, 12 May 2015 17:11:59 +0200 Christian Loos wrote: > playing around with the ticket content search I discovered many "false > positive" results. Yup -- known bug: https://issues.bestpractical.com/Ticket/Display.html?id=19237 - Alex
Re: [rt-users] Upgrading web/email server...should I upgrade RT itself too?
On Fri, 8 May 2015 14:11:08 + "Beachey, Kendric" wrote: > My question...is RT 4.0.17 itself old enough that I really ought to upgrade > it as well? Yes. 4.0.17 has published security vulnerabilities against it (CVE-2015-1464, CVE-2015-1165, CVE-2014-9472): http://blog.bestpractical.com/2015/02/security-vulnerabilities-in-rt.html > I'd like to minimize the amount of surprise for the users via new > looks, so I'm wondering if there are any huge problems with staying > at 4.0.17. Upgrading within a stable series will never cause any major user-visible UI changes, and should never break installed extensions. Upgrades within a stable series (from 4.0.17 to 4.0.23, for instance) are designed to be no-hassle bugfixes and security fixes. If we feel a change has the potential to give an administrator reason to _not_ upgrade within a stable series, it is unsuitable for that trunk. Please upgrade. All of this is hopefully also made clear on release policy page: https://bestpractical.com/rt/release-policy.html - Alex
[rt-users] [rt-announce] Assets 1.05 released
Assets 1.05 -- 2015-05-06 - We are pleased to announce Assets 1.05. https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.05.tar.gz https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.05.tar.gz.asc SHA1 sums 53b472ef543c99f3b1daae9e2e293c33dcd8cad8 RT-Extension-Assets-1.05.tar.gz 2e3a32e97ba720c3fd9f58d587424e05aec30e94 RT-Extension-Assets-1.05.tar.gz.asc Changes from 1.04: * Only call FillCache once if enabling plugin during "make initdb" * Ensure that grouping-less custom fields keep their values A complete changelog is available from git by running: git log 1.04..1.05 or visiting https://github.com/bestpractical/rt-extension-assets/compare/1.04...1.05 ___ rt-announce mailing list rt-annou...@lists.bestpractical.com http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce
Re: [rt-users] charset troubles
On Fri, 1 May 2015 10:01:02 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: > In database, I have in both cases Which database? In your original mail, you said: > p5-DBD-mysql-4.031 > [...] > postgresql-9.4.1 Which database are you using? - Alex
Re: [rt-users] charset troubles
On Thu, 30 Apr 2015 21:08:00 +0200 m...@netbsd.org (Emmanuel Dreyfus) wrote: > I just upgraded Apache to 2.4 and RT to latest 3.8 RT 3.8 reached end-of-life over a year ago. No release of Apache 2.4 had been made before RT 3.8 was in "critical security releases only." I'm unsurprised that there are incompatibilities between the two. Please upgrade to a supported version of RT. The unmaintained 3.8 series also now has disclosed security vulnerabilities against it. - Alex
Re: [rt-users] Including CSS on template - RT
On Wed, 29 Apr 2015 17:41:34 -0300 Murillo Azambuja Gonçalves
wrote:
> Hello,
Please do not cross-post your question to both rt-users and rt-devel.
rt-users is the appropriate venue for this question.
> [snip]
> But the template accuses an error when I use CSS:
This is because templates use { and } to delimit code that should be
run as perl. As such, you'll need to escape your brace characters;
either use:
a \{
text-decoration: none;
color: inherit;
\}
...or:
a {q{{
text-decoration: none;
color: inherit;
}}}
- Alex
Re: [rt-users] RT and SMTP
On Wed, 29 Apr 2015 15:14:40 +0100 ABD EL MALEK BOUBARNOUS wrote: > How can I configue RT to use an SMTP server instead of sendmail ? RT 4.2 removed the ability to speak directly to an SMTP server -- as operating in such a configuration meant that outgoing mail was lost if there was ever connectivity problems to the SMTP host. A local MTA is meant to solve precisely that problem, which is why RT relies on it. In short, that's no longer a supported configuration, to stop users from shooting themselves in the foot. Taking a step back, you really should look at your MTA's configuration and error logs. All of the workarounds and questions you're asking are about RT -- but RT is not actually the cause of your problem. Solve your actual problem, and the symptoms will go away. - Alex
Re: [rt-users] slowness in RT when creating a ticket
On Tue, 28 Apr 2015 16:43:03 +0100 ABD EL MALEK BOUBARNOUS wrote: > in my configuration $MailCommand is set to 'sendmailpipe', on the other > hand CommentAddress, CorrespondAddress and OwnerEmail are all set to my > personal gmail adress, can this be the source of the slowness ? Your source of slowness is that your MTA is being slow to accept the mail. Usually this is because of DNS failure, or that your service provider does not allow outgoing connections on port 25. Check your maillogs. If you're merely trying to use RT for local testing, use 'testfile'. - Alex
Re: [rt-users] slowness in RT when creating a ticket
On Tue, 28 Apr 2015 15:20:44 +0100 ABD EL MALEK BOUBARNOUS wrote: > Hi everyone, Please stop cross-posting. Your questions are best suited for rt-users, not rt-devel. > I've just deployed RT 4.2 on Apache2 in my localhost machine, I also > configured it to send mails with sendmail mail transfer agent, since then > I'm experiencing slowness when creating tickets,(it takes about a minute). > > Does anyone know why is that due to ? is it sendmail ? It is almost certainly due to your mail configuration. Most folks doing local testing set in their RT_SiteConfig.pm: Set( $MailCommand, 'testfile' ); ...which logs mail to a file rather than attempting to send it. - Alex
Re: [rt-users] rt 2.4.10
On Wed, 22 Apr 2015 23:22:56 +0200 Nick price wrote: > When will 4.2.11 be released We hope to get a release candidate out the door this week. To be clear, however, 4.2.11 is not necessary to run on Apache 2.4 -- the documentation is the only piece that was updated in that regard. The configuration instructions I linked to earlier will work just fine on any version of 4.2. - Alex
Re: [rt-users] rt 2.4.10
On Wed, 22 Apr 2015 18:12:27 + Nick Price wrote: > I have been told that the documentation shipped with 4.2.10 isn't OK to > use with apache 2.4 The upcoming 4.2.11 documentation has been extended to provide a configuration that works on Apache 2.0, 2.2, and 2.4. Until the documentation on https://docs.bestpractical.com/ is updated with the information from the release, you can read the documentation directly from git: https://github.com/bestpractical/rt/blob/stable/docs/web_deployment.pod#apache - Alex
Re: [rt-users] Theme editor in 4.2.10 says GD is disabled or not installed.
On Fri, 10 Apr 2015 12:25:02 -0500 "k...@rice.edu" wrote: > Okay, I just checked and once I upload a supported logo image type the error > goes away. Since it was there before doing anything, it threw me off and I > thought that there was a problem. Thank you for the feedback. This is fixed in 4.2-trunk and will be in 4.2.11, for reference: https://github.com/bestpractical/rt/commit/c51612a1 https://issues.bestpractical.com/Ticket/Display.html?id=30563 - Alex
Re: [rt-users] RT::Extension::Announce post install question
On Wed, 8 Apr 2015 15:36:31 + "Karres, Dean" wrote: > Hi, > > I am running RT 4.2.10. I just used cpan to install > RT::Extension::Announce and I looked at the resulting install messages. Installing using the cpan or cpanm command-line clients will never run the "make initdb" step. The new documentation (to be included in 4.2.11, the first rc of which should be out within a week) for installing plugins calls this out: https://github.com/bestpractical/rt/blob/stable/docs/extensions.pod#installing-extensions - Alex
Re: [rt-users] RT 4.2 DB access through DB session pooler?
On Thu, 2 Apr 2015 09:30:13 -0500 "k...@rice.edu" wrote: > We are working on an upgrade from RT 3.8.x to RT 4.2.x to get to a > supported version of RT and take advantage of the new features and > enhancements. In order to manage the PostgreSQL connections, we are > using pgbouncer. Pgbouncer has several types of pooling that can be > used: session, transaction and statement. The 'transaction' method > re-uses existing DB connections on a transaction (BEGIN/COMMIT) > boundary, while 'session' uses the same DB connection for each > fastCGI RT instance. > > With RT 3.8.x we initially tried to use 'transaction' pooling, but > ended up needing to move to 'session' pooling because updates were > ending up in the wrong RT ticket. Is 'transaction' pooling not > usable? It would really help our multi-instance support if we could > use it instead of 'session'. If it can be used, is there something > that needs to be done configuration-wise to have it work. Thank you > for any information that you can give. RT doesn't make sufficient use of database transactions that I think "transaction" will be reliable; I expect "session" is the best you can get. I'm not aware of any configuration knobs that would affect this; fixing it would take some internals development. - Alex
Re: [rt-users] HTML -> text conversion
On Tue, 31 Mar 2015 09:31:38 +0300 Vlad Shpolyanskiy wrote: > [warning]: Could not find or run external 'w3m' HTML formatter in $PATH > (/sbin:/bin:/usr/sbin:/usr/bin) -- you may need to install it or provide > the full path (/usr/local/lib/perl5/site_perl/RT/Interface/Email.pm:1849) This means that your system does not have the binary installed. You will need to "yum install w3m" or "apt-get install w3m" so that your system has it. Having the perl module that knows how to talk to w3m does no good if w3m itself is not installed. If you have suggestions for how the wording of the error message quoted above could be any clearer, I'd appreciate feedback. > Exactly, you are right. But RT falls back to internal converter as it does > not see w3m installed for ever reason. If the w3m binary is already installed, but not in /sbin, /bin, /usr/sbin, or /usr/bin (the $PATH listed above), then you'll need to set $HTMLFormatter to the full path to the w3m binary itself -- not the .pm file. - Alex
Re: [rt-users] HTML -> text conversion
On Mon, 30 Mar 2015 13:18:03 +0300 "Vlad V. Shpolyanskiy" wrote: > But while adding to RT_siteconfig string > Set($HTMLFormatter, '/usr/local/lib/perl5/site_perl/HTML/FormatText/W3m.pm'); If you want to use w3m, all you need is: Set($HTMLFormatter, 'w3m'); The reference to setting a full path is if the w3m _binary_ is not in your PATH. Setting it to the .pm file is incorrect. Also note that since w3m is the first program that RT will attempt to use, there is generally no need to explicitly set $HTMLFormatter to w3m _at _all_ unless you wish to use a binary in an unusual location. > The main idea is to avoid double newlines for outlook emails. > I'll appreciate any advice regarding this. You may be interested in https://docs.bestpractical.com/RT_Config#CheckMoreMSMailHeaders - Alex
Re: [rt-users] Mute notify requestor on resolve? (almost got it!)
On Wed, 25 Mar 2015 10:11:14 -0700 Roman Massey wrote: > Hello, > [snip] > Any tips will be greatly appreciated. Thanks! Move the Notify scrip to being in stage Batch. This will mean it won't appear in the preview of outgoing notifications on the ticket Reply page, but will have access to the complete, updated, ticket when it runs. - Alex
Re: [rt-users] Change transaction creator
On Wed, 25 Mar 2015 03:36:38 -0700 (MST) andriusk wrote: > I want to change the creator of the transaction, if it is done lets say via > REST, in other words if creator is REST_USER change it to USER_1. > > Is that possible? I guess that the Creator is set in RT::Record->Create > method? Stock RT does not support doing things on behalf of other users, via the REST interface or standard web UI. Doing the create via the perl API is the only way to do so. - Alex
Re: [rt-users] Search Query Builder Issues (Space Encoding?)
On Thu, 19 Mar 2015 11:20:51 -0400 Richard Stevens wrote: > Hello, > > We recently upgrades to RT 4.2.10 / RT-IR-3.2.0 and I'm seeing intermittent > issues on search commands involving custom fields with spaces: > [snip] The spaces are a red herring. The actual difference is that the working one includes &Queue=Incident+Reports. Where is the latter query being generated from? - Alex
Re: [rt-users] HTML emails from CLI
On Wed, 18 Mar 2015 10:44:43 + "Eierschmalz, Bernhard" wrote: > Is there any way to create Tickets with HTML email from CLI? rt create -t ticket -ct text/html - Alex
Re: [rt-users] RT 4.x multi-instances
On Sat, 14 Mar 2015 13:30:54 -0400 Gaston Huot wrote: > Hello. > > Is there something new about running 2 RT-4.x instances on the same > installation ? > Most comments are quite outdated, including the wiki. > > Someone doing it using version 4.x ? Just install into two directories and use one of the fastcgi solutions; mod_perl does not support multiple instances. - Alex
Re: [rt-users] Issues with Article Permissions
On Thu, 5 Mar 2015 12:06:55 +1100 "Matthew Crozier" wrote: > I'm having an issue when attempting to create a new article as the > root (SuperUser rights) account. It simply says "Permission denied" Check your error logs. Running rt-validator --check might also be a good idea. - Alex
Re: [rt-users] script to check out if max attachment size is reached
On Fri, 06 Mar 2015 12:42:38 +0100 Loïc Cadoret wrote: > Greetings rt-users followers, > > I would like to create a script that check out if the max attachment > size of a response is reached and if so, send an autoreply to the > requestor informing him that his documents are too heavy. Indeed, > customers send documents to our customer services but sometimes the > transaction is droped by the system because max attachment is reached > and neither the customer service or the client is notified of it. Starting in RT 4.2.7, RT records explicit transactions when attachments are dropped or truncated due to hitting $MaxAttachmentSize. While we explicitly prevent such transactions from triggering scrips (to prevent infinite loops), you can likely use them as the basis of your notifications -- or, at least the signal to the RT users to notify customers of the drop. - Alex
Re: [rt-users] relative date search
On Fri, 06 Mar 2015 06:48:53 + Alex Peters wrote: > You can enter "14 days ago" into the Created box, or > > Created < "14 days ago" > > into the Advanced screen of the search builder. Note that despite how it reads in English, 'Created < "14 days ago"' means tickets that were created _more_ than 14 days ago. That is, it is read as "Take the point in time 14 days ago; find tickets whose creation date is less than (prior to) that point." The connotation of the word "ago" in combination with less than / greater than is often a point of confusion. - Alex
Re: [rt-users] RT saves data in quoted-printable, why???
On Fri, 6 Mar 2015 00:06:32 +0100 Václav Ovsík wrote: > https://issues.bestpractical.com/Ticket/Display.html?id=29735 Aha -- thanks for digging that out! I thought I vaguely recalled something in this area previously. https://issues.bestpractical.com/Ticket/Attachment/286095/157750/utf8-encoding.patch looks to be functionally fairly similar to the branch. There are a few other, orthogonal fixes in there that may still be interesting to tease out into their own commits. It looks like I see changes to: * Fix the computed max size of base64'd attachments; I'd need to squint at it harder, but seems eminently reasonable. * Attempt to gracefully deal with TruncateLongAttachments truncating mid-byte of UTF-8 data. As above; the decode/encode is an interesting trick to attempt to ensure that the byte stream is consistent. I'd like to test it a bit, but seems not unreasonable. * Choose base64 vs QP based on which is shorter; I'm less convinced by this, since it means that for large data, it gets QP'd, base64'd, and then one of those _again_ -- which isn't terribly efficient. I'm less convinced by the tradeoff of computation time to stored in-database size. If you're interested in reworking the patch into a 2-3 commit series, I'm happy to apply for 4.2-trunk. - Alex
Re: [rt-users] RT saves data in quoted-printable, why???
On Thu, 5 Mar 2015 14:31:56 -0500 Alex Vandiver
wrote:
> I _believe_ that this code may be replaced by (untested):
>
> } elsif (!$RT::Handle->BinarySafeBLOBs
> && !eval { Encode::decode( "UTF-8", $Body,
> Encode::FB_CROAK); 1 } ) {
> $ContentEncoding = 'quoted-printable';
> }
>
> I may push a branch later that makes this change, and see what breaks.
https://github.com/bestpractical/rt/compare/4.2-trunk...4.2%2Fless-qp
- Alex
Re: [rt-users] Upgrade to 4.2.9 Lost Dashboards.
On Thu, 5 Mar 2015 14:52:39 -0500 Alex Vandiver wrote: > On Wed, 25 Feb 2015 11:52:37 + Sam Maher > wrote: > > I have moved to a new server and upgraded to the latest version but > > all my privileged users except one have lost their dashboards. Has > > anyone else had this problem? > > Please specify what you mean by "latest version". I missed that the subject specified you upgraded to 4.2.9. What version were you upgrading from? - Alex
Re: [rt-users] sphinx weirdness [explanation and possible solution]
On Wed, 25 Feb 2015 13:54:45 +0100 Arkadiusz Miśkiewicz wrote: > > Ok, mysql is too smart! For sphinx to work mysql needs to first > > query AttachmentsIndex_3 and then make joins to it. Otherwise > > sphinx won't work. That's due to sphinx architecture. > > > > Here mysql is too smart and differently optimizes query thus > > breaking sphinx support in rt. > > > > Now I've tried FORCE INDEX and such but wasn't able to force mysql > > to first query AttachmentsIndex_3. > > > > There is STRAIGHT_JOIN that forces joins orders, so maybe that is > > some solution. This limitation is unfortunately documented, and not easily fixable: https://bestpractical.com/docs/rt/latest/full_text_indexing.html#Caveats1 > STRAIGHT_JOIN also won't work since mysql still is able to make > changes and optimizations to the query. > > Fortunately code below seems to be working - using UNION and separate, > simple sphinx subquery: > [snip] That query is incorrect; it unions Attachment ids (from the Sphinx results table) with Ticket ids (from the "ti" subquery on Tickets). > Devs, could you please change querying code, so that sphinx will > always get its own subquery? Please try the straight MySQL FTS, included in 4.2.10, instead. It is much faster, and not nearly as fiddly. - Alex
Re: [rt-users] Upgrade to 4.2.9 Lost Dashboards.
On Wed, 25 Feb 2015 11:52:37 + Sam Maher wrote: > I have moved to a new server and upgraded to the latest version but > all my privileged users except one have lost their dashboards. Has > anyone else had this problem? Please specify what you mean by "latest version". Are the dashboards listed in Home → All Dashboards? - Alex
Re: [rt-users] RT saves data in quoted-printable, why???
On Thu, 5 Mar 2015 18:50:57 +0100 Palle Girgensohn
> * Ensure that all data containing non-ASCII is quoted-printable
> encoded for PostgreSQL, instead of merely all data not claiming to be
>"text/plain"
>
> Why is this? It seems it is doing more harm than good -- it makes it
> harder to use indexes and makes it harder to use it from the tables
> directly? PostgreSQL is very capable of storing any kind of character
> set? We use UTF-8 for everything, this fix breaks a lot of things for
> us.
The commit in question is 3a9c38ed, which changes:
} elsif (!$RT::Handle->BinarySafeBLOBs
&& $MIMEType !~ /text\/plain/gi
&& !Encode::is_utf8( $Body, 1 ) ) {
$ContentEncoding = 'quoted-printable';
}
...to:
} elsif (!$RT::Handle->BinarySafeBLOBs
&& $Body =~ /\P{ASCII}/
&& !Encode::is_utf8( $Body, 1 ) ) {
$ContentEncoding = 'quoted-printable';
}
That is, any data which claimed to be "text/plain" would blindly be
attempted to be shoved into the database; this includes content which
contained _invalid_ UTF-8 byte sequences. The commit was in RT 4.0; RT
4.2 is much better about transcoding to real UTF-8, or marking the part
as "application/octet-stream" if it cannot be decoded. In that sense,
this logic is now less necessary. However, the commit was absolutely
necessary at the time to not _lose_ data. Erring on the side of data
preservation, at the expense of possibly-unnecessary encoding, seems
like not an unreasonable choice.
That being said, the Encode::is_utf8() check attempts to verify that we
only quoted-printable encode data whose bytes are not currently a
correctly-encoded UTF-8 byte stream. The bug now lies there, as after
the "utf8-reckoning" branch (2620658..af9fe7c), the $Body argument is
guaranteed to contain bytes, not characters. Per the Encode::is_utf8()
documentation:
[INTERNAL] Tests whether the UTF8 flag is turned on in the STRING. If
CHECK is true, also checks whether STRING contains well-formed UTF-8.
Returns true if successful, false otherwise.
The UTF8 flag is almost certainly off on $Body (it _may_ be on, and
still contain only codepoints < 128, but this is unlikely), so the
well-formed-ness of the string (which is the relevant thing we wish to
confirm) is never checked.
I _believe_ that this code may be replaced by (untested):
} elsif (!$RT::Handle->BinarySafeBLOBs
&& !eval { Encode::decode( "UTF-8", $Body, Encode::FB_CROAK); 1 }
) {
$ContentEncoding = 'quoted-printable';
}
I may push a branch later that makes this change, and see what breaks.
All of that aside, note that "it makes it harder to use indexes" makes
little sense -- there are no indexes on Content. The full-text index
uses its own tsvector, which it constructs after decoding the Content,
so the transfer encoding of the Content column is irrelevant to that.
- Alex
[rt-users] [rt-announce] RT-Extension-Assets 1.04
Assets 1.04 -- 2015-03-02 - We are pleased to announce Assets 1.04: https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.04.tar.gz https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.04.tar.gz.asc SHA1 sums c933c7e8f9779a711021599ab38d3b011d655180 RT-Extension-Assets-1.04.tar.gz 08b6642151c32d172f854f2dd34d508a9b8b69a5 RT-Extension-Assets-1.04.tar.gz.asc Changes from 1.02: * Keep values of date CFs if asset creation fails (due to CF values that fail validation, or other reasons) * Speed up asset asset searches with multiple limits on the same role; this primarily effects installs with the RT-Extension-MergeUsers extension installed. A complete changelog is available from git by running: git log 1.02..1.04 or visiting https://github.com/bestpractical/rt-extension-assets/compare/1.02...1.04 ___ rt-announce mailing list rt-annou...@lists.bestpractical.com http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce
Re: [rt-users] Asset for RT 1.0.2 and rt-crontool
On Mon, 2 Mar 2015 21:33:12 +0300 "Andrey K" wrote: > So, is it impossible to act on assets like tickets from rt-crontool? As of now, yes. rt-crontool is still ticket-centric, in that its search predicates only return tickets, and its actions are still Scrips' actions, which assume they act on tickets. Writing a small amount of code to iterate over assets and modify them is fairly simple, however. - Alex
Re: [rt-users] Asset for RT 1.0.2 and rt-crontool
On Mon, 2 Mar 2015 19:46:36 +0300 "Andrey K" wrote: > Is it possible to use rt-crontool to change a status of an asset depend on > it's custom field? > I'd like to create "Expiration data" custom field to change status of assets > if data expired. Assets doesn't have support for triggered actions (scrips) yet. You can override the AddCustomFieldValue method on RT::Asset to accomplish this manually. - Alex
Re: [rt-users] RT 3.8.8 goes down intermittently
On Mon, 2 Mar 2015 14:14:20 + "Calafiore, Jason [USA]" wrote: > I am experiencing some issue when RT 3.8.8 RT 3.8 reached end-of-life nearly a year ago: http://blog.bestpractical.com/2014/04/rt-38-reaches-end-of-life.html RT 3.8.8 was released nearly 5 years ago, and has multiple published security vulnerabilities against it, including SQL injection and arbitrary execution of code. Upgrade -- which will also likely fix the bug you're seeing. - Alex
[rt-users] [rt-announce] RT 4.0.23 released
RT 4.0.23 -- 2015-02-26 --- RT 4.0.23 contains important security fixes, as well as minor bugfixes. https://download.bestpractical.com/pub/rt/release/rt-4.0.23.tar.gz https://download.bestpractical.com/pub/rt/release/rt-4.0.23.tar.gz.sig SHA1 sums 1067e0469184a6955e2822967eb7a2e287f7c17b rt-4.0.23.tar.gz 17a153215b95d12e5aa0500d676089aed081d6df rt-4.0.23.tar.gz.sig This release is primarily a security release; it addresses CVE-014-9472, a denial-of-service via RT's email gateway, as well as CVE-2015-1165 and CVE-2015-1464, which allow for information disclosure and session hijacking via RT's RSS feeds. As part of these security updates, RT's dependency on the Encode module has been changed, to Encode 2.64. If upgrading, be sure to run rt-test-dependencies to verify that your installed version of Encode meets this requirement; if not, you will need to install a newer version from CPAN. Other changes include: General user UI * Flush TSV download every 10 rows, for responsiveness * Pressing enter in user preference form fields no longer instead resets the auth token * Pressing enter in ticket create and modify form fields now creates or updates the ticket, instead being equivalent to "add more attachments", or the "search" on People pages. * Retain values in Quick Create on homepage if it fails Command-line * Fix server name displayed at password prompt when RT is deployed at a non-root path like /rt Admin * Empty email addresses are no longer caught as being "an RT address" if there exist queues without Correspond addresses set * Allow Parents/Children/Members/MemberOf in CreateTickets action * Allow RT-Originator to be overridden in templates * Add missing semicolon on Shredder suggested indexes * Ensure that HTML-encoded entities are indexed in FTS Developer * Make Obfuscate callback in configuration options be passed the current user, as was documented * Remove obsolete _CacheConfig parameters * ACL checks are now applied in the ->AddRecord stage, not in ->Next; this means that collections accessed via ->ItemsArrayRef are now correctly ACL'd. Documentation * New documentation on writing portlets * Add an =pod directive so the first paragraph of UPGRADING is not skipped * Clarify when UPGRADING-x.y steps should be run A complete changelog is available from git by running: git log rt-4.0.22..rt-4.0.23 or visiting https://github.com/bestpractical/rt/compare/rt-4.0.22...rt-4.0.23 ___ rt-announce mailing list rt-annou...@lists.bestpractical.com http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce
[rt-users] [rt-announce] RT 4.2.10 released
RT 4.2.10 -- 2015-02-26 --- RT 4.2.10 contains important security fixes, as well as minor bugfixes. https://download.bestpractical.com/pub/rt/release/rt-4.2.10.tar.gz https://download.bestpractical.com/pub/rt/release/rt-4.2.10.tar.gz.asc SHA1 sums 92af386e9c09a0e9489ec1cd55b66c65b77d22be rt-4.2.10.tar.gz 8e65ce02b62df85c7d679dab8d4bde8ef343ec48 rt-4.2.10.tar.gz.asc This release is primarily a security release; it addresses CVE-014-9472, a denial-of-service via RT's email gateway, as well as CVE-2015-1165 and CVE-2015-1464, which allow for information disclosure and session hijacking via RT's RSS feeds. As part of these security updates, RT's dependency on the Encode module has been changed, to Encode 2.64. If upgrading, be sure to run rt-test-dependencies to verify that your installed version of Encode meets this requirement; if not, you will need to install a newer version from CPAN. This release is also a bugfix release; most notably, it addresses a bug which causes RT to generate blank outgoing text/plain parts. This fix requires installing the HTML::FormatExternal module, and having an external tool (w3m, elinks, etc) installed on the server. It also introduces indexed full-text searching for MySQL without the need to recompile MySQL to use the external Sphinx tool; instead, a MyISAM table is used for indexing. On MySQL 5.6 and above, an additional InnoDB table can also be used. The complete list of changes includes: General user UI * Speed up the default simple search on all FTS-enabled installs by not OR'ing it with a Subject match. This returns equivalent results for almost all tickets, and allows the database to make full use of the FTS index. * Pressing enter in user preference form fields no longer instead resets the auth token (#19431) * Pressing enter in ticket create and modify form fields now creates or updates the ticket, instead being equivalent to "add more attachments", or the "search" on People pages (#19431) * Properly encode headers in forwarded emails that contain non-ASCII text (#29753) * Allow users to customize visibility of chart/table/TicketSQL in saved charts * Allow groups to be added as requestors on tickets * Perform group searches case-insensitively on People page (#27835) * Ticket create transactions for tickets created via the web UI now contain mocked-up From, To, and Date headers; this causes them to render more correctly when forwarded * Update wording of error message for saved searches without a description (#30435) * Flush TSV download every 10 rows, for responsiveness * Retain values in Quick Create on homepage if it fails (#19431) * Limit the custom field value autocomplete to 10 values, like other autocompletes (#30190) * Fix a regression in 4.0.20/4.2.4 which caused some users to have blank homepages (#30106) * Fix styling on "unread messages" box on Ballard and Web2 themes * Fix format of Date headers in RSS feeds (#29712) * Adjust width of transaction date to accommodate all date formats (#30176) * Allow searching for tickets by queue lifecycle Command-line * Fix server name displayed at password prompt when RT is deployed at a non-root path like /rt (#22708) Admin * If the optional HTML::FormatExternal module is installed, use w3m, elinks, links, html2text, or lynx to format HTML to text. This addresses problems with the pure-Perl HTML-to-text converted which resulted in blank outgoing emails. (#30176) * Add support for native (non-Sphinx) indexed full-text search on MySQL. This uses the InnoDB fulltext engine on MySQL 5.6, and an additional MyISAM table on prior versions of MySQL. * Support MySQL database names with dashes in them (#7568) * Properly escape quotes and backslashes in config options in web installer (#29990) * Increase length of template title form input * Clarify wording on updating old Organization values by rt-validator * Resolve a runtime error for SMIME without secret keys (#30436) * Empty email addresses are no longer caught as being "an RT address" if there exist queues without Correspond addresses set (#18380) * Allow Parents/Children/Members/MemberOf in CreateTickets action * Allow RT-Originator to be overridden in templates * Ensure that HTML-encoded entities are indexed in FTS * Fix uninitialized value warnings from charts grouped by date * Remove no-op $CanonicalizeOnCreate configuration variable; RT::User->CanonicalizeUserInfo is always called * Make NotifyGroup action respect AlwaysNotifyActor argument * Fix X-RT-Interface header on incoming email on existent tickets * Warn on startup if queues have invalid lifecycles set (#28352) Developer * Add AfterHeaders callback to ShowMessageHeaders * Update all upgrade steps to use .in files (#18856) * Add policy tests to enforce the new upgrade step standards * Remove +x bit from multiple non-executable files * Make Obfuscate callback in co
[rt-users] [rt-announce] Security vulnerabilities in RT
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 We have discovered security vulnerabilities which affect both RT 4.0.x and RT 4.2.x. We are releasing RT versions 4.0.23 and 4.2.10 to resolve these vulnerabilities, as well as patches which apply atop all released versions of 4.0 and 4.2. The vulnerabilities addressed by 4.0.23, 4.2.10, and the below patches include the following: RT 3.0.0 and above, if running on Perl 5.14.0 or higher, are vulnerable to a remote denial-of-service via the email gateway; any installation which accepts mail from untrusted sources is vulnerable, regardless of the permissions configuration inside RT. This denial-of-service may encompass both CPU and disk usage, depending on RT's logging configuration. This vulnerability is assigned CVE-2014-9472. RT 3.8.8 and above are vulnerable to an information disclosure attack which may reveal RSS feeds URLs, and thus ticket data; this vulnerability is assigned CVE-2015-1165. RSS feed URLs can also be leveraged to perform session hijacking, allowing a user with the URL to log in as the user that created the feed; this vulnerability is assigned CVE-2015-1464. We would like to thank Christian Loos for reporting CVE-2014-9472 and CVE-2015-1165; CVE-2015-1464 was found by internal review. Patches for all releases of 4.0.x and 4.2.x are available for download below. Versions of RT older than 4.0.0 are unsupported and do not receive security patches; please contact sa...@bestpractical.com if you need assistance with an older RT version. https://download.bestpractical.com/pub/rt/release/security-2015-02-26.tar.gz https://download.bestpractical.com/pub/rt/release/security-2015-02-26.tar.gz.asc aac58bf3aa6d918dbefbaa2b27a9694f27b32d58 security-2015-02-26.tar.gz 6abe9a58400db3ee2cdbdf17704f0d881d90d744 security-2015-02-26.tar.gz.asc The README in the tarball contains instructions for applying the patches. If you need help resolving this issue locally, we will provide discounted pricing for single-incident support; please contact us at sa...@bestpractical.com for more information. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlTvSZMACgkQMflWJZZAbqCj5gCgwmXReEL+TIUYrAzfTl0aj0rr +ZIAn2Uq8K12j3r+se6yZlg/B6myoJSM =kSeJ -END PGP SIGNATURE- ___ rt-announce mailing list rt-annou...@lists.bestpractical.com http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce
Re: [rt-users] RT 4.2.6 Need a scrip that will check email address and not send email multiple times
On Tue, 24 Feb 2015 14:42:23 + Daniel Moore wrote: > Example, I have a user that belongs to an exchange group that is currently > the email for a contractors group that I have set up. He receives multiple > emails when he is requestor, admin cc and so on. Use one scrip with a "Notify" action that notifies all of the folks you care about. This isn't the default because AdminCc's get different email than requestors. - Alex
Re: [rt-users] RT install, apache problem.
On Thu, 19 Feb 2015 18:06:05 -0500 "Joseph Mays" wrote: > I have an installation or RT4.2 under Apache 2.4 on FreeBSD 10.1. Sorry -- we've not updated the documentation for Apache 2.4 yet. Replace: Order allow,deny Allow from all ..with: Require all granted Or use the mod_access_compat module. See http://httpd.apache.org/docs/2.4/upgrading.html#access - Alex
Re: [rt-users] [SOLVED] Restoring from backup -> admins lost privileges
On Fri, 13 Feb 2015 18:41:45 + Andrew Wippler wrote: > After a few hours of sleep, I managed to get it working with this code: > [snip] For posterity, this is not the solution you want. If your users did not come across as Privileged, there are likely a huge slew of other hidden problems. Attempting to paper over it via this method is dangerous -- you just made every email address that has ever sent your RT spam into a Privileged user. Fix your database backups. - Alex
Re: [rt-users] Upgrade fail
On Fri, 6 Feb 2015 13:44:06 -0500 Joshua Lansford wrote: > Ok. So I have done some additional investigation and here is what I have > found. The new Queues table has two new fields named Lifecycle and > SubjectTag. Lifecycle field is accessed in the RT::Init which is called at > the top of action_insert before it can do the insert portion of the first > upgrade step... I think. It is looking like the code is assuming stuff > about the database structure which isn't true just in trying to set up to > do the upgrade steps. Please show the complete output of 'make upgrade-database', not just a small snippet. > I suppose this means I am trying to jump too many versions at once and I > need to do an intermediate upgrade at an intermediate step. Probably at > the version which introduces these columns. If that version couldn't have > upgraded everyone would have gotten stuck. It shouldn't be necessary to to the upgrade in multiple steps. - Alex
Re: [rt-users] changing ticket owner causes crash
On Thu, 5 Feb 2015 12:37:03 + Alexander Reintzsch wrote: > I have a problem, which seems to exist for quite some time in request > tracker. You didn't saw what version of RT you're running. > Here an old description: > http://www.gossamer-threads.com/lists/rt/users/41122 > > My Problem seems to be the same. > > The user that is the owner of the ticket had been deactivated. > Afterwards the ticket had to get a new owner. > I wanted to do this by changing the owner in the »People« menu of the > ticket. This failed. > > The log showed: > [8647] [Thu Feb 5 12:24:19 2015] [warning]: Couldn't delete cached > group submember 29891 (/opt/rt4/sbin/../lib/RT/GroupMember.pm:343) This is a symptom of database corruption. Did you modify your database by hand, or have disk problems at any point? Back up your database, then run rt-validator --check --resolve - Alex
Re: [rt-users] Change to SSL cert breaking incoming mail
On Wed, 4 Feb 2015 19:18:30 + asas wrote: > It is a trusted CA, and the cert for the CA is present on the server. > That's why I'm so baffled by the problem - the whole cert chain works > fine with the web interface. Have you tried passing the CA, and it didn't work, or did you note try yet? Perl may be working from a different certificate store than your web browser. Alternately, try upgrading Net::SSLeay and LWP::Protocol::https. - Alex
Re: [rt-users] Problem with attachments
On Wed, 4 Feb 2015 18:26:36 + Guadagnino Cristiano wrote: > Thank you Alex, > so basically my test is correct and my problem is not solved. > Back to the drawing board :-( > > Let's see if I can give some more detail, to help you in helping > me :-) Those two mails are sent by different scrips. Check that the template for the latter scrip has "RT-Send-Attachment: yes" in the headers section. - Alex
Re: [rt-users] Problem with attachments
On Wed, 4 Feb 2015 11:43:23 + Guadagnino Cristiano
wrote:
> I disabled CustomFieldsOnUpdate by commenting out the
> "Plugin('RT::Extension::CustomFieldsOnUpdate');" line in
> RT_SiteConfig.pm, then I stopped Apache, cleared the mason cache, and
> restarted Apache.
>
> I tested resolving a ticket adding an attachment, but had the same
> behavior as before.
>
> I then also disabled the "Announce" extension by following the same
> procedure, but still I had no success.
>
> Those are the only two extensions I added recently (other extensions
> are loaded but they have been there for years).
>
> Is it sufficient to disable the extensions by commenting them out in
> the RT_Siteconfig.pm file or should I completely uninstall them? If
> so, how do I do it?
Removing the Plugin line should mostly suffice; removing the files from
disk does no better. Fully removing a plugin requires backing out
database contents that relay on it, (like the RTAnnounce) queue, but
that's not relevant for this test.
- Alex
Re: [rt-users] New cert breaks mailgate
On Tue, 3 Feb 2015 16:27:02 -0500 Mitch Kyser wrote: > Thanks for the response. We tried that and could not get it to work > either. Turns out our CA is pretty old and still running on a 2003 > box. > > We were going to roll out RT to our staff first who all use domain > machines that include our root CA cert already. The web portion > worked fine. We were going to let our students eventually start > sending requests and planned to get a commercial cert. This just > pushed the time up a few months. So now we have a new project, > upgrade our CA. Try upgrading the LWP::Protocol::https and the Net::SSLeay modules. - alex
Re: [rt-users] rt-crontool EROR
On Tue, 03 Feb 2015 12:09:48 + "Mauricio Leite Ferreira da Silva" wrote: > Does anybody Know what can be happening? Upgrading to 4.2.7 or higher will resolve https://issues.bestpractical.com/Ticket/Display.html?id=22991 and allow you to see the actual error. Alternately, you can apply https://github.com/bestpractical/rt/commit/625e7b99.patch to your bin/rt-crontool - Alex
Re: [rt-users] Problem with attachments
On Tue, 3 Feb 2015 15:43:05 + Guadagnino Cristiano wrote: > I did this: I took a backup of my production RT and restored it on > our testing environment. I perused rt-validator till I had no more > warnings (well, I still have a few warnings related to articles: it > seems rt-validator cannot fix them). Then I created a test ticket and > resolved it attaching a file. > > As in our production system, the attachment did not get sent, and the > last line in RT's log was again this: > > > [15371] [Tue Feb 3 15:28:32 2015] [warning]: Couldn't load object > RT::Transaction #0 (/opt/rt4/sbin/../lib/RT/Interface/Web.pm:3026) > > Any hint? Try disabling the extensions you have installed and see if removing any of them fixes it. My suspicion is on CustomFieldsOnUpdate, from the previous list you gave. - Alex
Re: [rt-users] RT 4.2.8 reply emails go out blank
On Thu, 29 Jan 2015 15:01:07 -0500 Gilbert Rebeiro wrote: > Is there any work around for now? Switching to HTML templates is one solution. This will only mask the issue somewhat, however, as the plain-text alternative part will be blank, and the text/html will appear fine. > How would users know their messages went out blank? There's a warning in the logs. The end-users click on the "Show Outgoing Email" link, if it's available to them, to see what RT sent out. > Any idea on when 4.2.10 will be released? I expect a release candidate next week, with the official 4.2.10 a couple weeks thereafter. - Alex
Re: [rt-users] Problem with attachments
On Wed, 28 Jan 2015 11:53:53 + Guadagnino Cristiano wrote: > Unfortunately we send attachments very rarely, so I cannot exactly > say when this behaviour started. It could be that it started from the > day I upgraded our installation from 4.2.3 to 4.2.9 (in december), or > later when I added/upgraded some extensions > (RT-Extension-Announce-1.00, rt-extension-customfieldsonupdate, > RT-Extension-SpawnLinkedTicketInQueue-1.01). You may wish to try disabling those extensions temporarily to see if that helps resolve the issue. However, see below. > I just finished examining RT's log (in debug mode), regarding a test > ticket I created and then resolved adding an attachment. There is no > error at all in the log, let alone a warning message that says > > [warning]: Couldn't load object RT::Transaction #0 > (/opt/rt4/sbin/../lib/RT/Interface/Web.pm:3026) > > but I don't think this is a real problem, since we had have this type > of message for a long time with no apparent symptoms (please reply if > I'm wrong). That warning is extremely worrisome, and I expect the cause of your problem. You should run `/opt/rt4/sbin/rt-validator`. - Alex
Re: [rt-users] Cancel "Resolve" transaction based on custom fields
On Wed, 28 Jan 2015 14:27:22 +0100 Jasper Olbrich wrote: > Is it possible to suppress the status change from open to resolved > with a scrip or to include the scrip-induced status change in the > result box? Unfortunately, neither of these is possible in stock RT. Scrips can only run _after_ a chance has happened, and thus have no chance of preventing it. They also cannot currently push their results to the user. - Alex
Re: [rt-users] Connecting to remote MySQL via SSL
On Thu, 29 Jan 2015 10:59:27 +0100 Nathan Cutler wrote: > I have been tasked with migrating our RT database (MySQL) to a remote > server, connection between RT and the database server must be SSL > encrypted. > > I see at [1] that DBD::mysql supports SSL, but the RT documentation > [2] seems to imply that RT only supports SSL with PostgreSQL: > > $DatabaseRequireSSL > > If you're using PostgreSQL and have compiled in SSL support, set > $DatabaseRequireSSL to 1 to turn on SSL communication with the > database. From a quick skim of the DBIx::SearchBuilder::Handle code[1], that looks to add "requiressl=1" to the DSN -- which, even on Pg, doesn't do anything, since the current form is "sslmode=require"[1]. That option should die, and be replaced by a more generic one that allows one to append arbitrary things to the DSN -- particularly as "enabling SSL" isn't really an on/off bit, as one needs to talk about verification of the certificate in order for it to provide any security. > What is the status of out-of-the-box support for SSL with MySQL in RT > 4.2? Add a local overlay of RT::Handle::BuildDSN to add the relevant DSN attributes. A future release will provide rope to do this without an overlay. - Alex [1] https://github.com/bestpractical/dbix-searchbuilder/blob/master/lib/DBIx/SearchBuilder/Handle.pm#L152-L171 [2] https://metacpan.org/pod/DBD::Pg#connect > Thanks, > Nathan > > [1] https://metacpan.org/pod/DBD::mysql#mysql_ssl > [2] > https://www.bestpractical.com/docs/rt/4.2/RT_Config.html#DatabaseRequireSSL
