Re: [rt-users] RT::User::ExternalAuthId Unimplemented in RT::Record

2017-02-02 Thread Alex Vandiver
On Thu, 2 Feb 2017 19:59:47 +
Daniel Burchfield  wrote: 
> I am trying to get RT to pull in users from my local active directory
> and use AD for auth. Meaning when I change a user's password in AD it
> should reflect the change in RT. I'm running RT 4.4.1. Currently,
> when I run the import  I get the following error:
>
> Set($LDAPMapping, {
> Name=> 'sAMAccountName',
> EmailAddress=> 'mail',
> Organization=> 'department',
> RealName=> 'cn',
> NickName=> 'givenName',
> ExternalAuthId  => 'sAMAccountName',

This is the culprit line -- this column was removed in RT 4.4.  Remove
this line from your configuration, and it should resolve the issue.
 - Alex


Re: [rt-users] Problem with index

2017-01-11 Thread Alex Vandiver
On Wed, 11 Jan 2017 15:17:53 -0500
François Meehan  wrote:
> We are using RT 4.2.12. We noticed that the indexation stopped working.
> When trying to run the indexer manually, I would get:
> 
> [11238] [Wed Jan 11 19:53:56 2017] [warning]: DBD::mysql::st execute
> failed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer
> line 216. (/opt/rt4/sbin/rt-fulltext-indexer:216)
> [11238] [Wed Jan 11 19:53:56 2017] [warning]: DBD::mysql::st execute
> failed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer
> line 222. (/opt/rt4/sbin/rt-fulltext-indexer:222)
> [11238] [Wed Jan 11 19:53:56 2017] [critical]: Attachment 32267 cannot be
> indexed: MySQL server has gone away at /opt/rt4/sbin/rt-fulltext-indexer
> line 254. (/opt/rt4/sbin/../lib/RT.pm:389)
> Attachment 32267 cannot be indexed: MySQL server has gone away at
> /opt/rt4/sbin/rt-fulltext-indexer line 254.

This is likely due to a too-low `max_allowed_packet` in your MySQL
configuration.  Try bumping it higher and re-running the indexer.
 - Alex


Re: [rt-users] Queue still being CC'ed with $ParseNewMessageForTicketCcs and $RTAddressRegexp

2017-01-10 Thread Alex Vandiver
On Tue, 10 Jan 2017 20:26:05 +
"Cena, Stephen (ext. 300)"  wrote:
> Set($RTAddressRegexp, '(local-part1@domain1\.tld)|
> (local-part2@domain2\.tld)|
> (local-part3@domain3\.tld)|
> :
> :
> (local-partN@domainN\.tld)/xi');

If you want to use the /x modifier, provide a regular expression
object, not a string:

Set($RTAddressRegexp,
qr/
  (local-part1@domain1\.tld)|
  (local-part2@domain2\.tld)|
  (local-part3@domain3\.tld)|
  : 
  :
  (local-partN@domainN\.tld)
/xi );

Tangentially, the inner grouping ()s are not necessary here, but you
will want to anchor the regular expression to the start and end of what
it's matching. And you'll need to escape the "@" signs:

Set($RTAddressRegexp,
qr/
   ^
 (
   local-part1\@domain1\.tld
 | local-part2\@domain2\.tld
 | local-part3\@domain3\.tld
 |  : 
 |  :
 | local-partN\@domainN\.tld
 )
   $
/xi );


 - Alex


Re: [rt-users] Where to put crontool scripts?

2017-01-04 Thread Alex Vandiver
On Wed, 4 Jan 2017 11:13:38 -0500
Alex Hall  wrote:
> I'm considering putting them in /opt/rt4/etc, maybe in a "crontool-scripts"
> folder, but I don't know what RT upgrades might do to that.

RT upgrades won't remove any extra files you have lying around.
 - Alex


Re: [rt-users] RT 4.4.1 and transaction isolation level on Postgres

2017-01-04 Thread Alex Vandiver
On Tue, 3 Jan 2017 17:06:47 +0100
Václav Ovsík  wrote:
> How about the Mysql don't have this problem - is this caused by
> the different default transaction isolation level or not?

MySQL suffers from the exact same problem -- but, as it happens,
both more silently and more catastrophically.  See
https://github.com/bestpractical/rt/commit/e36364c5

> I can change isolation level in postgresql.conf to 'repeatable read'
> and things are different.

I advise against doing that.  Upon inspection, RT is not prepared to
deal with the "could not serialize access due to concurrent update"
errors that arise from updates to rows in multiple transactions in
Postgres' repeatable-read isolation.

Repeatable-read is only possible in MySQL because it has a fascinating
definition of "repeatable":

- Process 1 
mysql> set transaction isolation level repeatable read;
Query OK, 0 rows affected (0.00 sec)

mysql> start transaction with consistent snapshot;
Query OK, 0 rows affected (0.00 sec)

mysql> select id, Subject from Tickets where id = 1;
++-+
| id | Subject |
++-+
|  1 | foo |
++-+
1 row in set (0.00 sec)

- Process 2 

mysql> set transaction isolation level repeatable read;
Query OK, 0 rows affected (0.00 sec)

mysql> start transaction with consistent snapshot;
Query OK, 0 rows affected (0.00 sec)

mysql> update Tickets set Subject = 'bar' where id = 1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> commit;
Query OK, 0 rows affected (0.00 sec)

- Process 1 

mysql> select id, Subject from Tickets where id = 1;
++-+
| id | Subject |
++-+
|  1 | foo |
++-+
1 row in set (0.00 sec)

mysql> select id, Subject from Tickets where id = 1 FOR UPDATE;
++-+
| id | Subject |
++-+
|  1 | bar |
++-+
1 row in set (0.00 sec)



Contrast this with PostgreSQL, whose definition of repeatable read
acknowledges that fully consistent updates are not possible in all
cases:

- Process 1 
rt4=# start transaction;
START TRANSACTION
rt4=# set transaction isolation level repeatable read;
SET
rt4=# select id, Subject from Tickets where id = 1;
 id | subject 
+-
  1 | foo
(1 row)

- Process 2 
rt4=# start transaction;
START TRANSACTION
rt4=# set transaction isolation level repeatable read;
SET
rt4=# update Tickets set Subject = 'bar' where id = 1;
UPDATE 1
rt4=# commit;
COMMIT

- Process 1 
rt4=# select id, Subject from Tickets where id = 1;
 id | subject 
+-
  1 | foo
(1 row)

rt4=# select id, Subject from Tickets where id = 1 FOR UPDATE;
ERROR:  could not serialize access due to concurrent update



 ( Yes, MySQL requires SET TRANSACTION ISOLATION _outside_ the
   transaction, and PostgreSQL requires it to be _inside_.  See 
   https://dev.mysql.com/doc/refman/5.7/en/set-transaction.html
   https://www.postgresql.org/docs/9.1/static/sql-set-transaction.html )


> Should I change the default isolation level on Postgres for RT to
> 'repeatable read'?

No.  You should try the 4.4/previewscrips-race branch, which I've just
pushed:

https://github.com/bestpractical/rt/compare/4.4-trunk...4.4/previewscrips-race

The gory details are contained in the commits therein.
 - Alex


Re: [rt-users] RT 4.4.1 and transaction isolation level on Postgres

2017-01-02 Thread Alex Vandiver
On Mon, 2 Jan 2017 17:12:29 +0100
Václav Ovsík  wrote:
> Can anybody confirm on different system?

Thanks for the detailed replication instructions.  I can replicate, and
have tracked down a minimal replication case.  I'll drop my findings
and suggestion on your ticket.

The short form is that this is due to 4.4's new PreviewScrips
functionality, which simulates (then rolls back) all of the changes,
which is racing with the actual change.  The bad news is that it's the
real change, not the dry run, which gets killed in the deadlock
detector, meaning that the owner does go unchanged.  At least this is
bubbled up to the user in the ticket display page, but this is still
quite unfortunate.

Amusingly, there are some strong parallels to the canonical Therac-25
case[1] -- the race here requires that one trigger a PreviewScrips very
quickly before submitting, which is rare except with users quite
accustomed to the UI.  In the Therac-25 case, only skilled users
could navigate to the bottom of the form within 8 seconds and thus
deliver lethal doses of radiation to their patients.

...RT is designed to be robust, but there's a reason it doesn't rate
itself as meant to be used for safety- or life-critical applications.

 - Alex


[1] https://en.wikipedia.org/wiki/Therac-25


Re: [rt-users] Ticket SQL for dates?

2016-12-31 Thread Alex Vandiver
On Sat, 31 Dec 2016 12:41:21 -0500
Alex Hall  wrote:
> In a thread a week or two ago, I was asking about the syntax for finding
> tickets by relative dates, like "3 days ago". It wasn't working, and at
> least one other list member was able to confirm that it wasn't.

You're getting tripped up by the "ago", which in English
inverts the meaning of less-than and greater-than.  A search
for "LastUpdated > '3 days ago'" finds tickets whose LastUpdated stamp
is after the point in time 3 days ago -- that is, within the last three
days.  Here's that search on issues.bestpractical.com:

https://issues.bestpractical.com/Search/Results.html?Query=Queue+%3D+%27RT%27+AND+Status+%3D+%27__Active__%27+AND+LastUpdated+%3E+%273+days+ago%27

TicketSQL isn't SQL because we don't want to allow SQL injection
attacks.
 - Alex


Re: [rt-users] mysql DB engine ndbdcluster

2016-12-15 Thread Alex Vandiver
On Wed, 14 Dec 2016 09:42:42 -0500
Mike Diehl  wrote:
> At the risk of picking a fight, I'd like to understand this a bit better.

Happy to explain more -- and my instinct may have been wrong on one
count; see below.


> As long as the database supports minimum functions, such as transactions, 
> joins, datatypes, etc., why should an application care about the underlying 
> storage engine?

Because distributed databases have different properties around
atomicity and data locality than single-host databases.  Applications
running atop such databases need to be built to accommodate these
correctness and performance properties.


The biggest issue is that of transaction isolation[1] -- not all
transactions are equal.  RT assumes that a database transaction gives it
"repeatable read" isolation from other transactions.  This isolation
level, the default for InnoDB tables[2], means that essentially, upon
the first read, a snapshot of the state of the database is taken, and
it provides strong guarantees that regardless how long the transaction
is open, all queries within it will return consistent data[3].

I believe it likely (though I cannot prove, offhand) that RT assumes
repeatable read isolation semantics -- and NDB only offers "read
committed" isolation, which admits the possibility of different
results for the same query run twice within the same transaction.

However, upon writing this, it occurs to me that Postgres' default
isolation level is _also_ "read committed."[4]  Thus any possible race
conditions that might show up under NDB are also possible under
Postgres.  I'd need to do some close analysis to determine if this
means that Postgres is open to data corruption, or if both are safe
because the queries RT runs do not care about repeatable-read
semantics.

So NDB may actually be fine on this front.



The other property concerns data locality, and is purely a performance
constraint.  NDB stores data across a cluster of data notes,
optionally with replication, which are queried by other hosts that
serve as SQL nodes[5].  This means that joining data across tables
cannot be done in-memory, but instead may incur network-level
latencies to match up the data between data nodes -- meaning poor
query performance.

MySQL Cluster 7.2 (equivalent to MySQL 5.5) does provide some tricks to
prevent this performance hit[6], but it's not clear that those
optimizations will be able to be applied to RT's queries, as not all of
the column types match between tables.  It also doesn't get you all
of the way to InnoDB join performance.  Finally, the tables may also
need explicit hinting in order to partition the data to give any sort of
locality across the hosts.

On the other hand, if you're deploying an NDB cluster, you may already
have the MySQL DBAs on-hand to attend to those.  I've never heard of
an NDB deploy, discovering the correct partitioning scheme would be
all uncharted territory.



NDB clusters also don't support FULLTEXT indexes[7], though that's
clearly only an optional feature for RT.


Pescoller, consider me curious to hear back if you actually deploy RT
against and NDB cluster in production, and the performance
characteristics you see compared to single-host InnoDB.
 - Alex



[1] https://en.wikipedia.org/wiki/Isolation_(database_systems)
[2] 
https://dev.mysql.com/doc/refman/5.7/en/innodb-transaction-isolation-levels.html
[3] Repeatable read nominally opens up the possibility of "phantom
reads" where range queries can return inconsistent data from one query
to another; however, InnoDB uses some clever locking tricks to prevent
them.
[4] 
https://www.postgresql.org/docs/9.1/static/transaction-iso.html#XACT-READ-COMMITTED
[5] http://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-overview.html
[6] 
http://mysqlhighavailability.com/70x-faster-joins-with-aql-in-mysql-cluster-7-2/
[7] 
https://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-limitations-syntax.html
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Los Angeles - January 9-11 2017


Re: [rt-users] mysql DB engine ndbdcluster

2016-12-13 Thread Alex Vandiver
On Tue, 13 Dec 2016 12:25:37 +0100
Pescoller Reinhold  wrote:
> Thanks for your informations.
> 
> I tried to do so but rt give me an error that innodb is required and
> that I should upgrade my tables.
> Have I to change this direct in the code?


lib/RT/Handle.pm:
https://github.com/bestpractical/rt/blob/095caac2a4b4fc7baba0d7878a79f8b486579854/lib/RT/Handle.pm#L291

I'll reiterate that while RT may appear to work in trivial
conditions, you're setting yourself up for a world of both poor
performance and nasty race condition bugs. You get to keep all of the
pieces when it bursts into flames in production -- NDB is in no way a
supported, suggested, or sane backing engine for RT.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Los Angeles - January 9-11 2017


Re: [rt-users] mysql DB engine ndbdcluster

2016-12-13 Thread Alex Vandiver
On Tue, 13 Dec 2016 10:28:41 +0100
Reinhold Pescoller  wrote:
> Is there some possibilty to change the default db engine from innodb to 
> ndbdcluster in rt4?

RT assumes REPEATABLE READ isolation; you may encounter subtle and
difficult to diagnose bugs under READ COMMITTED isolation, which is all
that NDB supports.  Performance of joins is generally not great under
NDB, and RT assumes that joins do not incur a significant cost penalty.

In short, you _might_ be able to get it to run simply by adjusting the
table types, but I only expect it to get you into trouble down the road.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Los Angeles - January 9-11 2017


Re: [rt-users] supplying database credentials to rt-fulltext-indexer

2016-12-10 Thread Alex Vandiver
On Fri, 9 Dec 2016 17:30:31 -0500
Alex Hall  wrote:
> What this file never says is how to tell the indexer tool how to connect to
> the database. It clearly isn't pulling from the RT configuration, nor from
> /home/www-data/rtrc.

The indexer reads and uses the database configuration from
your /opt/rt4/etc/RT_Config.pm and /opt/rt4/tec/RT_SiteConfig.pm
files.  rtrc files are _only_ used by the "bin/rt" tool, which is meant
to be run from other machines than your RT host.

The rt-setup-fulltext-indexer wants to know your DBA username and
password because it needs to create new tables and indexes, which the
standard RT database user does not have permissions to do.  Once the
index is set up, updates to the index are done as RT's standard
database user.

 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Los Angeles - January 9-11 2017


Re: [rt-users] FTS enabled in SiteConfig, but not being enabled

2016-11-16 Thread Alex Vandiver
On Tue, 15 Nov 2016 11:36:53 -0500
Alex Hall  wrote:
> I found the problem. I've seen "Table" and "TableName" both used in sample
> configurations, but apparently only one is correct. As soon as I switched
> to "Table" in my config file, FTS enabled itself.

If any of the configurations with "TableName" that you found were on
BPS' site, please let them know where, so it can be fixed.  If they
were on the wiki, please update them.

When I start RT 4.4.1 with "TableName" instead of "Table", I see the
following in my error log:

[error]: No Table set for full-text index; disabling

It surprises me that you didn't see such an error.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Los Angeles - January 9-11 2017


Re: [rt-users] rt-setup-fulltext-index dropped DB connections with MySQL

2016-10-13 Thread Alex Vandiver
On Thu, 13 Oct 2016 20:32:36 +
James Zuelow  wrote:
> Package versions at the moment are MySQL 5.6.30, Perl 5.2.22, and
> request-tracker 4.2.13.  There is a Perl 5.4 upgrade available, but
> that breaks my rt4 installation.

For reference, your perl versions are not comprehensible.  Wheezy ships
perl 5.14.2, per https://packages.debian.org/wheezy/perl, and there has
never been a perl version 5.2 or 5.4 -- 5.002 was released in 1996,
5.004 in 1997, and 5.6 in 2000.


> Indexing existing data...
> Going to run the following in the DB:
>  CREATE FULLTEXT INDEX AttachmentsIndex ON AttachmentsIndex(Content)
> 
> [58876] [Thu Oct 13 02:23:42 2016] [warning]: DBD::mysql::db do failed:
> Lost connection to MySQL server during query at
> ./rt-setup-fulltext-index line 736,  line 2.

Check your "max_allowed_packet" setting in mysqld.  The most common
cause is the MySQL client (here, rt-fulltext-indexer) sending something
in the MySQL wire protocol which is larger than that limit, which
causes the server to unceremoniously drop the connection.

Raise that limit and restart mysql, and it should resolve the issue.

 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] Customizing with Overlays

2016-10-02 Thread Alex Vandiver
On Mon, 3 Oct 2016 08:58:25 +0530
Nilesh  wrote:
> Can I put my Overlay files in local/lib?

Yup; that's what's generally suggested.

> Do I have to maintain the same directory structure?

Yes.

> Let's say I have to customize RT::Action::SendEmail, so my Overlay should
> be in local/lib/RT/Action/SendEmail_Overlay.pm or just
> local/lib/SendEmail_Overlay.pm ??

The former.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] Bug about subject in utf-8

2016-09-01 Thread Alex Vandiver
On Thu, 1 Sep 2016 09:42:59 +0200
Albert Shih  wrote:
> > First, https://tools.ietf.org/html/rfc2047#page-5
> >   unencoded white space characters (such as SPACE and HTAB) are
> >   FORBIDDEN within an 'encoded-word'
> >
> > As such, "=?utf-8?q? #NUMBER=5D?=" is not a valid encoded-word.  
> 
> Well I think that's my bad, I change a little the subject to fit my first
> email about the tag. The real subject is
> 
>   =?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour 
> =?utf-8?q?=C3=A0?= vous

OK, that's a little different.  Rather better.  It still violates:

>   However, an 'encoded-word' that appears in a header field defined as
>   '*text' MUST be separated from any adjacent 'encoded-word' or 'text'
>   by 'linear-white-space'.

But:

> I'm a not very good with perl, but when I try using ruby to decode this
> line
> 
> irb(main):008:0> 
> Mail::Encodings.unquote_and_convert_to('=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= 
> Obspm =?utf-8?q?#31684=5D?= Bonjour =?utf-8?q?=C3=A0?= vous','utf-8')
> => "Re: [Info Obspm #31684] Bonjour à vous"  
> 
> the result seem correct.

For decoders that are lenient to encoded-words that aren't
space-separated, that's correct.  The difference between this and what
you had previously is the non-encoded word between the two
encoded-words, which makes the space significant.

And indeed, this does point to an RT bug.  Namely, for historical and
bad reasons, RT doesn't use the standard MIME-words decoding library,
which would produce:

> perl -MEncode -lE 'print Encode::encode("utf8",
>  Encode::decode("MIME-header",
>   "=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour 
> =?utf-8?q?=C3=A0?= vous"))'
>
> Re: [Info Obspm #31684] Bonjour à vous

Instead, it rolls its own, and gets it wrong:

> perl -Ilib -MRT=-init -le 'print RT::I18N::DecodeMIMEWordsToUTF8(
>   "=?utf-8?q?Re=3A?==?utf-8?q?_=5BInfo?= Obspm =?utf-8?q?#31684=5D?= Bonjour 
> =?utf-8?q?=C3=A0?= vous","Subject")'
>
> Re: [Info Obspm#31684] Bonjourà vous

Specifically, it removes spaces before the second and later
encoded-words, due to
https://github.com/bestpractical/rt/blob/stable/lib/RT/I18N.pm#L445

This looks to be a bug.  I've pushed 4.2/encoded-word-spaces to
address it; if you'd like to test the fix locally, you can apply
https://github.com/bestpractical/rt/commit/bdd6bd96 .

Thanks for the more complete bug report.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017

Re: [rt-users] Bug about subject in utf-8

2016-09-01 Thread Alex Vandiver
On Wed, 31 Aug 2016 23:12:28 +0200
Albert Shih  wrote:
> So until known everything is correct. The problem is when the person who
> answer this ticket encode the subject like this
> 
>   =?utf-8?q?Re=3A?==?utf-8?q?_=5BRTTAG =?utf-8?q? #NUMBER=5D?= Bonjour 
> =?utf-8?q?=C3=A0?= vous
> 
> because in that case RT drop the space between the RTTAG and the #NUMBER.

What mail client is generating that?  Whatever it is, it is violating
RFC 2047 spec in _multiple_ ways.

First, https://tools.ietf.org/html/rfc2047#page-5
  unencoded white space characters (such as SPACE and HTAB) are
  FORBIDDEN within an 'encoded-word'

As such, "=?utf-8?q? #NUMBER=5D?=" is not a valid encoded-word.

Secondly, https://tools.ietf.org/html/rfc2047#page-7
  However, an 'encoded-word' that appears in a header field defined as
  '*text' MUST be separated from any adjacent 'encoded-word' or 'text'
  by 'linear-white-space'.

As such, "=?utf-8?q?Re=3A?==?utf-8?" is not valid, as the two
"encoded-word"s are not separated by spaces.

Even ignoring those errors, the example you gave still isn't parsable.
My best attempt splits it into the following tokens:

 =?utf-8?q?Re=3A?= # "Re:
 =?utf-8?q?_=5BRTTAG   # " [RTTAG", but no closing "?=" ?!
 =?utf-8?q?#NUMBER=5D?=# "#NUMBER]"
 Bonjour   # "bonjour"
 =?utf-8?q?=C3=A0?=# "à 
 vous  # "vous"

Were it somehow parsed as the above, RT would _still_ be correct in
omitting the space before the number, because space between
encoded-words is removed, https://tools.ietf.org/html/rfc2047#page-10 :

  When displaying a particular header field that contains multiple
  'encoded-word's, any 'linear-white-space' that separates a pair of
  adjacent 'encoded-word's is ignored.


In short, fix the mail client.  Failing that, set
$ExtractSubjectTagMatch, as this is not a bug in RT.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017

Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!

2016-08-26 Thread Alex Vandiver
On Tue, 23 Aug 2016 13:21:20 +0200
Claes Merin  wrote:

> I beg to differ about the attachments table, in our setup we rely on
> customfields and there parameters. We have quite a few tickets that does
> not have any attachments.
>   If you do not write anything in the message body when you create a
> ticket (through web-ui or REST), there will be no attachment connected
> to that ticket.

Creating tickets via the web UI creates empty Attachment rows if the
user enters no content, from my testing on 4.4. I thought we'd
resolved that REST didn't do the same, but I concur that still isn't the
case.

> Also i did a comparison between mysql/mariadb FULLTEXT index and sphinx,
> and it seems that sphinxsearch is able to extract more info than the
> mysql FTS. I guess it's about the data processing that sphinx does with
> encoding and other things.

I'd be curious to hear more about this.  What do you mean by "extract
more info"?  Do you mean better stemming, or what?

> Our goal is to include CF's in the fulltext index as well. As far as I
> can see right now, there are two ways to do this.

Be aware that this can have security implications, as RT has no way to
apply ACLs if all of the different content sources are in the same
index; it becomes possible to find tickets based on custom field values
you cannot actually see.

> a) Modify the RT simplesearch use an "TicketsIndex" instead of the
> "AttachmentsIndex".
> 
> Use sphinx to create the TicketsIndex table and modify the collector
> query to include both attachment, ticket.subject and customfields in the
> the database. (Simple join statements)
> 
> b) Create "null" attachments and attach these to tickets that does not
> have any current attachments, just to create and relation between ticket
> and attachment. The advantage with this is that we do not have to modify
> any of the RT code, but of course it adds some rows to the database.

(b) makes more sense to me.  You currently can limit on transaction
date and creator, and those limits apply in conjunction with content
limits.  If you effectively lump all attachment content to be
cross-referenced by ticket-id instead of attachment-id, you lose that
functionality.

 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] Full text indexing error with MySQL

2016-08-25 Thread Alex Vandiver
On Thu, 25 Aug 2016 20:49:48 -0700
Omen Wild  wrote:
> I activated the full text indexing under Ubuntu 16.04 (request-tracker4
> 4.2.12-5) and (mysql-server 5.7.13-0ubuntu0.16.04.2). Both the initial
> indexing and subsequent calls to rt-fulltext-indexer have the same error:
> 
> - Begin quote -
> [3159] [Fri Aug 26 03:03:08 2016] [warning]: DBD::mysql::st execute failed: 
> MySQL server has gone away

That error is almost always that RT tried to do an insert or update
of data larger that the MySQL server's max_allowed_packet.  Try
increasing that setting, restarting MySQL, and indexing again.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!

2016-08-19 Thread Alex Vandiver
On Fri, 19 Aug 2016 16:35:07 +0200
Claes Merin  wrote:
> We are using sphinx (since last night) for FTS searches.
> 
> Setup is as described in this excellent guide:
> 
> http://blog.christosoft.de/2016/07/rt4-request-tracker4-fulltext-search-sphinx-debian-jessie/

Is there a reason you're not using MySQL's built-in search, available
since 4.2.10?  It's more performant, and has none of the caveats:
https://docs.bestpractical.com/rt/4.2/full_text_indexing.html#Caveats1

Especially since it allows for incremental indexing of new attachment
rows, I know of no reason to prefer Sphinx.

> We also ran in to the problem described here, and I was thinking if one
> could resolve it by including the subject in the sphinx database...

I expect that should work fine.

> Modifying the query in the guide to simply include the t.Subject like
> this...
> 
> ## /etc/sphinxsearch/sphinx.conf
> sql_query = \
>   SELECT a.id id, t.Subject, a.content FROM Attachments a \
>   JOIN Transactions txn ON a.TransactionId = txn.id AND txn.ObjectType =
> 'RT::Ticket' \
>   JOIN Tickets t ON txn.ObjectId = t.id \
>   WHERE a.ContentType LIKE '%text%' AND t.Status != 'deleted'
> ##
> 
> The problem with this is, if there is a ticket that does not have an
> attachment, it will not get indexed and is not searchable through FTS...

The "attachments" table is used to index all content in RT.  This
includes textual content, such as the bodies of messages.  As such, it
is impossible for a ticket to not have at least one Attachment record.

 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] manipulate correspond content in history

2016-08-19 Thread Alex Vandiver
On Fri, 19 Aug 2016 15:26:12 +0300
Woody - Wild Thing Safaris  wrote:
> My correspond scrip adds a table of CF values to the correspond email 
> from the template, and sends to the client. I would like those same 
> changes appended to the transaction content that appears in the history, 
> so it matches what was sent to the client and we know easily what was sent.

This is what the "Show" link next to the "Outgoing email recorded"
button is for.

> $self->TransactionObj->ContentObj->SetContent($content);
>
> no errors, but the content is not changed

All ->Set... methods return a tuple of ($success, $msg).  You'll find
that $success is false, and $msg is telling you that the field is
read-only.

RT views attachments as effectively immutable.  Changing this invariant
is complex, and not generally recommended.
 - Alex
-
RT 4.4 and RTIR training sessions, and a new workshop day! 
https://bestpractical.com/training
* Boston - October 24-26
* Los Angeles - Q1 2017


Re: [rt-users] RT adding words on the beginning of the emails

2016-08-04 Thread Alex Vandiver
On Thu, 4 Aug 2016 14:26:58 +
Renos Nikolaou  wrote:
> We are using RT 4.4.1 and we would like to know why our RT adding the
> word 'Capture' (When we forward screenshot from Snipping Tool to
> outlook and then to open a ticket) and some other word like our
> organization name on the beginning of each emails ?
>
> Is there any way to disable this option ?

This is not an "option" from RT, merely a property of RT's HTML
rendering of the mail that Snipping Tool creates.  If you click the
"Download (untitled)" link on the right edge, you can view the HTML
that is being generated, which may make clear where the "Capture" word
is coming from.
 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Los Angeles - September, 2016


Re: [rt-users] How to sync translations?

2016-07-27 Thread Alex Vandiver
On Tue, 26 Jul 2016 10:53:04 -0700 (MST)
fleon  wrote:
> I have my own spanish translation (which i uploaded today in launchpad). It's
> about 99% complete. But today i upgraded to 4.4.1 and lost my changes (i did
> backup it first).

Looks like there were failures importing your changes:
https://translations.launchpad.net/rt/4.4/+imports
Namely, line 5997 has an extra " at end of line.

Fix those up and re-upload.  Once that imports, you can export the
translations (see below) and look for new untranslated strings.

> What method should i use to put my file back and know which new lines need
> translating, if any? Are all new strings at the bottom of the file?

The .po file is sorted alphabetically.  

> I tried using diff but since the line numbers in the comments changed it
> shows up too many changes. Perhaps some script-fu to ignore all lines with
> comments?

If you export the translations from Launchpad as a tarball from
https://translations.launchpad.net/rt/4.4/+export , it will email you a
link.  Pass that link to:

devel/tools/rt-message-catalog rosetta the-url-from-email-here

...and it will merge in the translations from Launchpad into your local
share/po/*.po files, preferring translations from launchpad but leaving
any new local translations.

 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Los Angeles - September, 2016


Re: [rt-users] Bizarre errors with shredder

2016-04-13 Thread Alex Vandiver
On Tue, 12 Apr 2016 10:41:24 +0200 (CEST)
"Sternberger, Sven"  wrote:
> root@rt:~# /opt/rt4/sbin/rt-shredder --force --plugin 
> "Users=no_tickets,1;status,any;replace_relations,Nobody;limit,3;member_of,unprivileged"
> SQL dump file is '/root/20160411T134517-0001.sql'
> [2329] [Mon Apr 11 13:45:50 2016] [critical]: Couldn't wipeout object: 
> Bizarre copy of CODE in list assignment at 
> /usr/local/share/perl/5.18.2/Devel/StackTrace.pm line 61. 
> (/opt/rt4/sbin/../lib/RT.pm:388)

"Bizarre copy of ..." is always a sign of an internal error in perl.
In this case, it's caused by the fact that objects on the perl stack
aren't refcounted, which means that in some cases they can already have
been garbage-collected by the time the object is examined.
Specifically, when RT produces an error, Devel::StackTrace tries to
suss out the values that were passed to each function in the
call stack -- and in some cases, those values may no longer exist.

So this is a sign of some other error, which Devel::StackTrace is
trying to build the stack trace for, which itself triggers an internal
perl error.  RT catches _that_ error and gives the stack trace for
_that_, which is why what you're seeing is a stack trace of the log
infrastructure.

You can likely work around this by disabling LogStackTraces, which I
believe you've turned on.  You'll then see the underlying RT error that
it was trying to report the stack trace of.

Perl 5.18 is technically out of support, but "stack isn't refcounted"
isn't exactly fixed in any recent Perls, so upgrading might only fix
this if it happened to shift the global destruction order.

 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Washington DC - May 23 & 24, 2016


Re: [rt-users] Full text indexer issue with img html tag

2016-03-31 Thread Alex Vandiver
On Thu, 31 Mar 2016 09:47:06 +0200
Peter Viskup  wrote:
> Just discovered we didn't ran rt-fulltext-indexer enough many times.
> It proceeded with only 200 attachments at once. Is that some
> limitation of PgSQL or rt-fulltext-indexer? Didn't read that in the
> documentation.

See
https://docs.bestpractical.com/rt/4.4.0/full_text_indexing.html#Updating-the-index

You can pass --all to index all attachments, which looks to not be
documented.
 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Washington DC - May 23 & 24, 2016


Re: [rt-users] Full text indexer issue with img html tag

2016-03-30 Thread Alex Vandiver
On Wed, 30 Mar 2016 21:32:37 -0700
Alex Vandiver <a...@chmrr.net> wrote:
> What version of RT?  That's a symptom of RT 4.0.9 or earlier; 4.0.10
> contains a fix that makes RT simply skip that attachment.

Actually, my mistake -- it was fixed in 4.0.3:
https://github.com/bestpractical/rt/commit/692b5bcb

If you're running pre-4.0.3, you _certainly_ want to upgrade.
 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Washington DC - May 23 & 24, 2016


Re: [rt-users] Full text indexer issue with img html tag

2016-03-30 Thread Alex Vandiver
On Wed, 30 Mar 2016 17:41:02 +0200
Peter Viskup  wrote:
> we are experiencing issue with full text indexer

What version of RT?  That's a symptom of RT 4.0.9 or earlier; gradi4.0.10
contains a fix that makes RT simply skip that attachment.
 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Washington DC - May 23 & 24, 2016


Re: [rt-users] PostgreSQL full-text index default type: GiST or GIN

2016-03-19 Thread Alex Vandiver
On Thu, 17 Mar 2016 17:47:50 -0500
Matt Zagrabelny  wrote:
> > I am curious about the benchmarks you used to make this determination?
> > In particular, the GIN fastscan option can dramatically improve search
> > performance in versions 9.4 and above. Here is a nice discussion:
> >
> > http://blog.pgaddict.com/posts/performance-since-postgresql-7-4-to-9-4-fulltext
> >
> > In addition, GIN indexes are much smaller in PostgreSQL 9.4 and above.
> > So I am curious about the data that motivated the change to GiST.

You are quite correct; GIN indexes are superior in virtually every
way for our use case.  And the documentation is unfortunately in error;
the default switched in the other direction, from GiST to GIN. See
https://github.com/bestpractical/rt/commit/e103f6da for the actual
behavior change.

This mistake is entirely my fault, and my only excuse for getting it
backwards is that the documentation commit was written several months
after the code was. Apologies!

> Here is a correspondence between myself and a BP engineer you  may find 
> helpful:
> [snip]

That correspondence is out of date; at the time, GiST was the
default, based on no particular research.  The change to default to
GIN was made late 2014/early 2015 after I conducted additional research
on real-world data -- as well as the Postgres documentation, which is
quite clear that static data should use GIN.

You can see the results of that research at
https://chmrr.net/fts-charts/pg.html and
https://chmrr.net/fts-charts/query.html#pg

Those are from performing speed analysis on rebuilding the index on a
clone of Best Practical's own ticketing system, which was taken as
being a representative real-life sample of data.  It was run on
Postgres 9.3, and resulted in the following branch:
https://github.com/bestpractical/rt/compare/1d1ffe44...7c48294a
The last ~7 commits are the most Postgres-specific.

The end result is an indexing engine which, in its default
configuration, indexes data an order of magnitude faster, as well as
provides results an order of magnitude faster.  Wins all around!

I'm happy to supply a patch to fix the documentation, but I suspect
Shawn or someone at BPS will be faster to simply fix it directly. :)
 - Alex
-
RT 4.4 and RTIR Training Sessions https://bestpractical.com/training
* Washington DC - May 23 & 24, 2016


Re: [rt-users] Help! I enabled Full Text search and now Simple search won't look at Subjects !!

2015-10-09 Thread Alex Vandiver
On Wed, Oct 07, 2015 at 04:23:02PM -0500, k...@rice.edu wrote:
> It looks like you should be able to change the following function in
> lib/RT/Search/Simple.pm: [snip]

RT 4.2.10 made the change to only search the Content field, and not
also the Subject field, intentionally; see the commit the did so [1]
for the full rationale.

In short, it is impossible to have a performant full-text search if
you search both Subject and Content.  In most RT installs, the
auto-reply from the ticket creation includes the subject of the
ticket, which means it is indexed along with the ticket, so removing
the Subject clause still results in finding the appropriate ticket.

You're welcome to revert the change locally -- though I would suggest
doing so via an overlay and not by editing the file directly, or your
change will be lost when you next upgrade.  If you do make the change,
I expect you'll find the simple search to be noticeably (and perhaps
unusably) slower.
 - Alex



[1] 
https://github.com/bestpractical/rt/commit/8450f0a9f233d6a761ac22dbdf14926abc54d7fa


Re: [rt-users] GnuPG key management issues

2015-10-09 Thread Alex Vandiver
On Thu, Oct 08, 2015 at 08:40:37AM +0200, Christopher Kunz wrote:
> we are using the RT::Crypt::GnuPG module and have run into an issue that
> I'm afraid we cannot solve.
> [snip]
> This is where the problems start. With our current setup (and a local
> keyring on the RT server), RT will try handling the .asc attachment as
> encrypted data (which it isn't), fail and not handle the rest of the
> ticket.

I believe the 4.2/skip-asc-keys branch[1] addresses this particular
issue, of interpreting .asc as encrypted data.  I'm sure that BPS
would appreciate the feedback if it resolves the issue for you.

> We are using RT 4.2.8. Is there any configuration setting that I might
> have missed, or any other possibility to get RT to automatically import
> keys that come into the system as a .asc attachment? I am aware of the
> security implications of this (and, in fact, have set the trust settings
> to "always", as authentication of user requests is always performed
> separately).

The security implications of such a flag almost certainly preclude its
inclusion in core RT -- though you understand the security
implications, many sites might not, and might enable it regardless of
any warnings placed on it.  Operating in such configurations is far
worse than operating without GPG at all.

If you wish to implement this yourself, there are a couple options for
where to implement the behavior.  The first two of them will require
that the web user have write access to the keyring.

  1. Implement it as a mail plugin, run before RT::Crypt::GnuPG.  This
 will allow it to import the key before the decryption happens.
 Mail plugins in 4.2 are cumbersome, but an ApplyBeforeDecode
 plugin that is in @MailPlugins before Auth::Crypt should work fine.

 I would suggest using RT::Extension::FutureMailgate[2] backport
 of the 4.4/mailgate-refactor branch[3] to make writing the mail
 plugin entirely more friendly, but I just checked, and it looks
 like that backport doesn't include the Crypt changes from the
 branch[3], so running that may be unhelpful.

  2. Write a scrip to parse out the keys after the mail comes in.
 This means that if the original mail is signed by the key it
 includes, the key will be added after it has been examined, so it
 will be marked as unverified.

  3. Pre-process the keys in the mail gateway, or before.  If the mail
 has a standard format, you can potentially do all of the work by
 having procmail or similar match the message and pipe it to a
 custom program that extracts and adds the key.  This removes RT
 from the loop entirely.

If this is central to your workflow, you may wish to consider
contacting sa...@bestpractical.com to see if they can help you
implement one of the above solutions.
 - Alex

[1] https://github.com/bestpractical/rt/compare/4.2-trunk...4.2/skip-asc-keys
[2] https://github.com/bestpractical/rt-extension-futuremailgate
[3] https://github.com/bestpractical/rt/compare/master...4.4/gateway-refactor


Re: [rt-users] 'Started' not being set when ticket status changes from 'New' -> 'xxx'

2015-09-12 Thread Alex Vandiver
On Sat, 12 Sep 2015 13:37:51 -0700 Alex Vandiver <a...@chmrr.net> wrote:
> On Fri, 11 Sep 2015 14:55:47 -0500 "k...@rice.edu" <k...@rice.edu>
> wrote:
> > Just to follow-up. I made a global scrip to set the 'Started' date
> > field when it is un-set and the transaction is a status change from
> > 'new' to either 'open' or 'resolved'. I thought that this was
> > handled by RT internally when the status changed from 'new' but I
> > could not find any place in the code that handled it.
> 
> This was in core code in RT 4.0, but was moved to a global scrip in RT
> 4.2: [snip]

Sorry, I mis-read your question; the scrip in question is for the
opposite (opening the ticket if Started is set).  The code that sets
Started when a ticket is moved to a non-initial state is:

https://github.com/bestpractical/rt/blob/stable/lib/RT/Ticket.pm#L2393-L2403

The only thing that could be causing that to not fire is if you have a
custom lifecycle with all-Initial statuses.
 - Alex


Re: [rt-users] 'Started' not being set when ticket status changes from 'New' -> 'xxx'

2015-09-12 Thread Alex Vandiver
On Fri, 11 Sep 2015 14:55:47 -0500 "k...@rice.edu"  wrote:
> Just to follow-up. I made a global scrip to set the 'Started' date
> field when it is un-set and the transaction is a status change from
> 'new' to either 'open' or 'resolved'. I thought that this was handled
> by RT internally when the status changed from 'new' but I could not
> find any place in the code that handled it.

This was in core code in RT 4.0, but was moved to a global scrip in RT
4.2:

https://bestpractical.com/docs/rt/4.2/UPGRADING-4.2.html (search for
Started)

If this was an RT instance that was upgraded from 4.0, the lack of this
behavior my imply that not all upgrade steps were run.
 - Alex


Re: [rt-users] forwarding correspondence lacks important data in headers

2015-09-08 Thread Alex Vandiver
On Tue, Sep 08, 2015 at 08:56:53AM +0200, Arkadiusz Miśkiewicz wrote:
> 
> Hi.
> 
> rt 4.2.12, scenario - there is a ticket, user enters new correspondence via 
> web interface which is send via email to requestor.
> 
> Then user clicks "forward" near that correspondence and forwards it to third 
> email address /  third person.
> 
> Unfortunately that email sent to third person is lacking important 
> information. It lacks From, To, Subject headers in attached forwarded message.

This should have been fixed in 4.2.10:
  https://bestpractical.com/release-notes/rt/4.2.10
  
https://github.com/bestpractical/rt/commit/134a478be1b63ca418380fc355923ea849812bfa

Are you still seeing a lack of such headers for tickets created since
your upgrade to 4.2.12?  Tickets created _prior_ to an upgrade to
4.2.10 will still lack the headers in their Create transactions, of
course.

> I would expect to see proper From, To, Subject, Date (like in original 
> correspondence) and possibly other headers, too. Also there is incorrect 
> Content-Length in it.

Hm -- the content-length problem doesn't seem to be replicated on
issues.bestpractical.com.  Are you seeing it with a vanilla 4.2.12
install?
 - Alex


Re: [rt-users] forwarding correspondence lacks important data in headers

2015-09-08 Thread Alex Vandiver
On Tue, Sep 08, 2015 at 03:35:46AM -0400, Alex Vandiver wrote:
> > I would expect to see proper From, To, Subject, Date (like in original 
> > correspondence) and possibly other headers, too. Also there is incorrect 
> > Content-Length in it.
> 
> Hm -- the content-length problem doesn't seem to be replicated on
> issues.bestpractical.com.  Are you seeing it with a vanilla 4.2.12
> install?

Wait, the content-length of a multipart/mixed part should indeed be
0 -- it itself has no content.  It is the sub-parts that have
content.  Arguably Content-Length should be entirely omitted on
multipart/* parts, but its presence is unlikely to cause
interoperability concerns.

But that does beg the question -- are you seeing the headers bug only
on multipart/mixed messages?  How are you creating them via the web
UI, by adding attachments?
 - Alex


Re: [rt-users] rt-mailgate needs http for comment

2015-09-06 Thread Alex Vandiver
On Sat, Sep 05, 2015 at 11:22:48PM -0700, Joseph D. Wagner wrote:
> I had apache set to allow rt over https only.  Trying over http would fail.
> 
> rt-mailgate was working perfectly fine over https when using
> "--action correspond".  However, when using "--action comment", it
> fails saying it is unable to connect.
>
> When I changed my apache configuration to allow http on local
> connections, it started working.
> 
> I suspect there is some code in the comment path of rt-mailgate that
> is forcing it over the http connection, rather than properly
> deriving the connection from the url parameter.
> 
> Being new to rt, I am open to the possibility I misconfigured
> something.  Is anyone else able to reproduce this?

I strongly suspect misconfiguration in your /etc/aliases.  The only
difference between correspond and comment paths is the value of a
query parameter that they POST:

https://github.com/bestpractical/rt/blob/stable/bin/rt-mailgate.in#L168-L170

Check to make sure that you have https:// on all of your aliases, and
that you've run newaliases (or equivalent) after updating them.
 - Alex


Re: [rt-users] rt-mailgate needs http for comment

2015-09-06 Thread Alex Vandiver
On Sun, Sep 06, 2015 at 12:31:28AM -0700, Joseph D. Wagner wrote:
> Here it is.  I left everything intact except the url.
> [snip]

Those look fine.  Double-check that you have not multiply-defined
prc-staff elsewhere in aliases, and that you've run `newaliases`.
Short of that, my only suggestion is to turn on bug logging in your
MTA -- I can say with certainty that rt-mailgate doesn't deal
differently with correspond vs comment and http/https connections.
 - Alex


Re: [rt-users] Customize QuickSearch for RT 4.0.19 - Removing all

2015-09-04 Thread Alex Vandiver
On Fri, Sep 04, 2015 at 09:45:56AM -0700, ewanchic wrote:
> I'm not Looking to change the Lifecycles because that look fine to me, but I
> would like to just remove all the extra criteria and just focus on the
> criteria requested in the Quick Search. Is this possible? Thanks

   
https://docs.bestpractical.com/RT_Config#OnlySearchActiveTicketsInSimpleSearch

Set this to 0 in your RT_SiteConfig.pm:

   Set( $OnlySearchActiveTicketsInSimpleSearch, 0);

 - Alex


Re: [rt-users] RT and Disaster Recovery - problem

2015-09-02 Thread Alex Vandiver
On Wed, Sep 02, 2015 at 03:11:52PM +, Guadagnino Cristiano wrote:
> Our RT servers are virtualized on VMware.
> [snip]
> However, the problem is that - after reconfiguring the VMs - RT becomes 
> slow as a snail (tens of seconds for each page change/refresh).

Do the VMs have any snapshots enabled?  I know that historically, at
least, the mere presence of a snapshot file caused all I/O to be COW,
which caused order-of-magnitude I/O bandwidth degradation.
 - Alex


Re: [rt-users] scrip to delete a Cc adress in a ticket

2015-09-01 Thread Alex Vandiver
On Mon, Aug 31, 2015 at 05:24:51PM +0200, Loïc Cadoret wrote:
> We are running RT 3.8.11 (update to RT 4.2.x is currently not an option)

Your RT instance contains arbitrary remote execution of code, session
fixation and hijacking, XSS injection, SQL injection, and weak
password hashing that allows trivial reconstruction of passwords from
said SQL injection.

Whatever your reasons are for 4.2 being "not an option," you should at
_very_ least upgrade to 3.8.17, which resolves the worst of those.  It
will still, of course, be unsupported, and vulnerable to other
vulnerabilities (including CVE-2014-9472, a denial-of-service via RT's
email gateway, as well as CVE-2015-1165 and CVE-2015-1464, which allow
for information disclosure and session hijacking via RT's RSS feeds)
but will be slightly less exploitable.

Please upgrade.
 - Alex


Re: [rt-users] Pg FTS query works, RT search returns 0 results ( was: '.' as delimiter/boundary breaks domain name searches)

2015-09-01 Thread Alex Vandiver
On Mon, Aug 31, 2015 at 09:39:19AM -0400, Jeff Blaine wrote:
> I'm reviving this one time in case anyone has further ideas.
> 
> * PostgreSQL 8.4.20 (RHEL 6.6) with FTS does the right
>   thing when parsing an email address[1]
> 
> * An RT 4.2.12 search for the same string returns 0 results[2].
>   I'm 98% certain the ticket *is* indexed though as other queries
>   return it[2].
> 
> * The rt-fulltext-indexer script runs every 10 minutes and has
>   no errors.

Is the ticket in question a merged ticket?  You had a mail from the
same timeframe which dealt with merged ticket FTS, which is a known
bug: https://issues.bestpractical.com/Ticket/Display.html?id=9370
Sadly, there's not good solution to that bug at this time.

Otherwise, seeing the query RT is generating, and stripping clauses
out of it until it matches, is likely your best bet.  You can see the
SQL generated from a TicketSQL query by running:

perl -MRT=-init -le 
'$t=RT::Tickets->new(RT->SystemUser);$t->FromSQL("@ARGV");print 
$t->BuildSelectQuery' \
"Content LIKE 'f...@domain.com'"

  - Alex


Re: [rt-users] All Incoming SMIME Signed Messages Showing as No Trust

2015-07-24 Thread Alex Vandiver
On Thu, 16 Jul 2015 15:52:34 -0400 Zoey Schutt z...@braincoral.io
wrote:
 OS and RT4 Info:
 
 Debian GNU/Linux 7 (wheezy)
 Apache/2.2.22 (Debian)
 PHP 5.5.26-1~dotdeb+7.4
 Request Tracker 4.2.11

As a note, RT is written in Perl, not PHP.  The other useful version to
know is the version of openssl, which you can find by running:

   openssl version

 I am attempting to configure S/MIME support in my RT4 instance, and I
 have every piece working other than the verification of signatures on
 incoming email.

What software is generating your certificates, and sending the incoming
mail?  I suspect your certificates are weird in a way that is throwing
openssl off.

Can you send me a simple S/MIME signed message and your CA's PEM file,
off-list, so I can inspect it?

 - Alex


Re: [rt-users] can't create ticket when On Create Autoreply To Requestors scrip is enabled

2015-07-24 Thread Alex Vandiver
On Wed, 15 Jul 2015 11:46:08 -0700 Roman Massey romanmas...@gmail.com
wrote:
 On a fairly fresh RT 4.2.9 install, having a weird problem. Unable to
 create tickets, when creating through web interface it sits for about
 a minute then gets an error. When creating through email, it gets
 same error and fires a confirmation email every fetch mail.
 
 If I disable the scrip “On Create Autoreply to Requestors” then
 everything works fine (other than the requestors don’t get auto
 reply, which i require)

Do other mail-sending scrips work as expected?  Can you show your
RT_SiteConfig.pm?  What are you using as your local MTA?
 - Alex


Re: [rt-users] Question about pgp headers using GnuPG Inline option

2015-07-14 Thread Alex Vandiver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tue, 14 Jul 2015 22:51:55 -0500 Mario Góngora
mariogongor...@gmail.com wrote:
 I am trying to implement GnuPG on my RT (version  4.2.10) with the
 Inline option, but when I send a message people are receiving this
 type of wrapping :

Can you show your %Crypt, %GnuPG and %GnuPGOptions settings?  What
version of GPG does `gpg --version` say you're running?

 - Alex
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iEYEARECAAYFAlWl4ZEACgkQMflWJZZAbqBi1gCdHYg/6/ZAbfv2a1DyCDggmJYY
MPYAn1Hz3vr0JA4LUhJ4m2ckFC3QSXc2
=/E5T
-END PGP SIGNATURE-


Re: [rt-users] Notification sent to requestor even when squelched

2015-06-22 Thread Alex Vandiver
On Mon, 22 Jun 2015 19:10:50 +0200 Marcos Orallo m_ora...@yahoo.es
wrote:
 I manage an RT 4.2.11 deployment using Debian Wheezy, Apache 2.2,
 mod_fcgid
 
 Today we had a very concerning issue regarding a notification being
 sent to a requestor even when it had been explicitly unchecked in the
 recipients list. At first I didn't believe the user, but then I found
 this in the log:
 [snip]

There's a 47-second interval between the first and the second of
those.  Can you show the full log of everything between them?  My guess
is that the first one was a dry run to list the potential recipients,
and the second was the actual submission.  This would explain the
different process IDs, as well as the different behaviors.

 You can see that the requestor address was unchecked, but he was
 notified anyway. The notificatoin script uses a custom condition to
 just send the notification when using the web interface, with the
 possibility of squelching recipients manually. The default behaviour
 (implicit notification to all requestors even for correspondence by
 mail) was confusing for my team mates.

Your wording is confusing.  Is this jut the stock Notify Requestor and
Ccs action, with a custom condition?
 - Alex


Re: [rt-users] Full-Text Search: RT-4.2.11 MySQL 5.6.25 vs MariaDB 10.0.19+Sphinx

2015-06-10 Thread Alex Vandiver
On Wed, 10 Jun 2015 19:47:27 + Paul Hirose pthir...@ucdavis.edu
wrote:
 In short, trying to find the best mechanism for FTS, whether I can
 use MySQL 5.6.25 InnoDB, or MariaDB 10.0.19 + Sphinx on RHEL 7.

My answers below assume RT 4.2.11 -- there were some order-of-magnitude
speed improvements to the indexer since 4.2.10.  I'd recommend pure
MySQL FTS (either InnoDB or MyISAM), and not Sphinx, as there are a
large number of caveats with the Sphinx search (no-result responses and
false negatives, primarily).  Interestingly the performance of InnoDB
FTS is slower than MyISAM's: https://chmrr.net/fts-charts/query.html

 I'd prefer to use MySQL, since that seems to support FTS by itself.
 Otherwise, I opt for MariaDB, simply because that has the Sphinx
 engine built-in to the RPM I can get (saving me the hassle of
 build-from-source) although I'd still have to separately maintain the
 actual Sphinx/searchd daemon package.
 
 Also, when running rt-fulltext-indexer for MySQL the notes say it
 does 200 tickets at a time.  I take it the script remembers the last
 ticket it did, and continues where it left off?   Does running this
 place any locks on any tables/rows, or can it be run at any time?

It indexes in batches, based on the high-water-mark of the last
successfully index attachment.  It does not use database-level
locking, instead using taking a lock on the indexer script itself;
there is no impact on availability of RT for other operations. It can be
canceled and resumed at will.

 And to be clear, for Sphinx, the rt-setup-fulltext-indexer  indexer
 rt is a one-time thing, and subsequent is simply indexer rt
 --rotate?  And similarly, running indexer rt --rotate can be done at
 any time even while RT is being used?

Yes -- however, `indexer rt` re-indexes _all_ attachments, since Sphinx
does not support incremental indexes.  As such, having close-to-live
FTS indexes with Sphinx requires either a small database, or a main +
delta technique -- see
http://www.sphinxconsultant.com/sphinx-search-delta-indexing/#delta-2
for an example of such.

 Oh, I do intend to DontSearchFileATtachments = true, although we
 might try it undef for a while, and see.

As a note, DontSearchFileAttachments is respected on search, not on
index, so it won't impact indexing time.
 - Alex


Re: [rt-users] Configuring fetchmail to poll emails to respective queues.

2015-06-04 Thread Alex Vandiver
On Thu, 4 Jun 2015 12:31:57 + Bob Shaker
rsha...@ardencompanies.com wrote:
 You don’t need 2 separate emails for those fields. Our instance works
 fine with the same email in both fields.

This isn't a good idea, though it does _work_.  The problem is when
someone records a comment, and it gets sent out just to your company --
but From the one address you have set.  When some well-meaning person
replies to that mail, it will come into the address RT uses for
correspondence -- and then the end-user will get the information that
you intended to be a comment.  This is at best confusing, and at worst
horribly embarrassing.
 - Alex


Re: [rt-users] [SOLVED] Re: charset troubles

2015-05-14 Thread Alex Vandiver
On Thu, 14 May 2015 16:50:23 +0200 m...@netbsd.org (Emmanuel Dreyfus)
wrote:
 I fixed it. Replying to myself with the whole story for someone else's
 future reference.

Good to hear the full debugging story.

 The problem was database encoding. RT can use PostgreSQL with encoding
 UTF-8 or the default SQL_ASCII. That later encoding means PostgreSQL
 does not care about encoding and just gives back the bytes it was given
 without any check. The former enforces UTF-8 usage and is able to
 automatically transcode if the client claims to use another encoding.
 
 My RT installation had been configured with the PostgreSQL database
 using UTF-8 encoding for a while. At some time I upgraded PostgreSQL
 and I reloaded the data from a dump after reinitializing the database.
 But since I did not check for it, it got SQL_ASCII, a setup where the
 application must take care of data encoding.

So this is the first place where things went awry.  How was the
database created to reload the database dump, such that it got
SQL_ASCII?  By hand using 'createdb' from the command line?  And is
your template0 database marked as 'SQL_ASCII' ?

For reference,
https://docs.bestpractical.com/backups#Restoring-from-backups1 is the 
documented technique for loading in a Pg backup.
 - Alex


Re: [rt-users] ticket content search maybe doesn't work as excepted

2015-05-12 Thread Alex Vandiver
On Tue, 12 May 2015 17:11:59 +0200 Christian Loos cl...@netcologne.de
wrote:
 playing around with the ticket content search I discovered many false
 positive results.

Yup -- known bug:

https://issues.bestpractical.com/Ticket/Display.html?id=19237
 - Alex


Re: [rt-users] Upgrading web/email server...should I upgrade RT itself too?

2015-05-08 Thread Alex Vandiver
On Fri, 8 May 2015 14:11:08 + Beachey, Kendric
kendric.beac...@garmin.com wrote:
 My question...is RT 4.0.17 itself old enough that I really ought to upgrade 
 it as well?

Yes.  4.0.17 has published security vulnerabilities against it
(CVE-2015-1464, CVE-2015-1165, CVE-2014-9472): 

  http://blog.bestpractical.com/2015/02/security-vulnerabilities-in-rt.html

 I'd like to minimize the amount of surprise for the users via new
 looks, so I'm wondering if there are any huge problems with staying
 at 4.0.17.

Upgrading within a stable series will never cause any major
user-visible UI changes, and should never break installed extensions.
Upgrades within a stable series (from 4.0.17 to 4.0.23, for instance)
are designed to be no-hassle bugfixes and security fixes.

If we feel a change has the potential to give an administrator reason to
_not_ upgrade within a stable series, it is unsuitable for that trunk.
Please upgrade.

All of this is hopefully also made clear on release policy page:
https://bestpractical.com/rt/release-policy.html

 - Alex


[rt-users] [rt-announce] Assets 1.05 released

2015-05-06 Thread Alex Vandiver
Assets 1.05 -- 2015-05-06
-

We are pleased to announce Assets 1.05.

https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.05.tar.gz
https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.05.tar.gz.asc

SHA1 sums

53b472ef543c99f3b1daae9e2e293c33dcd8cad8  RT-Extension-Assets-1.05.tar.gz
2e3a32e97ba720c3fd9f58d587424e05aec30e94  RT-Extension-Assets-1.05.tar.gz.asc

Changes from 1.04:
 * Only call FillCache once if enabling plugin during make initdb
 * Ensure that grouping-less custom fields keep their values

A complete changelog is available from git by running:
git log 1.04..1.05
or visiting
https://github.com/bestpractical/rt-extension-assets/compare/1.04...1.05
___
rt-announce mailing list
rt-annou...@lists.bestpractical.com
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce


Re: [rt-users] charset troubles

2015-05-01 Thread Alex Vandiver
On Fri, 1 May 2015 10:01:02 +0200 m...@netbsd.org (Emmanuel Dreyfus)
wrote:
 In database, I have in both cases

Which database?  In your original mail, you said:

 p5-DBD-mysql-4.031
 [...]
 postgresql-9.4.1

Which database are you using?
 - Alex


Re: [rt-users] charset troubles

2015-04-30 Thread Alex Vandiver
On Thu, 30 Apr 2015 21:08:00 +0200 m...@netbsd.org (Emmanuel Dreyfus)
wrote:
 I just upgraded Apache to 2.4 and RT to latest 3.8

RT 3.8 reached end-of-life over a year ago.  No release of Apache 2.4
had been made before RT 3.8 was in critical security releases only.
I'm unsurprised that there are incompatibilities between the two.

Please upgrade to a supported version of RT.  The unmaintained 3.8
series also now has disclosed security vulnerabilities against it.
 - Alex


Re: [rt-users] RT and SMTP

2015-04-29 Thread Alex Vandiver
On Wed, 29 Apr 2015 15:14:40 +0100 ABD EL MALEK BOUBARNOUS
abdelmalekboubarn...@student.emi.ac.ma wrote:
 How can I configue RT to use an SMTP server instead of sendmail ?

RT 4.2 removed the ability to speak directly to an SMTP server -- as
operating in such a configuration meant that outgoing mail was lost if
there was ever connectivity problems to the SMTP host.  A local MTA is
meant to solve precisely that problem, which is why RT relies on it.

In short, that's no longer a supported configuration, to stop users from
shooting themselves in the foot.

Taking a step back, you really should look at your MTA's configuration
and error logs.  All of the workarounds and questions you're asking are
about RT -- but RT is not actually the cause of your problem.  Solve
your actual problem, and the symptoms will go away.
 - Alex


Re: [rt-users] Including CSS on template - RT

2015-04-29 Thread Alex Vandiver
On Wed, 29 Apr 2015 17:41:34 -0300 Murillo Azambuja Gonçalves
muri...@ifi.unicamp.br wrote:
 Hello,

Please do not cross-post your question to both rt-users and rt-devel.
rt-users is the appropriate venue for this question.

 [snip]
 But the template accuses an error when I use CSS:

This is because templates use { and } to delimit code that should be
run as perl.  As such, you'll need to escape your brace characters;
either use:

   a \{
   text-decoration: none;
   color: inherit;
   \}

...or:

   a {q{{
   text-decoration: none;
   color: inherit;
   }}}

 - Alex


Re: [rt-users] slowness in RT when creating a ticket

2015-04-28 Thread Alex Vandiver
On Tue, 28 Apr 2015 15:20:44 +0100 ABD EL MALEK BOUBARNOUS
abdelmalekboubarn...@student.emi.ac.ma wrote:
 Hi everyone,

Please stop cross-posting.  Your questions are best suited for
rt-users, not rt-devel.

 I've just deployed RT 4.2 on Apache2 in my localhost machine, I also
 configured it to send mails with sendmail mail transfer agent, since then
 I'm experiencing slowness when creating tickets,(it takes about a minute).
 
 Does anyone know why is that due to ? is it sendmail ?

It is almost certainly due to your mail configuration.  Most folks
doing local testing set in their RT_SiteConfig.pm:

   Set( $MailCommand, 'testfile' );

...which logs mail to a file rather than attempting to send it.
 - Alex


Re: [rt-users] slowness in RT when creating a ticket

2015-04-28 Thread Alex Vandiver
On Tue, 28 Apr 2015 16:43:03 +0100 ABD EL MALEK BOUBARNOUS
abdelmalekboubarn...@student.emi.ac.ma wrote:
 in my configuration $MailCommand is set to 'sendmailpipe', on the other
 hand  CommentAddress, CorrespondAddress and OwnerEmail are all set to my
 personal gmail adress, can this be the source of the slowness ?

Your source of slowness is that your MTA is being slow to accept the
mail. Usually this is because of DNS failure, or that your service
provider does not allow outgoing connections on port 25.  Check your
maillogs.

If you're merely trying to use RT for local testing, use 'testfile'.
 - Alex


Re: [rt-users] rt 2.4.10

2015-04-22 Thread Alex Vandiver
On Wed, 22 Apr 2015 23:22:56 +0200 Nick price np...@hotmail.com wrote:
 When will 4.2.11 be released

We hope to get a release candidate out the door this week.  To be clear,
however, 4.2.11 is not necessary to run on Apache 2.4 -- the
documentation is the only piece that was updated in that regard.  The
configuration instructions I linked to earlier will work just fine on
any version of 4.2.
 - Alex


Re: [rt-users] rt 2.4.10

2015-04-22 Thread Alex Vandiver
On Wed, 22 Apr 2015 18:12:27 + Nick Price np...@hotmail.com wrote:
 I have been told that the documentation shipped with 4.2.10 isn't OK to
 use with apache 2.4

The upcoming 4.2.11 documentation has been extended to provide
a configuration that works on Apache 2.0, 2.2, and 2.4.  Until the
documentation on https://docs.bestpractical.com/ is updated with the
information from the release, you can read the documentation directly
from git:

https://github.com/bestpractical/rt/blob/stable/docs/web_deployment.pod#apache

 - Alex


Re: [rt-users] Theme editor in 4.2.10 says GD is disabled or not installed.

2015-04-10 Thread Alex Vandiver
On Fri, 10 Apr 2015 12:25:02 -0500 k...@rice.edu k...@rice.edu wrote:
 Okay, I just checked and once I upload a supported logo image type the error
 goes away. Since it was there before doing anything, it threw me off and I 
 thought that there was a problem. Thank you for the feedback.

This is fixed in 4.2-trunk and will be in 4.2.11, for reference:

https://github.com/bestpractical/rt/commit/c51612a1
https://issues.bestpractical.com/Ticket/Display.html?id=30563

 - Alex


Re: [rt-users] RT::Extension::Announce post install question

2015-04-08 Thread Alex Vandiver
On Wed, 8 Apr 2015 15:36:31 + Karres, Dean kar...@illinois.edu
wrote:
 Hi,
 
 I am running RT 4.2.10.  I just used cpan to install
 RT::Extension::Announce and I looked at the resulting install messages.

Installing using the cpan or cpanm command-line clients will never run
the make initdb step.  The new documentation (to be included in
4.2.11, the first rc of which should be out within a week) for
installing plugins calls this out:

https://github.com/bestpractical/rt/blob/stable/docs/extensions.pod#installing-extensions

 - Alex


Re: [rt-users] RT 4.2 DB access through DB session pooler?

2015-04-02 Thread Alex Vandiver
On Thu, 2 Apr 2015 09:30:13 -0500 k...@rice.edu k...@rice.edu wrote:
 We are working on an upgrade from RT 3.8.x to RT 4.2.x to get to a
 supported version of RT and take advantage of the new features and
 enhancements. In order to manage the PostgreSQL connections, we are
 using pgbouncer. Pgbouncer has several types of pooling that can be
 used: session, transaction and statement. The 'transaction' method
 re-uses existing DB connections on a transaction (BEGIN/COMMIT)
 boundary, while 'session' uses the same DB connection for each
 fastCGI RT instance.
 
 With RT 3.8.x we initially tried to use 'transaction' pooling, but
 ended up needing to move to 'session' pooling because updates were
 ending up in the wrong RT ticket. Is 'transaction' pooling not
 usable? It would really help our multi-instance support if we could
 use it instead of 'session'. If it can be used, is there something
 that needs to be done configuration-wise to have it work. Thank you
 for any information that you can give.

RT doesn't make sufficient use of database transactions that I think
transaction will be reliable; I expect session is the best you can
get.  I'm not aware of any configuration knobs that would affect this;
fixing it would take some internals development.
 - Alex


Re: [rt-users] HTML - text conversion

2015-03-31 Thread Alex Vandiver
On Tue, 31 Mar 2015 09:31:38 +0300 Vlad Shpolyanskiy
vlad.shpolyans...@gmail.com wrote:
 [warning]: Could not find or run external 'w3m' HTML formatter in $PATH
 (/sbin:/bin:/usr/sbin:/usr/bin) -- you may need to install it or provide
 the full path (/usr/local/lib/perl5/site_perl/RT/Interface/Email.pm:1849)

This means that your system does not have the binary installed.  You
will need to yum install w3m or apt-get install w3m so that your
system has it.  Having the perl module that knows how to talk to w3m
does no good if w3m itself is not installed.

If you have suggestions for how the wording of the error message quoted
above could be any clearer, I'd appreciate feedback.

 Exactly, you are right. But RT falls back to internal converter as it does
 not see w3m installed for ever reason.

If the w3m binary is already installed, but not
in /sbin, /bin, /usr/sbin, or /usr/bin (the $PATH listed above), then
you'll need to set $HTMLFormatter to the full path to the w3m binary
itself -- not the .pm file.

 - Alex


Re: [rt-users] HTML - text conversion

2015-03-30 Thread Alex Vandiver
On Mon, 30 Mar 2015 13:18:03 +0300 Vlad V. Shpolyanskiy
vlad.shpolyans...@gmail.com wrote:
 But while adding to RT_siteconfig string
 Set($HTMLFormatter, '/usr/local/lib/perl5/site_perl/HTML/FormatText/W3m.pm');

If you want to use w3m, all you need is:

Set($HTMLFormatter, 'w3m');

The reference to setting a full path is if the w3m _binary_ is not in
your PATH.  Setting it to the .pm file is incorrect.  Also note that
since w3m is the first program that RT will attempt to use, there is
generally no need to explicitly set $HTMLFormatter to w3m _at _all_
unless you wish to use a binary in an unusual location.

 The main idea is to avoid double newlines for outlook emails.
 I'll appreciate any advice regarding this.

You may be interested in
https://docs.bestpractical.com/RT_Config#CheckMoreMSMailHeaders

 - Alex


Re: [rt-users] Change transaction creator

2015-03-25 Thread Alex Vandiver
On Wed, 25 Mar 2015 03:36:38 -0700 (MST) andriusk
andrius.kul...@gmail.com wrote:
 I want to change the creator of the transaction, if it is done lets say via
 REST, in other words if creator is REST_USER change it to USER_1.
 
 Is that possible? I guess that the Creator is set in RT::Record-Create
 method?

Stock RT does not support doing things on behalf of other users, via
the REST interface or standard web UI.  Doing the create via the perl
API is the only way to do so.
 - Alex


Re: [rt-users] Mute notify requestor on resolve? (almost got it!)

2015-03-25 Thread Alex Vandiver
On Wed, 25 Mar 2015 10:11:14 -0700 Roman Massey romanmas...@gmail.com
wrote:
 Hello,
 [snip]
 Any tips will be greatly appreciated. Thanks!

Move the Notify scrip to being in stage Batch.  This will mean it won't
appear in the preview of outgoing notifications on the ticket Reply
page, but will have access to the complete, updated, ticket when it
runs.
 - Alex


Re: [rt-users] HTML emails from CLI

2015-03-19 Thread Alex Vandiver
On Wed, 18 Mar 2015 10:44:43 + Eierschmalz, Bernhard
bernhard.eierschm...@scheppach.com wrote:
 Is there any way to create Tickets with HTML email from CLI?

rt create -t ticket -ct text/html

 - Alex


Re: [rt-users] Search Query Builder Issues (Space Encoding?)

2015-03-19 Thread Alex Vandiver
On Thu, 19 Mar 2015 11:20:51 -0400 Richard Stevens
piratef...@gmail.com wrote:
 Hello,
 
 We recently upgrades to RT 4.2.10 / RT-IR-3.2.0 and I'm seeing intermittent
 issues on search commands involving custom fields with spaces:
 [snip]

The spaces are a red herring.  The actual difference is that the
working one includes Queue=Incident+Reports.  Where is the latter
query being generated from?
 - Alex 


Re: [rt-users] RT 4.x multi-instances

2015-03-16 Thread Alex Vandiver
On Sat, 14 Mar 2015 13:30:54 -0400 Gaston Huot gas...@huot.me wrote:
 Hello.
 
 Is there something new about running 2 RT-4.x instances on the same
 installation ?
 Most comments are quite outdated, including the wiki.
 
 Someone doing it using version 4.x ?

Just install into two directories and use one of the fastcgi solutions;
mod_perl does not support multiple instances.
 - Alex


Re: [rt-users] script to check out if max attachment size is reached

2015-03-06 Thread Alex Vandiver
On Fri, 06 Mar 2015 12:42:38 +0100 Loïc Cadoret lcado...@keyyo.com
wrote:
 Greetings rt-users followers,
 
 I would like to create a script that check out if the max attachment 
 size of a response is reached and if so, send an autoreply to the 
 requestor informing him that his documents are too heavy. Indeed, 
 customers send documents to our customer services but sometimes the 
 transaction is droped by the system because max attachment is reached 
 and neither  the customer service or the client is notified of it.

Starting in RT 4.2.7, RT records explicit transactions when attachments
are dropped or truncated due to hitting $MaxAttachmentSize.  While we
explicitly prevent such transactions from triggering scrips (to prevent
infinite loops), you can likely use them as the basis of your
notifications -- or, at least the signal to the RT users to notify
customers of the drop.

 - Alex


Re: [rt-users] Issues with Article Permissions

2015-03-06 Thread Alex Vandiver
On Thu, 5 Mar 2015 12:06:55 +1100 Matthew Crozier
matt...@staff.broadbandsolutions.com.au wrote:
 I'm having an issue when attempting to create a new article as the
 root (SuperUser rights) account. It simply says Permission denied

Check your error logs.  Running rt-validator --check might also be a
good idea.
 - Alex


Re: [rt-users] RT saves data in quoted-printable, why???

2015-03-05 Thread Alex Vandiver
On Thu, 5 Mar 2015 18:50:57 +0100 Palle Girgensohn
 * Ensure that all data containing non-ASCII is quoted-printable
 encoded for PostgreSQL, instead of merely all data not claiming to be
text/plain
 
 Why is this? It seems it is doing more harm than good -- it makes it
 harder to use indexes and makes it harder to use it from the tables
 directly? PostgreSQL is very capable of storing any kind of character
 set? We use UTF-8 for everything, this fix breaks a lot of things for
 us.

The commit in question is 3a9c38ed, which changes:

} elsif (!$RT::Handle-BinarySafeBLOBs
   $MIMEType !~ /text\/plain/gi
   !Encode::is_utf8( $Body, 1 ) ) {
  $ContentEncoding = 'quoted-printable';
}

...to:

} elsif (!$RT::Handle-BinarySafeBLOBs
   $Body =~ /\P{ASCII}/
   !Encode::is_utf8( $Body, 1 ) ) {
  $ContentEncoding = 'quoted-printable';
}

That is, any data which claimed to be text/plain would blindly be
attempted to be shoved into the database; this includes content which
contained _invalid_ UTF-8 byte sequences.  The commit was in RT 4.0; RT
4.2 is much better about transcoding to real UTF-8, or marking the part
as application/octet-stream if it cannot be decoded.  In that sense,
this logic is now less necessary.  However, the commit was absolutely
necessary at the time to not _lose_ data.  Erring on the side of data
preservation, at the expense of possibly-unnecessary encoding, seems
like not an unreasonable choice.

That being said, the Encode::is_utf8() check attempts to verify that we
only quoted-printable encode data whose bytes are not currently a
correctly-encoded UTF-8 byte stream.  The bug now lies there, as after
the utf8-reckoning branch (2620658..af9fe7c), the $Body argument is
guaranteed to contain bytes, not characters.  Per the Encode::is_utf8()
documentation:

[INTERNAL] Tests whether the UTF8 flag is turned on in the STRING. If
CHECK is true, also checks whether STRING contains well-formed UTF-8.
Returns true if successful, false otherwise.

The UTF8 flag is almost certainly off on $Body (it _may_ be on, and
still contain only codepoints  128, but this is unlikely), so the
well-formed-ness of the string (which is the relevant thing we wish to
confirm) is never checked.

I _believe_ that this code may be replaced by (untested):

} elsif (!$RT::Handle-BinarySafeBLOBs
   !eval { Encode::decode( UTF-8, $Body, Encode::FB_CROAK); 1 } 
) {
  $ContentEncoding = 'quoted-printable';
}

I may push a branch later that makes this change, and see what breaks.



All of that aside, note that it makes it harder to use indexes makes
little sense -- there are no indexes on Content.  The full-text index
uses its own tsvector, which it constructs after decoding the Content,
so the transfer encoding of the Content column is irrelevant to that.

 - Alex


Re: [rt-users] Upgrade to 4.2.9 Lost Dashboards.

2015-03-05 Thread Alex Vandiver
On Thu, 5 Mar 2015 14:52:39 -0500 Alex Vandiver
ale...@bestpractical.com wrote:
 On Wed, 25 Feb 2015 11:52:37 + Sam Maher
 sam.ma...@lawsonlewisblakers.co.uk wrote:
  I have moved to a new server and upgraded to the latest version but
  all my privileged users except one have lost their dashboards. Has
  anyone else had this problem?
 
 Please specify what you mean by latest version.

I missed that the subject specified you upgraded to 4.2.9.  What
version were you upgrading from?
 - Alex


Re: [rt-users] sphinx weirdness [explanation and possible solution]

2015-03-05 Thread Alex Vandiver
On Wed, 25 Feb 2015 13:54:45 +0100 Arkadiusz Miśkiewicz
ar...@maven.pl wrote:
  Ok, mysql is too smart! For sphinx to work mysql needs to first
  query AttachmentsIndex_3 and then make joins to it. Otherwise
  sphinx won't work. That's due to sphinx architecture.
  
  Here mysql is too smart and differently optimizes query thus
  breaking sphinx support in rt.
  
  Now I've tried FORCE INDEX and such but wasn't able to force mysql
  to first query AttachmentsIndex_3.
  
  There is STRAIGHT_JOIN that forces joins orders, so maybe that is
  some solution.

This limitation is unfortunately documented, and not easily fixable:
https://bestpractical.com/docs/rt/latest/full_text_indexing.html#Caveats1

 STRAIGHT_JOIN also won't work since mysql still is able to make
 changes and optimizations to the query.
 
 Fortunately code below seems to be working - using UNION and separate,
 simple sphinx subquery:
 [snip]

That query is incorrect; it unions Attachment ids (from the Sphinx
results table) with Ticket ids (from the ti subquery on Tickets).

 Devs, could you please change querying code, so that sphinx will
 always get its own subquery?

Please try the straight MySQL FTS, included in 4.2.10, instead.  It is
much faster, and not nearly as fiddly.
 - Alex


Re: [rt-users] Upgrade to 4.2.9 Lost Dashboards.

2015-03-05 Thread Alex Vandiver
On Wed, 25 Feb 2015 11:52:37 + Sam Maher
sam.ma...@lawsonlewisblakers.co.uk wrote:
 I have moved to a new server and upgraded to the latest version but
 all my privileged users except one have lost their dashboards. Has
 anyone else had this problem?

Please specify what you mean by latest version.  Are the dashboards
listed in Home → All Dashboards?
 - Alex


Re: [rt-users] RT saves data in quoted-printable, why???

2015-03-05 Thread Alex Vandiver
On Thu, 5 Mar 2015 14:31:56 -0500 Alex Vandiver
ale...@bestpractical.com wrote:
 I _believe_ that this code may be replaced by (untested):
 
 } elsif (!$RT::Handle-BinarySafeBLOBs
!eval { Encode::decode( UTF-8, $Body,
 Encode::FB_CROAK); 1 } ) {
 $ContentEncoding = 'quoted-printable';
 }
 
 I may push a branch later that makes this change, and see what breaks.

https://github.com/bestpractical/rt/compare/4.2-trunk...4.2%2Fless-qp

 - Alex


Re: [rt-users] relative date search

2015-03-05 Thread Alex Vandiver
On Fri, 06 Mar 2015 06:48:53 + Alex Peters a...@peters.net wrote:
 You can enter 14 days ago into the Created box, or
 
 Created  14 days ago
 
 into the Advanced screen of the search builder.

Note that despite how it reads in English, 'Created  14 days ago'
means tickets that were created _more_ than 14 days ago.  That is, it
is read as Take the point in time 14 days ago; find tickets whose
creation date is less than (prior to) that point.

The connotation of the word ago in combination with less than /
greater than is often a point of confusion.
 - Alex


Re: [rt-users] RT saves data in quoted-printable, why???

2015-03-05 Thread Alex Vandiver
On Fri, 6 Mar 2015 00:06:32 +0100 Václav Ovsík vaclav.ov...@i.cz
wrote:
 https://issues.bestpractical.com/Ticket/Display.html?id=29735

Aha -- thanks for digging that out!  I thought I vaguely recalled
something in this area previously.
https://issues.bestpractical.com/Ticket/Attachment/286095/157750/utf8-encoding.patch
looks to be functionally fairly similar to the branch.

There are a few other, orthogonal fixes in there that may still be
interesting to tease out into their own commits.  It looks like I see
changes to:

 * Fix the computed max size of base64'd attachments; I'd need to
squint at it harder, but seems eminently reasonable.

 * Attempt to gracefully deal with TruncateLongAttachments truncating
mid-byte of UTF-8 data.  As above; the decode/encode is an interesting
trick to attempt to ensure that the byte stream is consistent.  I'd
like to test it a bit, but seems not unreasonable.

 * Choose base64 vs QP based on which is shorter; I'm less convinced by
this, since it means that for large data, it gets QP'd, base64'd, and
then one of those _again_ -- which isn't terribly efficient.  I'm less
convinced by the tradeoff of computation time to stored in-database
size.

If you're interested in reworking the patch into a 2-3 commit series,
I'm happy to apply for 4.2-trunk.
 - Alex


[rt-users] [rt-announce] RT-Extension-Assets 1.04

2015-03-02 Thread Alex Vandiver
Assets 1.04 -- 2015-03-02
-

We are pleased to announce Assets 1.04:

https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.04.tar.gz
https://download.bestpractical.com/pub/rt/release/RT-Extension-Assets-1.04.tar.gz.asc

SHA1 sums

c933c7e8f9779a711021599ab38d3b011d655180 RT-Extension-Assets-1.04.tar.gz
08b6642151c32d172f854f2dd34d508a9b8b69a5 RT-Extension-Assets-1.04.tar.gz.asc

Changes from 1.02:
 * Keep values of date CFs if asset creation fails (due to CF values that fail
   validation, or other reasons)
 * Speed up asset asset searches with multiple limits on the same role;
   this primarily effects installs with the RT-Extension-MergeUsers
   extension installed.

A complete changelog is available from git by running:
git log 1.02..1.04
or visiting
https://github.com/bestpractical/rt-extension-assets/compare/1.02...1.04
___
rt-announce mailing list
rt-annou...@lists.bestpractical.com
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce


Re: [rt-users] Asset for RT 1.0.2 and rt-crontool

2015-03-02 Thread Alex Vandiver
On Mon, 2 Mar 2015 19:46:36 +0300 Andrey K a...@rambler.ru wrote:
 Is it possible to use rt-crontool to change a status of an asset depend on 
 it's custom field?
 I'd like to create Expiration data custom field to change status of assets 
 if data expired.

Assets doesn't have support for triggered actions (scrips) yet.  You
can override the AddCustomFieldValue method on RT::Asset to accomplish
this manually.
 - Alex


Re: [rt-users] RT 3.8.8 goes down intermittently

2015-03-02 Thread Alex Vandiver
On Mon, 2 Mar 2015 14:14:20 + Calafiore, Jason [USA] 
calafiore_ja...@bah.com wrote:
 I am experiencing some issue when RT 3.8.8

RT 3.8 reached end-of-life nearly a year ago:
http://blog.bestpractical.com/2014/04/rt-38-reaches-end-of-life.html

RT 3.8.8 was released nearly 5 years ago, and has multiple published
security vulnerabilities against it, including SQL injection and
arbitrary execution of code.  Upgrade -- which will also likely fix the
bug you're seeing.

 - Alex


Re: [rt-users] Asset for RT 1.0.2 and rt-crontool

2015-03-02 Thread Alex Vandiver
On Mon, 2 Mar 2015 21:33:12 +0300 Andrey K a...@rambler.ru wrote:
 So, is it impossible to act on assets like tickets from rt-crontool?

As of now, yes.  rt-crontool is still ticket-centric, in that its
search predicates only return tickets, and its actions are still
Scrips' actions, which assume they act on tickets.

Writing a small amount of code to iterate over assets and modify them
is fairly simple, however.
 - Alex


[rt-users] [rt-announce] RT 4.2.10 released

2015-02-26 Thread Alex Vandiver
RT 4.2.10 -- 2015-02-26
---

RT 4.2.10 contains important security fixes, as well as minor bugfixes.

https://download.bestpractical.com/pub/rt/release/rt-4.2.10.tar.gz
https://download.bestpractical.com/pub/rt/release/rt-4.2.10.tar.gz.asc

SHA1 sums

92af386e9c09a0e9489ec1cd55b66c65b77d22be  rt-4.2.10.tar.gz
8e65ce02b62df85c7d679dab8d4bde8ef343ec48  rt-4.2.10.tar.gz.asc


This release is primarily a security release; it addresses CVE-014-9472,
a denial-of-service via RT's email gateway, as well as CVE-2015-1165 and
CVE-2015-1464, which allow for information disclosure and session
hijacking via RT's RSS feeds.

As part of these security updates, RT's dependency on the Encode module
has been changed, to Encode 2.64.  If upgrading, be sure to run
rt-test-dependencies to verify that your installed version of Encode
meets this requirement; if not, you will need to install a newer version
from CPAN.

This release is also a bugfix release; most notably, it addresses a bug
which causes RT to generate blank outgoing text/plain parts.  This fix
requires installing the HTML::FormatExternal module, and having an
external tool (w3m, elinks, etc) installed on the server.

It also introduces indexed full-text searching for MySQL without the
need to recompile MySQL to use the external Sphinx tool; instead, a
MyISAM table is used for indexing.  On MySQL 5.6 and above, an
additional InnoDB table can also be used.


The complete list of changes includes:

General user UI
 * Speed up the default simple search on all FTS-enabled installs by not
   OR'ing it with a Subject match.  This returns equivalent results for
   almost all tickets, and allows the database to make full use of the
   FTS index.
 * Pressing enter in user preference form fields no longer instead
   resets the auth token (#19431)
 * Pressing enter in ticket create and modify form fields now creates or
   updates the ticket, instead being equivalent to add more
   attachments, or the search on People pages (#19431)
 * Properly encode headers in forwarded emails that contain non-ASCII
   text (#29753)
 * Allow users to customize visibility of chart/table/TicketSQL in saved
   charts
 * Allow groups to be added as requestors on tickets
 * Perform group searches case-insensitively on People page (#27835)
 * Ticket create transactions for tickets created via the web UI now
   contain mocked-up From, To, and Date headers; this causes them to
   render more correctly when forwarded
 * Update wording of error message for saved searches without a
   description (#30435)
 * Flush TSV download every 10 rows, for responsiveness
 * Retain values in Quick Create on homepage if it fails (#19431)
 * Limit the custom field value autocomplete to 10 values, like other
   autocompletes (#30190)
 * Fix a regression in 4.0.20/4.2.4 which caused some users to have
   blank homepages (#30106)
 * Fix styling on unread messages box on Ballard and Web2 themes
 * Fix format of Date headers in RSS feeds (#29712)
 * Adjust width of transaction date to accommodate all date formats
   (#30176)
 * Allow searching for tickets by queue lifecycle

Command-line
 * Fix server name displayed at password prompt when RT is deployed at
   a non-root path like /rt (#22708)

Admin
 * If the optional HTML::FormatExternal module is installed, use w3m,
   elinks, links, html2text, or lynx to format HTML to text.  This
   addresses problems with the pure-Perl HTML-to-text converted which
   resulted in blank outgoing emails.  (#30176)
 * Add support for native (non-Sphinx) indexed full-text search on
   MySQL.  This uses the InnoDB fulltext engine on MySQL 5.6, and an
   additional MyISAM table on prior versions of MySQL.
 * Support MySQL database names with dashes in them (#7568)
 * Properly escape quotes and backslashes in config options in web
   installer (#29990)
 * Increase length of template title form input
 * Clarify wording on updating old Organization values by rt-validator
 * Resolve a runtime error for SMIME without secret keys (#30436)
 * Empty email addresses are no longer caught as being an RT address
   if there exist queues without Correspond addresses set (#18380)
 * Allow Parents/Children/Members/MemberOf in CreateTickets action
 * Allow RT-Originator to be overridden in templates
 * Ensure that HTML-encoded entities are indexed in FTS
 * Fix uninitialized value warnings from charts grouped by date
 * Remove no-op $CanonicalizeOnCreate configuration variable;
   RT::User-CanonicalizeUserInfo is always called
 * Make NotifyGroup action respect AlwaysNotifyActor argument
 * Fix X-RT-Interface header on incoming email on existent tickets
 * Warn on startup if queues have invalid lifecycles set (#28352)

Developer
 * Add AfterHeaders callback to ShowMessageHeaders
 * Update all upgrade steps to use .in files (#18856)
 * Add policy tests to enforce the new upgrade step standards
 * Remove +x bit from multiple non-executable files
 * Make Obfuscate callback in 

[rt-users] [rt-announce] Security vulnerabilities in RT

2015-02-26 Thread Alex Vandiver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

We have discovered security vulnerabilities which affect both RT 4.0.x
and RT 4.2.x.  We are releasing RT versions 4.0.23 and 4.2.10 to resolve
these vulnerabilities, as well as patches which apply atop all released
versions of 4.0 and 4.2.

The vulnerabilities addressed by 4.0.23, 4.2.10, and the below patches
include the following:

RT 3.0.0 and above, if running on Perl 5.14.0 or higher, are vulnerable
to a remote denial-of-service via the email gateway; any installation
which accepts mail from untrusted sources is vulnerable, regardless of
the permissions configuration inside RT.  This denial-of-service may
encompass both CPU and disk usage, depending on RT's logging
configuration.  This vulnerability is assigned CVE-2014-9472.

RT 3.8.8 and above are vulnerable to an information disclosure attack
which may reveal RSS feeds URLs, and thus ticket data; this
vulnerability is assigned CVE-2015-1165.  RSS feed URLs can also be
leveraged to perform session hijacking, allowing a user with the URL to
log in as the user that created the feed; this vulnerability is assigned
CVE-2015-1464.

We would like to thank Christian Loos cl...@netcologne.de for
reporting CVE-2014-9472 and CVE-2015-1165; CVE-2015-1464 was found by
internal review.

Patches for all releases of 4.0.x and 4.2.x are available for download
below.  Versions of RT older than 4.0.0 are unsupported and do not
receive security patches; please contact sa...@bestpractical.com if you
need assistance with an older RT version.

https://download.bestpractical.com/pub/rt/release/security-2015-02-26.tar.gz
https://download.bestpractical.com/pub/rt/release/security-2015-02-26.tar.gz.asc

aac58bf3aa6d918dbefbaa2b27a9694f27b32d58  security-2015-02-26.tar.gz
6abe9a58400db3ee2cdbdf17704f0d881d90d744  security-2015-02-26.tar.gz.asc

The README in the tarball contains instructions for applying the
patches.  If you need help resolving this issue locally, we will provide
discounted pricing for single-incident support; please contact us at
sa...@bestpractical.com for more information.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iEYEARECAAYFAlTvSZMACgkQMflWJZZAbqCj5gCgwmXReEL+TIUYrAzfTl0aj0rr
+ZIAn2Uq8K12j3r+se6yZlg/B6myoJSM
=kSeJ
-END PGP SIGNATURE-
___
rt-announce mailing list
rt-annou...@lists.bestpractical.com
http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-announce


Re: [rt-users] RT 4.2.6 Need a scrip that will check email address and not send email multiple times

2015-02-24 Thread Alex Vandiver
On Tue, 24 Feb 2015 14:42:23 + Daniel Moore daniel.mo...@osbornewood.com 
wrote:
 Example, I have a user that belongs to an exchange group that is currently 
 the email for a contractors group that I have set up. He receives multiple 
 emails when he is requestor, admin cc and so on.

Use one scrip with a Notify action that notifies all of the folks you
care about.  This isn't the default because AdminCc's get different
email than requestors.
 - Alex


Re: [rt-users] [SOLVED] Restoring from backup - admins lost privileges

2015-02-24 Thread Alex Vandiver
On Fri, 13 Feb 2015 18:41:45 + Andrew Wippler 
andrew.wipp...@lancasterbaptist.org wrote:
 After a few hours of sleep, I managed to get it working with this code:
 [snip]

For posterity, this is not the solution you want.  If your users did
not come across as Privileged, there are likely a huge slew of other
hidden problems.  Attempting to paper over it via this method is
dangerous -- you just made every email address that has ever sent your
RT spam into a Privileged user.

Fix your database backups.
 - Alex


Re: [rt-users] RT install, apache problem.

2015-02-24 Thread Alex Vandiver
On Thu, 19 Feb 2015 18:06:05 -0500 Joseph Mays m...@win.net wrote:
 I have an installation or RT4.2 under Apache 2.4 on FreeBSD 10.1.

Sorry -- we've not updated the documentation for Apache 2.4 yet.

Replace:
Order allow,deny
Allow from all
..with:
Require all granted

Or use the mod_access_compat module.  See
http://httpd.apache.org/docs/2.4/upgrading.html#access

 - Alex


Re: [rt-users] Upgrade fail

2015-02-06 Thread Alex Vandiver
On Fri, 6 Feb 2015 13:44:06 -0500 Joshua Lansford 
joshua.lansf...@laserlinc.com wrote:
 Ok.  So I have done some additional investigation and here is what I have
 found.  The new Queues table has two new fields named Lifecycle and
 SubjectTag.  Lifecycle field is accessed in the RT::Init which is called at
 the top of action_insert before it can do the insert portion of the first
 upgrade step... I think.  It is looking like the code is assuming stuff
 about the database structure which isn't true just in trying to set up to
 do the upgrade steps.

Please show the complete output of 'make upgrade-database', not just a
small snippet.

 I suppose this means I am trying to jump too many versions at once and I
 need to do an intermediate upgrade at an intermediate step.  Probably at
 the version which introduces these columns.  If that version couldn't have
 upgraded everyone would have gotten stuck.

It shouldn't be necessary to to the upgrade in multiple steps.
 - Alex


Re: [rt-users] changing ticket owner causes crash

2015-02-05 Thread Alex Vandiver
On Thu, 5 Feb 2015 12:37:03 + Alexander Reintzsch
alexander.reintz...@netsystem.de wrote: 
 I have a problem, which seems to exist for quite some time in request
 tracker.

You didn't saw what version of RT you're running.

 Here an old description:
 http://www.gossamer-threads.com/lists/rt/users/41122
 
 My Problem seems to be the same.
 
 The user that is the owner of the ticket had been deactivated.
 Afterwards the ticket had to get a new owner. 
 I wanted to do this by changing the owner in the »People« menu of the
 ticket. This failed.
 
 The log showed:
 [8647] [Thu Feb  5 12:24:19 2015] [warning]: Couldn't delete cached
 group submember 29891 (/opt/rt4/sbin/../lib/RT/GroupMember.pm:343)

This is a symptom of database corruption.  Did you modify your
database by hand, or have disk problems at any point?

Back up your database, then run rt-validator --check --resolve
 - Alex


Re: [rt-users] Problem with attachments

2015-02-04 Thread Alex Vandiver
On Wed, 4 Feb 2015 18:26:36 + Guadagnino Cristiano
guadagnino.cristi...@creval.it wrote:
 Thank you Alex,
 so basically my test is correct and my problem is not solved.
 Back to the drawing board :-(
 
 Let's see if I can give some more detail, to help you in helping
 me :-)

Those two mails are sent by different scrips.  Check that the template
for the latter scrip has RT-Send-Attachment: yes in the headers
section.

 - Alex


Re: [rt-users] Change to SSL cert breaking incoming mail

2015-02-04 Thread Alex Vandiver
On Wed, 4 Feb 2015 19:18:30 + asas a...@uw.edu wrote:
 It is a trusted CA, and the cert for the CA is present on the server.
 That's why I'm so baffled by the problem - the whole cert chain works
 fine with the web interface.

Have you tried passing the CA, and it didn't work, or did you note try
yet? Perl may be working from a different certificate store than your
web browser.

Alternately, try upgrading Net::SSLeay and LWP::Protocol::https.
  - Alex


Re: [rt-users] Problem with attachments

2015-02-04 Thread Alex Vandiver
On Wed, 4 Feb 2015 11:43:23 + Guadagnino Cristiano
guadagnino.cristi...@creval.it wrote:
 I disabled CustomFieldsOnUpdate by commenting out the 
 Plugin('RT::Extension::CustomFieldsOnUpdate'); line in 
 RT_SiteConfig.pm, then I stopped Apache, cleared the mason cache, and 
 restarted Apache.
 
 I tested resolving a ticket adding an attachment, but had the same 
 behavior as before.
 
 I then also disabled the Announce extension by following the same 
 procedure, but still I had no success.
 
 Those are the only two extensions I added recently (other extensions
 are loaded but they have been there for years).
 
 Is it sufficient to disable the extensions by commenting them out in
 the RT_Siteconfig.pm file or should I completely uninstall them? If
 so, how do I do it?

Removing the Plugin line should mostly suffice; removing the files from
disk does no better.  Fully removing a plugin requires backing out
database contents that relay on it, (like the RTAnnounce) queue, but
that's not relevant for this test.
 - Alex 


Re: [rt-users] New cert breaks mailgate

2015-02-03 Thread Alex Vandiver
On Tue, 3 Feb 2015 16:27:02 -0500 Mitch Kyser mky...@albion.edu wrote:
 Thanks for the response. We tried that and could not get it to work
 either.  Turns out our CA is pretty old and still running on a 2003
 box.
 
 We were going to roll out RT to our staff first who all use domain
 machines that include our root CA cert already.  The web portion
 worked fine.  We were going to let our students eventually start
 sending requests and planned to get a commercial cert.  This just
 pushed the time up a few months.  So now we have a new project,
 upgrade our CA.

Try upgrading the LWP::Protocol::https and the Net::SSLeay modules.
 - alex


Re: [rt-users] rt-crontool EROR

2015-02-03 Thread Alex Vandiver
On Tue, 03 Feb 2015 12:09:48 + Mauricio Leite Ferreira da Silva
mauricio.le...@planalto.gov.br wrote:
 Does anybody Know what can be happening?

Upgrading to 4.2.7 or higher will resolve
https://issues.bestpractical.com/Ticket/Display.html?id=22991 and allow
you to see the actual error.  Alternately, you can apply
https://github.com/bestpractical/rt/commit/625e7b99.patch to your
bin/rt-crontool

 - Alex


Re: [rt-users] Problem with attachments

2015-02-03 Thread Alex Vandiver
On Tue, 3 Feb 2015 15:43:05 + Guadagnino Cristiano
guadagnino.cristi...@creval.it wrote:
 I did this: I took a backup of my production RT and restored it on
 our testing environment. I perused rt-validator till I had no more
 warnings (well, I still have a few warnings related to articles: it
 seems rt-validator cannot fix them). Then I created a test ticket and
 resolved it attaching a file.
 
 As in our production system, the attachment did not get sent, and the
 last line in RT's log was again this:
 
 
 [15371] [Tue Feb  3 15:28:32 2015] [warning]: Couldn't load object
 RT::Transaction #0 (/opt/rt4/sbin/../lib/RT/Interface/Web.pm:3026)
 
 Any hint?

Try disabling the extensions you have installed and see if removing any
of them fixes it.  My suspicion is on CustomFieldsOnUpdate, from the
previous list you gave.

 - Alex


Re: [rt-users] Request Tracker

2015-01-29 Thread Alex Vandiver
On Wed, 28 Jan 2015 12:29:00 +0200 Andrius Kulbis
andrius.kul...@ktu.lt wrote:
 Maybe you can look through my question? 
 http://requesttracker.8502.n7.nabble.com/Uploading-image-to-Image-CustomField-via-REST-td59382.html

Support on the mailing list is provided as we have time to provide it.
We support development of RT via support contracts; if you have
need of a line to Best Practical to reliably answer your questions, you
can chat with sa...@bestpractical.com.  Going to personal mail
(repeatedly) is a bit rude -- doing so may _decrease_ the chance that
your question is answered.

Regardless, it looks like the code for setting CF values only deals
with the Value part of a CF, not the LargeContent part, which is
where file upload CF and image CFs are stored.  So no, it looks like it
doesn't.
 - Alex


Re: [rt-users] RT 4.2.8 reply emails go out blank

2015-01-29 Thread Alex Vandiver
On Thu, 29 Jan 2015 11:50:25 -0700 (MST) juniorsa gilb...@dido.ca
wrote:
 On certain email replies the mail arrives blank to the recipient.

This is caused by failure of the HTML - Text rendering engine.  It
triggers on email (or comments/correspondence) which need to be
embedded into plain-text templates; the most common failing case is
HTML that includes tables.

We're testing a branch (4.2/html-external-formatter) which we expect to
be in 4.2.10; it addresses the problem by adding an optional dependency
(HTML::FormatExternal) which uses an external text-only browser
(such as w3m or elinks) to render the HTML to text.

 - Alex


Re: [rt-users] Connecting to remote MySQL via SSL

2015-01-29 Thread Alex Vandiver
On Thu, 29 Jan 2015 10:59:27 +0100 Nathan Cutler
presnyprek...@gmail.com wrote:
 I have been tasked with migrating our RT database (MySQL) to a remote
 server, connection between RT and the database server must be SSL
 encrypted.
 
 I see at [1] that DBD::mysql supports SSL, but the RT documentation
 [2] seems to imply that RT only supports SSL with PostgreSQL:
 
 $DatabaseRequireSSL
 
 If you're using PostgreSQL and have compiled in SSL support, set
 $DatabaseRequireSSL to 1 to turn on SSL communication with the
 database.

From a quick skim of the DBIx::SearchBuilder::Handle code[1], that looks
to add requiressl=1 to the DSN -- which, even on Pg, doesn't do
anything, since the current form is sslmode=require[1].  That option
should die, and be replaced by a more generic one that allows one to
append arbitrary things to the DSN -- particularly as enabling SSL
isn't really an on/off bit, as one needs to talk about verification of
the certificate in order for it to provide any security.

 What is the status of out-of-the-box support for SSL with MySQL in RT
 4.2?

Add a local overlay of RT::Handle::BuildDSN to add the relevant DSN
attributes.  A future release will provide rope to do this without an
overlay.
 - Alex

[1]
https://github.com/bestpractical/dbix-searchbuilder/blob/master/lib/DBIx/SearchBuilder/Handle.pm#L152-L171

[2] https://metacpan.org/pod/DBD::Pg#connect

 Thanks,
 Nathan
 
 [1] https://metacpan.org/pod/DBD::mysql#mysql_ssl
 [2]
 https://www.bestpractical.com/docs/rt/4.2/RT_Config.html#DatabaseRequireSSL



Re: [rt-users] Cancel Resolve transaction based on custom fields

2015-01-29 Thread Alex Vandiver
On Wed, 28 Jan 2015 14:27:22 +0100 Jasper Olbrich
jasper.olbr...@students.uni-marburg.de wrote:
 Is it possible to suppress the status change from open to resolved
 with a scrip or to include the scrip-induced status change in the
 result box?

Unfortunately, neither of these is possible in stock RT.  Scrips can
only run _after_ a chance has happened, and thus have no chance of
preventing it.  They also cannot currently push their results to the
user.
 - Alex


Re: [rt-users] Problem with attachments

2015-01-29 Thread Alex Vandiver
On Wed, 28 Jan 2015 11:53:53 + Guadagnino Cristiano
guadagnino.cristi...@creval.it wrote:
 Unfortunately we send attachments very rarely, so I cannot exactly
 say when this behaviour started. It could be that it started from the
 day I upgraded our installation from 4.2.3 to 4.2.9 (in december), or
 later when I added/upgraded some extensions
 (RT-Extension-Announce-1.00, rt-extension-customfieldsonupdate,
 RT-Extension-SpawnLinkedTicketInQueue-1.01).

You may wish to try disabling those extensions temporarily to see if
that helps resolve the issue.  However, see below.

 I just finished examining RT's log (in debug mode), regarding a test
 ticket I created and then resolved adding an attachment. There is no
 error at all in the log, let  alone a warning message that says
 
 [warning]: Couldn't load object RT::Transaction #0
 (/opt/rt4/sbin/../lib/RT/Interface/Web.pm:3026)
 
 but I don't think this is a real problem, since we had have this type
 of message for a long time with no apparent symptoms (please reply if
 I'm wrong).

That warning is extremely worrisome, and I expect the cause of your
problem.  You should run `/opt/rt4/sbin/rt-validator`.

 - Alex


Re: [rt-users] RT 4.2.8 reply emails go out blank

2015-01-29 Thread Alex Vandiver
On Thu, 29 Jan 2015 15:01:07 -0500 Gilbert Rebeiro gilb...@dido.ca
wrote:
 Is there any work around for now?

Switching to HTML templates is one solution.  This will only mask the
issue somewhat, however, as the plain-text alternative part will be
blank, and the text/html will appear fine.

 How would users know their messages went out blank?

There's a warning in the logs.  The end-users click on the Show
Outgoing Email link, if it's available to them, to see what RT sent
out.

 Any idea on when 4.2.10 will be released?

I expect a release candidate next week, with the official 4.2.10 a
couple weeks thereafter.
 - Alex


Re: [rt-users] RT Installation

2015-01-27 Thread Alex Vandiver
On Tue, 27 Jan 2015 21:31:26 +0100 Rainer Duffner
rai...@ultra-secure.de wrote:
 But I would be tempted to use a „clean“ Perl installed somewhere else
 as the basis for RT even in the case of FreeBSD (which I know best,
 admittedly).

See https://docs.bestpractical.com/rt_perl

 How difficult would it be for BestPractical to tar up such a custom
 perl install done in e.g. /opt/rt/perl, together with all the
 dependencies and throw in a „complete“ install of RT+RTIR with
 FastCGI-support?

The problem is in the binary libraries that RT (and its CPAN
dependencies) use.  Any such deploy would be specific to one particular
release of a particular distribution, and as such would be quite
resource-heavy to produce in sufficient variety to be useful.
Shipwright is a BPS tool that builds relocatable installs of
software that package everything above glibc, but that doesn't quite
solve the right problem.

We've begun pondering building Docker vessels for RT, but the database
dependency complicates matters.

 While historically, the mod_perl installation is the
 most common one

These days, we suggest fastcgi over mod_perl, even for Apache.

 the FastCGI installation can work with nginx, which
 faster and also decouples the web server from the installed version
 of Perl (I don’t know if you can have and RT-installation and
 mod_perl compiled against different versions of perl for example).

You generally cannot.  This is one reason we prefer fastcgi deployments.
 - Alex


  1   2   3   4   5   >