Re: [rt-users] RT 4.4.1 and transaction isolation level on Postgres

[Alex Vandiver]

On Mon, 2 Jan 2017 17:12:29 +0100
Václav Ovsík  wrote:
> Can anybody confirm on different system?

Thanks for the detailed replication instructions.  I can replicate, and
have tracked down a minimal replication case.  I'll drop my findings
and suggestion on your ticket.

The short form is that this is due to 4.4's new PreviewScrips
functionality, which simulates (then rolls back) all of the changes,
which is racing with the actual change.  The bad news is that it's the
real change, not the dry run, which gets killed in the deadlock
detector, meaning that the owner does go unchanged.  At least this is
bubbled up to the user in the ticket display page, but this is still
quite unfortunate.

Amusingly, there are some strong parallels to the canonical Therac-25
case[1] -- the race here requires that one trigger a PreviewScrips very
quickly before submitting, which is rare except with users quite
accustomed to the UI.  In the Therac-25 case, only skilled users
could navigate to the bottom of the form within 8 seconds and thus
deliver lethal doses of radiation to their patients.

...RT is designed to be robust, but there's a reason it doesn't rate
itself as meant to be used for safety- or life-critical applications.

 - Alex


[1] https://en.wikipedia.org/wiki/Therac-25

[rt-users] RT 4.4.1 and transaction isolation level on Postgres

[Václav Ovsík]

Hi,
to reproduce the deadlock problem with RT 4.4.1 on Postgres I tried to install
RT 4.4.1 as clean new instalation:
1. On Debian 8.6 stable + Postgres 9.4
2. On Debian sid (unstable) + Postgres 9.6 (there was a need to
   change one sql command, option NOCREATEUSER -> NOCREATEROLE)

To my surprise I created ticket in queue General (default) and the bug
appeared immediately when I tried to change owner of a ticket and record
comment together.

Can anybody confirm on different system?

Steps:
1. Create Postgres cluster (server instance):

pg_createcluster 9.6 main --locale cs_CZ.UTF-8

2. Change access to rt4 database, so I don't need to set password for rt_user
   in /etc/postgresql/9.6/main/pg_hba.conf

diff --git a/postgresql/9.6/main/pg_hba.conf 
b/postgresql/9.6/main/pg_hba.conf
index b708a885..85857f64 100644
--- a/postgresql/9.6/main/pg_hba.conf
+++ b/postgresql/9.6/main/pg_hba.conf
@@ -86,6 +86,8 @@ local   all postgres  
  peer
 
 # TYPE  DATABASEUSERADDRESS METHOD
 
+local  rt4 rt_user trust
+
 # "local" is for Unix domain socket connections only
 local   all all peer
 # IPv4 local connections:

3. Start Postgres & check

pg_ctlcluster 9.6 main start
pg_lsclusters 

4. Install RT 4.4.1 from RT sources...

cd rt-4.4.1-sources
./configure --prefix=/opt/rt  --with-db-type=Pg
sudo make install

   Change db host, so the connection will go through unix domain socket

cd /opt/rt
cat <<'EOT' >etc/RT_SiteConfig.pm
Set($WebPort, );
Set($DatabaseHost, undef);
Set($DatabaseRTHost, undef);
1;
EOT

   Postgres user needs to read schame etc

chmod a+r etc/{acl,initialdata,schema}*

   Postgres 9.6 no longer support NOCREATEUSER, but NOTCREATEROLE
   (https://www.postgresql.org/docs/9.6/static/release-9-6.html)

sed -i -e 's/NOCREATEUSER/NOCREATEROLE/;' etc/acl.Pg

   Initialize database (if something went wrong, remove not finished database
   using 'sudo -u postgres dropdb rt4' and try better again)

sudo -u postgres /usr/bin/perl -I/opt/rt/local/lib -I/opt/rt/lib 
sbin/rt-setup-database --action init 

5. Start RT built int HTTP server by hand:

sudo -u www-data /opt/rt/sbin/rt-server --port 


6. Go to browser and type location localhost:

- login as root / password
- click on "New ticket in" General, enter something into subject and message
  and click "Create"
  (Owner is nobody, requestor is root)
- Go Actions / Comment, type something into message and change owner
  to root. Click "Update ticket"
- If owner was changed, try repeat previous step switching owner
  between nobody & root, until error.

My session:

bobek:/opt/rt# sudo -u www-data /opt/rt/sbin/rt-server --port 
[21132] [Mon Jan  2 15:58:50 2017] [info]: Successful login for root from 
127.0.0.1 (/opt/rt/sbin/../lib/RT/Interface/Web.pm:831)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 #1/31 - Scrip 7 On Create 
Autoreply To Requestors (/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:285)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 sent  To: root@localhost 
(/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:316)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 #1/31 - Scrip 8 On Create 
Notify Owner and AdminCcs (/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:285)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 No recipients found. Not 
sending. (/opt/rt/sbin/../lib/RT/Interface/Email.pm:806)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 #1/31 - Scrip 9 On Create 
Notify Ccs (/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:285)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 No recipients found. Not 
sending. (/opt/rt/sbin/../lib/RT/Interface/Email.pm:806)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 #1/31 - Scrip 10 On Create 
Notify Other Recipients (/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:285)
[21133] [Mon Jan  2 16:00:21 2017] [info]: 
 No recipients found. Not 
sending. (/opt/rt/sbin/../lib/RT/Interface/Email.pm:806)
[21133] [Mon Jan  2 16:00:21 2017] [info]: Ticket 1 created in queue 'General' 
by root (/opt/rt/sbin/../lib/RT/Ticket.pm:567)
[21132] [Mon Jan  2 16:02:37 2017] [info]: 
 not sending to 
root@localhost, creator of the transaction, due to NotifyActor setting 
(/opt/rt/sbin/../lib/RT/Action/SendEmail.pm:901)
[21133] [Mon Jan  2 16:02:48 2017] [info]: