The mailing list for listmasters using Sympa

[Patrick Rynhart <address@concealed>]

Hello,

In our production Sympa environment running version v6.1.14 (Élias) we 
have found a message which makes a bulk.pl hang.  Further, it seems to 
stop all lists on our listserv from processing (until we manually kill 
off affected bulk.pl process(es) and restart Sympa services).

Stopping Sympa servers, manually killing the affected bulk.pl process 
with a SIGKILL or “kill -9”, and starting Sympa again fails to clear the 
problem or result in the email being delivered.  We have tried stopping 
Sympa, clearing the lock_bulkmailer entry (by setting this to NULL) and 
starting again.  However, this results in the next bulk.pl process (that 
the message is assigned to) also hanging.  (i.e. the fault is 
reproducible every time).

Symptoms are as follows:

1. An “old” message (which is assigned to a bulkmailer but the bulk.pl 
doesn’t finish) remains in bulkmailer_table (until it is cleaned up by 
the daily Sympa tables clean process):

mysql> select 
delivery_date_bulkmailer,listname_bulkmailer,lock_bulkmailer from 
bulkmailer_table \G;
*************************** 1. row ***************************
delivery_date_bulkmailer: 1384190815
     listname_bulkmailer: stream-git-ci
         lock_bulkmailer: tur-lists230426
1 row in set (0.00 sec)

At the time of writing, the message is 28460 seconds old (~ 7 hours)

$ echo "$(date +%s) - 1384190815" | bc
28460

2. The bulk mailer that the message is assigned to is doing nothing. 
(Inspection via strace -p 30426 shows that the process exists but is 
doing nothing.)

3. Sometimes (upon restarting Sympa) the following error is listed 
indicating that the issue may relate to regular expression(s):

Previous process has died unexpectedly.
Date of crash: 12 Nov 2013  13:40
Errors: Complex regular subexpression recursion limit (32766) exceeded 
at /usr/local/lib64/perl5/Template/Parser.pm line 633.

Check log file for further details.

(Note: There are no further details available in the log file(s).)

4. Upon decoding the (large) Base64 encoded message

mysql> select message_bulkspool from bulkspool_table where 
messagekey_bulkspool = 'ece13b624d536dd23cc4596e9066e2e2' \G;
*************************** 1. row ***************************
message_bulkspool: 
UmV0dXJuLVBhdGg6IDxnaXRAbWFzc2V5LmFjLm56PgpYLU9yaWdpbmFsL…...

it is found that the email contains a lot of regexp.

The decoded gzipped version of the message is available at the following 
location:

http://www.massey.ac.nz/~prynhart/downloads/message_that_makes_sympa_bulk_pl_hang.gz

Would it please be possible for someone to look into what is going wrong 
here ?  I'm hoping that the message causing this problem can be used as 
a 'payload' to make a Sympa bulk.pl process hang, with a view to a fix 
being developed.

Thank you,

Patrick

--
Patrick Rynhart
Systems Engineer
Infrastructure Support Section
Information Technology Services
Massey University
Palmerston North

T: +64 6 356 9099 ext 81075