Skip to Content.
Sympa Menu

en - Re: increasing SMTP concurrency, was: [sympa-users] SMTP rate limiting

Subject: The mailing list for listmasters using Sympa

List archive

Chronological Thread  
  • From: Serge Aumont <address@concealed>
  • To: Micah Anderson <address@concealed>
  • Cc: address@concealed
  • Subject: Re: increasing SMTP concurrency, was: [sympa-users] SMTP rate limiting
  • Date: Thu, 30 Oct 2008 06:23:24 +0100

Micah Anderson a écrit :
I'm actually looking to do the opposite of what was originally posted on
this thread.

In our setup, we have a few lists with a very large number of
subscribers. When a post is sent to one of these lists, any other
sympa processing is blocked until it completes delivery to the MTA (in
our case, postfix). On the below example, there are almost 60k
subscribers, and the hand-off from sympa to postfix takes 2843 seconds
(about 47 minutes):

If I am correct, what this means is that for 40 minutes there are no
other sympa operations, so all other messages that are in the queue,
or that arrive in the queue during this period are blocked by this

Sympa.conf distribution_mode fork option will make sympa fork, one process analyse incomming messages forward messages to editor, answers to command message etc. The second distribute message. It better but only one message distribution process is running at time.

Why does it take so long, and how can I either speed up this process
or parallelize it?

I have increased sympa.conf's maxsmtp 200 from 100 (as well as
postfix's smtp process limits), but that didn't change anything in the
The requested delay is comming from the number of system call to sendmail or postfix. maxsmtp if the maximum number of "sendmail" processus called by Increase this number will make distribution faster only if many of thoses processus stay a lot of time in the machine. This is possible when using sendmail and when thoses processus are managing very slow smtp sessions. In your case (postfix) the maxsmtp parameter is not the problem because postfix just queue messages and finish very fast.
You should increase nrcpt parameter. nrcpt is the number of recipient as argument of each sendmail/postfix call. The number of processus that sympa have to run is #subscribers divided by nrcpt . (if you don't use verp).
Sendmail as a limit for that parameter. If you put a hight value for it each process become too complex and slow. Postfix is better for that so you may increase it. Let's say from 25 to 100 or more.
Is there some postfix guru that can confirm the value we can use with postfix ?

We did a major change in Sympa bulk mailer this summer. Next Sympa version 6.0 will parallelize this process and it will be possible to start a new message distribution for a hight priority list even if so hugue distribution process is started.

Let us known if nrcpt change is a good solution for you.

Archive powered by MHonArc 2.6.19+.

Top of Page