The mailing list for listmasters using Sympa

[JOSE MANUEL ROLDAN HERRAIZ <address@concealed>]

Hi

So we are migrating from sympa 6.1 to 6.2 and, since we are changing the whole infrastructure, we would like to be able to scalate this service horizontally.
We took a look at the manual Adam linked (   http://www.sympa.org/manual/server_architecture_for_big_services ) but since it's outdated (last update was on 2010) and sometimes confusing ("spools stored in a database") we almost gave up. But this post gave us hope again :-)

I woud like to know if it is feasible to export via NFS and share the spool folder between two servers, both running sympa.pl, wwsympa and bulk.pl processes. The expected behaviour is that when a message is sent to a large list, both of the server's bulk processes could simultaneously take chunks of it and send the mails instead of only the bulk processes from one of the servers.

Also, is there a way to set how many mails each bulk.pl processes each time?

Thank you very much.
José


Quoting IKEDA Soji <address@concealed>:

Adam and folks,

On Thu, 14 Jul 2016 14:20:33 -0700
"Adam Bernstein" (via sympa-users Mailing List) <address@concealed> wrote:

Quick question for the Sympa folks, unless anyone else has gotten this
working:

Largely in response to the exact slowdown mentioned below when enabling
the merge_feature, we're working on spreading the sending load over more
servers. The only document I can find is this one:
   http://www.sympa.org/manual/server_architecture_for_big_services

It's rather old, but still I think mostly relevant, and I'm thinking
that moving bulk.pl onto a different server is most likely our best
option. (We could just split the MTA by using relay hosts, but I'd
rather go further.) And I want to confirm one thing: We can run bulk.pl
at the same time on *multiple* servers, yes? There won't be any
contention or locking issues or duplicated sends or anything?

Theoretically yes.  Why I say "theoretically" is that I have not
heard of any cases on production systems.  It may not necessarily
mean "no".


Sympa 6.0 and 6.1 used spools based on database tables.  Exclusive
processing over multiple processes or servers was intended to be
guaranteed by lock fields in central database.

However, (in my understanding) it turned out that overhead by
database access and query execution can make operations very
inefficient, even on single server.

Sympa 6.2 introduced spools based on filesystem: Spools shared by
NAS / SAN were supposed.  Exclusive processing was intended to be
guaranteed by file locking thanks to File::NFSLock module.  It
satisfactorily works at least on single server with multiple
processes.

However, availability beyond single server has not been proven based
on certain evidence.


In short, it is worth trying on clustering with Sympa 6.2 and shared
spool.  I look forward the good news :-).


Regards,

-- Soji

I believe the fact that multiple bulk.pl child processes can run on the
same server means the answer is yes, but just want to be sure.

Thanks!
         adam

On 7/13/2016 1:44 PM, Steve Rich wrote:
We implemented this and I regret it almost daily.  The first problem we
encountered was what Steve mentioned with ESPs checking links and auto
unsubscribing so I wrote an interstitial custom action that prevented
auto unsubscribes by link checking.  Secondly, delivery performance took
a nose dive as a result of not being able to bundle recipients of the
same domain.  My problems went from "where is my unsubscribe link" to
"why do I have to click so many times" and "why is my email delayed".

Thanks,
Steve

--
Electric Embers Cooperative
Handcrafted hosting, powering the fires of change
electricembers.coop
(800) 843-6197 / (415) 992-6916
 

--
株式会社 コンバージョン  セキュリティ&OSSソリューション部   池田荘児
〒140-0014 東京都品川区大井1-49-15 アクセス大井町ビル4F
e-mail address@concealed  TEL 03-6429-2880http://www.conversion.co.jp/

Jose Manuel ROLDAN HERRAIZ
Departamento de Correo Electrónico