Developers of Sympa

[address@concealed]

Salut tout le monde,

My proposition for implementing a tie'd @lists array has not yet been tested 
in 
in our production environment (lists.riseup.net) - hopefully over the 
holidays 
we will have the time to do so.  Of course, we will share the results of this
test with other developers.  We might have PGP integration soon, as well.  =)

As for the hard numbers as to the limits of Sympa.  It is my opinion that
this mailing list manager is close to being very scalable.  We currently 
operate a server hosting over 10,000 mailing lists with 2.5 million 
subscribers.

Our Sympa is different from the upstream version in just a few ways; one is 
welcome to see our changes at https://labs.riseup.net/code/projects/show/sympa
(using 'git' as the source control).  

The biggest change, the change which enables us to run Sympa on commodity 
hardware, is caching the configurations for list objects in a RDBMS.  Note,
we have not moved away from storing the list configurations in files.  The 
RDBMS merely stores the list information required to SELECT a a subset of 
lists-  allowing our wwsympa.fcgi processes to requesting an array 
of only lists that are 'closed', for example.

There are some uses of get_lists that are inconvenient to refactor to operate
in this manner - task_manager often needs to iterate through all of the lists
to perform some functions.  The tie'd @lists array mentioned above is a method
I am eager to test in a real environment.

I can provide a diff of our sources with the upstream upon request.

Seasons Greetings,
Charles Paul



On Tue, Dec 23, 2008 at 12:09:14PM +0100, Olivier Sala?n wrote:
> I complete Serge's response with optimization hypothesis we need to explore.
> We have noticed that Sympa processes gets fat sometimes ; Charles Paul 
> has made a proposition we need to test, see 
> <http://sourcesup.cru.fr/tracker/?func=detail&atid=170&aid=4787&group_id=23>
> 
> Serge Aumont a ?crit :
> >Martin, Nicolas wrote:
> >>
> >>Hi,
> >>
> >>Concerning the scalability of Sympa, the only information I found is 
> >>that it can undergo "up to 100.000 subscribers" per list. My 
> >>questions are :
> >>
> >This information is not up to date. Some guys recently distributes a 
> >big message (110 Ko) to 750.000 subscribers using Sympa version 5.4.3 
> >with 2 servers : one dedicated to Sympa and one to Sendmail 
> >distribution. The delay for Sympa process was 2h30, and during this 
> >initial process more than 90% of the messages was sent up to the 
> >destination MTA (99% in 4h) ; the MTA was already in many white list.
> >
> >Using Sympa 6.0 with its new bulk mailer we could do it in much more 
> >better way because the repartition of the distribution process on 
> >defferent servers will be very easy.
> >>
> >>-> is there a maximum number of lists / host ?
> >>
> >Yes there is a maximum of lists because lists configs are stored on 
> >files and list object are stored in memory. We don't known exactly 
> >this number. Our own server have more then 1000 lists, we known 
> >servers with 10000. may be some can share their experience ?
> >>
> >>-> is there a maximum number of messages / list / day ?
> >>
> >The limit for this is probably very high. I think the real limit will 
> >not comes from Sympa itself but from remote antispam and may be also 
> >because subscribers will not accept so many messages ;-)
> >>
> >>-> Any other kind of limitation ?
> >>
> >The real limitation is the one I described first : lists object are 
> >stored in memory (I mean list configs, not subscribers). This 
> >limitation is mainly related to the web interface where the 
> >distribution process could support extremely high number of lists, 
> >subscribers and messages. We are working on that the first step is to 
> >move spools in database. That's what does Sympa 6.0.
> >
> >Serge Aumont
> >
>