The mailing list for listmasters using Sympa

[Sergiy Zhuk <address@concealed>]

hi

On Thu, 25 Aug 2005, Edward Rynes wrote:

> What is the practical limit on the number of mailing lists that Sympa
> can manage?

Depends on how much power you have on your server.

> We love Sympa as a list management tool but as the number of lists
> grows (now over 500) it's performance is slowing dramatically.

Memory consumption is one of the culprits.
Every sympa daemon is caching all list configs, so the more lists you have,
the more memory you will need, e.g. with about 7,800 lists the memory
footprint per sympa process is about 500Mb.
You also need a huge amount of CPU to process all that data, especially upon
start-up, when it takes several minutes to populate the cache.
Second problem is queue processing, which if left untouched, will re-read
the sympa mail queue after delivering every message, which doesn't work very
well if you have large number of messages in the queue.
And another problem is archiving, which is very inefficient for lists with
high traffic (2,000+ messages a month).
At the end of the month the archival queue can grow enormously, since it
takes forever to archive messages for high traffic lists.
It can take days to process that backlog.
You can change queue processing so it doesn't re-read queue dir every time,
but the first problem remains and is not easy to solve.
Theorethically you would either use a shared cache or change how you treat
the data, so you don't need caching at all.
Another problem is that you can't scale the system by adding more servers,
since sympa is not designed to work in that environment.
Ideally you would have several delivery boxes, several web front-end boxes
and a shared network storage like a netapp.
Unfortunately, this has never been a priority for sympa developers and it
requires significant changes.
I hope this will change, since I'm at the point where I can't scale the
system by adding more RAM or CPUs.
It's just not practical any more...

thanks

-- 
rgds,
serge