The mailing list for listmasters using Sympa

[Roy Bixler <address@concealed>]

Hello:

I am new to this list and have some questions about possible patches
we could apply to improve Sympa scalability.  The items of most
concern are responsiveness of the Web interface and also the SOAP
interface.  The latter has only become a requirement in the past
month.  One of our developers has written a portal application and he
hopes to call the SOAP interface to do things such as create a new
list or list subscribers of an existing list.

What we have noticed is that, at least on the initial loading of the
server suite, the programs must crawl the disk for configuration
information of all of the lists.  We currently have about 6000 lists,
so this can take anywhere from about 3 minutes for a local disk
configuration to about 15 minutes if we use NFS.  I think that FCGI
tuning may alleviate some of these problems.

I have also looked into applying some patches which may help to rely
less on crawling the disk for configuration data.  First, we have a 2
node environment.  One node is at version 5.3.4 and runs the Sympa
suite to process mail and the other node is at version 5.4.7 and runs
wwsympa.fgci for the Web interface.  The data is shared between the 2
via NFS and we also use a remote MySQL database.

We used to suffer from slow Web performance, especially when listing
all of the lists, but we improved that by using binary configuration
files and by cutting down on the size of the session tables.  Now that
we have to concern ourself with the SOAP interface as well, I wonder
if we can make further improvements on performance for operations like
listing the lists and and listing the subscribers of lists.

I have looked at the available patches and see that there is one which
uses the Perl "tie" feature to improve caching or that there is
another larger patch from riseup.net which stores more configuration
information in a database.  I also note a proposal to use "memcached"
as a cache of list configuration.  It seems like the riseup.net patch
is most promising, but I suppose that the others might be acceptable
if the loading of list configuration data is only slow on startup.  I
mainly wonder which of these approaches seems the most consistent with
future versions of Sympa.  Also, if anyone has some pointers on FCGI
tuning or other comments, that would be welcome.

Thanks,

-- 
Roy Bixler <address@concealed>
IT Services / Collaborative Systems