Developers of Sympa

[Roy Bixler <address@concealed>]

On Thu, Dec 22, 2011 at 06:15:13AM -0600, Charles Paul wrote:
> Forwarding this email from sympa-users to sympa-dev.  Roy, you may
> wish to consider joining sympa-dev to discuss scalability issues.  I
> helped write the Riseup patches.

Thank you for that.

It looks like Google mangled my post a little, so I'll add spaces back
to improve readability.

> """
> 
> Hello:
> I am new to this list and have some questions about possible patches we
> could apply to improve Sympa scalability. The items of most concern are
> responsiveness of the Web interface and also the SOAP interface. The
> latter has only become a requirement in the past month. One of our
> developers has written a portal application and he hopes to call the
> SOAP interface to do things such as create a new list or list
> subscribers of an existing list.
> What we have noticed is that, at least on the initial loading of
> the server suite, the programs must crawl the disk for
> configuration information of all of the lists. We currently have about
> 6000 lists, so this can take anywhere from about 3 minutes for a local
> disk configuration to about 15 minutes if we use NFS. I think that
> FCGI tuning may alleviate some of these problems.
> I have also looked into applying some patches which may help to
> rely less on crawling the disk for configuration data. First, we have a
> 2 node environment. One node is at version 5.3.4 and runs the
> Sympa suite to process mail and the other node is at version 5.4.7 and
> runs wwsympa.fgci for the Web interface. The data is shared between the
> 2 via NFS and we also use a remote MySQL database.
> We used to suffer from slow Web performance, especially when
> listing all of the lists, but we improved that by using binary
> configuration files and by cutting down on the size of the session
> tables. Now that we have to concern ourself with the SOAP interface as
> well, I wonder if we can make further improvements on performance for
> operations like listing the lists and and listing the subscribers of
> lists.
> I have looked at the available patches and see that there is one
> which uses the Perl "tie" feature to improve caching or that there
> is another larger patch from riseup.net which stores more
> configuration information in a database. I also note a proposal to use
> "memcached" as a cache of list configuration. It seems like the
> riseup.net patch is most promising, but I suppose that the others might
> be acceptable if the loading of list configuration data is only slow on
> startup. I mainly wonder which of these approaches seems the most
> consistent with future versions of Sympa. Also, if anyone has some
> pointers on FCGI tuning or other comments, that would be welcome.
> Thanks,
> --?Roy Bixler?< address@concealed>IT Services / Collaborative Systems"""
> 

-- 
Roy Bixler <address@concealed>
IT Services / Collaborative Systems