Developers of Sympa

[Marc Chantreux <address@concealed>]

On Thu, Jan 24, 2013 at 10:45:03AM +0100, David Verdin wrote:
> What's slow exactly? I don't remember reading the details about
> which operation on your server takes so long.

i haéd some nytprof reports there

http://ramirez.u-strasbg.fr/~mc/nytprof

this one is very instructive

http://ramirez.u-strasbg.fr/~mc/nytprof/r.nytprof.out.12544/nytprof/

so i dumped a config.bin to figure out what's de pb.. We haven't
investigated so much for the moment but i guess this file store much too
much things because it is called for almost everything so in most cases,
much of the data are just useless.

also, maintaining a pending list queue could fix the timeout we have
when we are asking for pending lists.

> I don't remember reading your different configuration settings,
> server characteristics and overall lod: number of lists, users, etc. 

> There are probably things to be done to improve performances, beyond
> changing the code - which certainly ALSO a solution.

we investinguated a lot on this way and yet we have some improvements.
but it is still way too slow for our users.

the next step would be to store the config.bin files locally (NFS for
the moment).

> We should probably cache something here, but I don't see how.

I have some ideas about it. having a lazy load of parameters in a redis
cache could be a good solution. also remove all the lock tests and use
visudo system to update mailing lists configuration by hand could help.

> 1- either we cache the visibility result for each user and each list
> in database. That's a lot of entries to store in DB  - and to keep
> up to date.

using a DB for cache seems really awkward for me. but yeah: we agreed on
the visibility parameter

> 2- either we change the way visibility is defined. Switching to a
> larger granularity, such as yes/no/subscribers only could probably
> improve performances on the list of lists. The thing here, is to NOT
> load the full lists.

as i said: i haven't investiguate much but i really think this is the
main issue! lazy, granular loading of the parameters.

> >I'm all right to swich 6.2 as a release candidate.
> I will have to disagree with this proposal.
> Even if most of the work is done, I still have too much crashes on
> my developemnt server to release a beta.

ok: so let us have our branch and begin to test and fix the problems on
our dev. instance of sympa.

i really think we have only two options here:
* fork/continue sympa 6.1 to add granularity and/or cache methods
* switch to 6.2 on our dev/test instance and help you to fix things. i
  would really prefer this solution. so if you don't want to release an
  RC, please consider giving us clear instructions to help on sympa 6.2
  fixing.

can we have a phone call on it ? it would speed up the discution.

regards
-- 
Marc Chantreux
Université de Strasbourg, Direction Informatique
14 Rue René Descartes,
67084  STRASBOURG CEDEX
☎: 03.68.85.57.40
http://unistra.fr
"Don't believe everything you read on the Internet"
    -- Abraham Lincoln