Developers of Sympa

[IKEDA Soji <address@concealed>]

Hi Marc and all,

On Wed, 23 Jan 2013 15:26:35 +0100
Marc Chantreux <address@concealed> wrote:

> … and still f****ing slow! 
> 
> hello everybody.
> 
> here we are, stuck with a very up to date sympa 6.1 which is too slow.
> we really have some ideas to speed up the things but it means a lot of
> code. I don't want to fork sympa 6.1 and i see that the renater instance
> is still on 6.2.

6.2 branch incorporated db_list_cache branch and some improvements
by me.

List.pm:

o List::get_lists() takes charge of querying lists.
  It is based on the idea Riseup patch had been realized.

  Additionally,

  - Querying and sorting are available on extended number of list
    parameters.

  - Perl regexp or SQL RLIKE (extension by MySQL) is no longer used
    for search on list subjects.  Substring matches are performed
    by fc() and index().

    # Fold-cased search keys are stored into database field.
    # On Perl earlier than 5.12.6, Unicode::CaseFold external module
    # is used to support fc().

  - Now get_which() is a thin wrapper of get_lists() above.  SQL
    queries on subscriber/admin_table and list_table were unified
    to single query with JOIN condition.

  - In addition to binary file (LIST_DIR/config.bin), list config
    may be cached into database BLOB field.
    It can reduce the time to scan under list_data directory.

    # Set "config_list_cache" robot parameter to be "database".

o Introducing accessor methods, abstraction of List parameters.

  - By my profiling, Scenario objects were heavily reloaded during
    Lists objects are loaded.
    Now Scenario objects will be loaded when they are really
    needed, if they are accesssed through accessors.

  - Default parameter values will no longer saved into LIST_DIR/config
    file, LIST_DIR/config.bin nor list_table cache.

wwsympa.fcgi:

o Several search operation on lists was made faster thanks to
  List::get_lists() above.

o Limiting number of "Your lists" side menu.  Pages can be loaded
  faster.
  # This will be effectual in the case that someone (e.g. the system
  # administrator) is an owner of almost all lists of the site.


Issues NOT yet solved:

o Searching users.
  It has not supported substring match.

o Slow "List of lists" page
  Currently WWSympa trys to load all lists.  This behavior is a big
  performance hit on the site with hundreds or many of lists.

o Codes to handle robot config parameters and config files would be
  refactored and improved, however, I shall not be able to finish
  it in this winter (if I did).

o Bi-directional layout support by CSS.
  My private experiment using CSSJanus showed promising result,
  however, slight changes on code will be nessessary.

o ..., etc.


> We need to find a solution fast but we have to decide what's our next
> step before monday: maintaining a fork or test the 6.2 upgrade. what
> would be your move? any good idea to share? 

I'm all right to swich 6.2 as a release candidate.

My planned work on 6.2 branch has almost over.  Unsolved issues above
would be postponed.


> regards

Regards,

-- Soji


> -- 
> Marc Chantreux
> Université de Strasbourg, Direction Informatique
> 14 Rue René Descartes,
> 67084  STRASBOURG CEDEX
> ☎: 03.68.85.57.40
> http://unistra.fr
> "Don't believe everything you read on the Internet"
>     -- Abraham Lincoln


-- 
株式会社 コンバージョン  セキュリティ&OSSソリューション部   池田荘児
〒231-0004　神奈川県横浜市中区元浜町3-21-2 ヘリオス関内ビル7F
e-mail address@concealed  TEL 045-640-3550
http://www.conversion.co.jp/