The mailing list for listmasters using Sympa

[Tom Fillmore <address@concealed>]

Hi, there -

Thanks very much for this info, it was really useful in my troubleshooting efforts.  Lots of good ideas and hints. :)  Unfortunately I could not get fcgid to work nearly as quickly as fastcgi so I think I will stick with fastcgi for the moment...

Despite all this great info and my best efforts I could not make the problem go away completely, but I think I have a better understanding as to why one particular problem is still happening (see 'Lingering Problem') but I think it's more of a Perl/bug issue than anything (and I'm not a perl guy).

Results:
* On my stock Debian 8.1 64 bit (all patched) fastcgi works better than fcgid.  Hands down.  I suspect it has to do with the version of fcgid that comes with the distro.  I've used fcgid in the past without issue and it is very stable.

* I had to turn the list cache on.  This helped a lot.

* Changing tabs to spaces in sympa.conf increased the speed.  This is weird but it was noticeable.  I came on this one when I changed the email of the listmaster in sympa.conf then restarted sympa and it did not pick up the changes.  I always test on two different browsers (firefox and chromium) and got the same result from each after clearing their cache.  Going back into sympa.conf and changing the tab to spaces on the listmaster email line then restarting the process picked up the change in both browsers.  Waaa???  Weird.  Anyway, for good measure I replaced all the tabs in the file with spaces and things sped up a little more largely because the cpu load never rose as high.  Now I know that really shouldn't make a difference, but it did and I'm not going to complain about it.  :)


Lingering problem:
* I continue to get consistent 404 errors when it tries to grab the Raleway fonts.  It eventually finds them after 5-6 seconds but why it has problems I don't know.  From the log:

        "GET /fonts/Raleway/Raleway-Regular.ttf HTTP/1.1" 404 524 "http://mydomain.email/static-sympa/css/style.css"

I think that if this issue is resolved somehow that will be the bulk of the solution for me.  It feels like a loop is eventually timing out then the process moves on to grab these files.  Once it 'finds' them everything else is returned almost immediately and cpu usage drops like a stone.  If this is already fixed in a later solution (assuming it's not me) please accept my apologies.  I'm on 6.2.3 and want to iron out as much as possible before an upgrade.  As part of the review for this problem I confirmed that all the files are owned sympa:sympa, folders are 755 and files are 644.  If this has been fixed can you reply with which version the fix showed up in?


In response to my request for some working apache configs, here is what works in my setup for a virtual host.  I run Debian 8.1.

Notes:
* Debian scatters content in a few files.  I chose to place them all in the file for a specific 'site' for sanity sake
* Nothing is in apache2.conf
* Your mileage may vary + the usual note about how it may not work for you, etc....

-------------------------------------------------
<VirtualHost 123.123.123.123:80>

        ServerName domain.email
        ServerAlias www.domain.email

        ServerAdmin address@concealed
        DocumentRoot /var/www/html


        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined


        Alias /static-sympa /home/sympa/static_content

        <Directory /home/sympa/static_content>
                Require all granted
        </Directory>


        <Directory /home/sympa/bin>
                Require all granted
        </Directory>


        FastCgiServer /home/sympa/bin/wwsympa-wrapper.fcgi -processes 2


        <Location /wws>
                SetHandler fastcgi-script
        </Location>


        ScriptAlias /wws /home/sympa/bin/wwsympa-wrapper.fcgi

</VirtualHost>
-------------------------------------------------


Thanks very much, I appreciate all the help.

Tom Fillmore
southern California

On 09/25/2015 01:45 AM, David Verdin wrote:

address@concealed"> Hi Tom,

Le 25/09/2015 00:14, Fillmore, Tom a écrit :

Hi, all -

I am at present running a server with about 50 lists and 10,000 users.  Until I got to about 30 lists the web interface loaded quickly, but it has slowed down to a crawl as more lists have been added.  Now its takes no less than 20 seconds to reply, and ultimately causes 500 errors about one-half the time.  During that time CPU usage always goes over 95%. 

I am using Apache2, fcgid and have the value for that set to '1' in sympa.conf.  I've reviewed my Apache configs and they seem to be OK.

Machine specs:  Dual core mac mini, Debian 8.1, MySQL, 6GB RAM.  It's on a virtualbox VM, which some might say is the problem, but in the past I've run on VirtualBox vm's with the same basic configuration and triple the number of lists, users and traffic.

You should not have any trouble.
I'm running a server with 8 GB, 8 CPU cores (but mainly one is actually used) on an old RHEL5. It is enough for 2,400 lists, 16 virtual hosts, 225,000 users and 360,000 subscriptions. It's on a VMware VM.

Based on what I've read online in posts + the docs it seems that the problem lies in one or more of these areas:
1) The new web interface is massively slower
I don't think so. Everything funky in the web interface is actually run in _javascript_, thus impacting the web client only. So it has no impact on cpu / memory usage on the server.
2) There is something wrong with my Apache configs
It's a possibility. Could you show us the part regarding fcgi?
3) There is a problem with the cgi code that generates the web interface
Also a possibility. But I should investigate what'w wrong and why I can't reproduce it.
4) fcgid is a dog, speed-wise
:-)
I never noticed it before.

Out of these three items I really can only control 2 & 4

I'm open to all ideas, but can a few folks send me their Apache config files that work?  Please be sure to indicate the location where they are installed (eg /etc/apache2/... , /home/sympa/..., etc)
OK. Here is what I have:

in /etc/httpd/conf/httpd.conf

## FastCGI
LoadModule fcgid_module modules/mod_fcgid.so

Alias /static-sympa /home/sympa/static_content

<IfModule mod_fcgid.c>
    IPCCommTimeout 120
    MaxProcessCount 30
    SocketPath /var/run/fcgid/sock
  </IfModule>

ScriptAlias /sympa /home/sympa/bin/wwsympa-wrapper.fcgi


And we have separate files for each virtual host:

/etc/httpd/conf.d/domain.tld:80

<VirtualHost <IP>:80>

RedirectMatch (/wws.*)$ https://domain.tld$1
Alias /static-sympa/css/domain.tld /home/sympa/static_content/css/domain.tld

## Redirecting all towards HTTPS
RedirectMatch ^(.*)$ https://domain.tld$1

DocumentRoot "/var/www/domain.tld"
ServerName domain.tld:80
ServerAdmin email@ddress

CustomLog logs/domain.tld/access_log combined
ErrorLog logs/domain.tld/error_log
CustomLog logs/access_log combined-with-vhost

RedirectMatch ^/$ http://domain.tld/sympa

</VirtualHost>                                 

/etc/httpd/conf.d/domain.tld:443

<VirtualHost <IP>:443>
DocumentRoot "/var/www/secure"
ServerName domain.tld:443
ServerAdmin email@ddress

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" **%T/%D**" combined_timed

CustomLog logs/domain.tld/access_log combined_timed
ErrorLog logs/domain.tld/error_log
CustomLog logs/access_log combined-with-vhost

<Location /sympa>
   SSLOptions +StdEnvVars
</Location>

Alias /static-sympa/css/domain.tld /home/sympa/static_content/css/domain.tld

RedirectMatch /wws/(.*) https://domain.tld/sympa/$1
RedirectMatch ^/$ https://domain.tld/sympa
Redirect /wws https://domain.tld/sympa

SSLEngine on

SSLCertificateFile /path/to/cert
SSLCertificateKeyFile /path/to/key
SSLCertificateChainFile /path/to/chain
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>

</VirtualHost>                                 


Hope this helps.

Regards,

David

Other than this problem everything is working just fine.

Thanks!

Tom Fillmore

--
A bug in Sympa? Quick! To the bug tracker!

David Verdin Études et projets applicatifs
Tél : +33 2 23 23 69 71 Fax : +33 2 23 23 71 21   www.renater.fr	RENATER 263 Avenue du Gal Leclerc 35042 Rennes Cedex

Attachment: pngL92dEPXoph.png
Description: PNG image