Developers of Sympa

[David Verdin <address@concealed>]

Hi Soji,

Le 09/09/14 11:38, IKEDA Soji a écrit :

David and all.

On Tue, 09 Sep 2014 10:25:13 +0200
David Verdin <address@concealed> wrote:

Hi Soji,

Le 08/09/14 10:53, IKEDA Soji a e'crit :

Hi developers,

What I worked about in August:

- Adopting Crypt-SMIME to support S/MIME features: OpenSSL is no
  longer mandatory (libcrypto is still needed, though).

Great. So this topic is fixed. I'm happy with it.

- Catching up layout changes in trunk by Guillaume: Still in the
  middle.

Good, that will ease moving bug fixes to the trunk.

- Refactoring about message workflow: As a result, messages may be
  encrypted after personalization (hopefully. I've not made test
  cases).

That's good. Obviously, S/MIME signed messages should not be
personnalized whatever.

In this week, I plan to make refactoring on bulk spool: Switch
from SQL tables to filesystem.  As required functions have already
been prepared, this work will finish in a week.

No. I'm sorry, but this is a veto here.
There is no added value to moving bulk spool to file system for the 6.2
version. In addition, some people already use the database features to
distribute message distribution accross several servers. So moving to
filesystem will force them to change their architecture, get a
distributed filesystem, check for NFS locks and such.
This is clearly a change that is likely to unstabilize the 6.2 branch
and our aim right now is to stabilize it.

I thought you would say that, however, I'll carry out it.

Bulk spool based on SQL tables was the cause of some troubles.

- It often cannot handle large messages properly.  E.g. the lists
  on my company must handle 50 MB or over.

- It cannot handle large number of subscribers properly: The size
  of recipient_bulkmailer field is fixed and don't have enough room
  to keep recipients.

- DKIM private keys are stored into database without appropriate
  protection.

- To prevent the same situation for S/MIME, encryption cannot be
  carried out before personalization.

Note that these are not my private opinion.  These have been pointed
out by several users.

I'm sure you did not get these issues out of a hat. However...
The thing here is that you did not tell why you wanted to move from database to filesystem.
When you make such big moves, you should at least tell us what you plan and why. That is the open door to discussion. Would you have told us these problems before writing about switching to filesystem, we could have made suggestions. I have some:

- increase the size of the field use to store mails.
- increase the size of receipient_bulkmailer. Or change it to a join key to a dedicated table to store receipients
- replacing database storage by filesystem storage for DKIM private key onlys
- For S/MIME, it was not important to have this personnalization working under 6.2.

All these could solve the problems without changing the current functionning. I remember we told about tackling the issue related to filesystem / database in a later version. Why tackling it now? We are all eager to have finally the 6.2 out and this will delay it again. How about migration for previous instances? How will we deal with it?
We need to stop working on it. As soon as the two first columns of the texte matrix are green, it's okay for going beta. The rest of the columns are marginal usages. It can be improved in beta.

Finally, though you have disagreed to my proposal to refactaring
several times because of unstabilization, I cannot support it.

During the last month I made not a little refactoring on codebase.
I have been casually running my test suit and have been confirming
that refactoring did not affect to stability.

Conversely, if Sympa-6.2 is unstable, it was unstable from the past.

Obviously. That's why we are trying to stabilize it. We try for months to get it out so we can finally focus on code improvment.
Listen, I just want to issue the 6.2 so we can have some quiet time to work on the trunk. Let's just focus on that. Please? Pretty please with a cherry on top?

I want the 6.2 ready for beta in two weeks. That leaves us enough time
to fix the remnant bugs, catch up with the version producytion process
(that was borken when I had to install the new Pootle version) and run
the new version in production for a while. It is really important that
we finally issue the 6.2, even if the code doesn't look like you would
like to. Keep in mind that we can add a lot of improvments in future 7.x
version, but the 6.2 is awaited for too long (almost four years!). We
can't wait anymore. That would endanger the whole project.
So please, keep this feature for 7.x version.

I have waited for so long time, too.

I know. sorry about the mess last year.

In the first place, I was invited to this project so that I can
commit DB-based list cache feature by myself, I remember.  But I soon
realized that codebase is so complicated, slightly outdated and partly
buggy, to add new feature.

So I changed my priority and abondoned my orignal plan: I decided to
pour my energies on bug fixes and renewal of codes.  Two years later,
it still continues.

And I'm very grateful for this!
You made a lot to improve Sympa and I know it is better and better greatly thanks to you.
I just want to pinpoint the fact that, someday, It is time to call it a version and release it.

Afterwards, I'll fill up test matrix.  Here is the cases currently
filled by me: http://sympa-ja.org/tests/sympa-tests.tar.bz2

I had a look at the matrix. It looks good. I'm testing it too, so
hopefully, the beta production will speed up.

In this week, I'm making refactoring on bulk spool: Switch
from SQL tables to filesystem.  As required functions have already
been prepared, this work will finish in a week.

Please consider my proposal before coding. Just tell me if I'm wrong and why.
If you make a point, I'll revise my judgement.

Afterward, I'll fill up test matrix.

Best regards,

David

Regards,

--- Soji

Best regards,

David

Regards,

--- Soji

-- 
A bug in Sympa? Quick! To the bug tracker!
<https://sourcesup.renater.fr/tracker/?group_id=23>
RENATER logo
*David Verdin*
E'tudes et projets applicatifs

Te'l : +33 2 23 23 69 71
Fax : +33 2 23 23 71 21

www.renater.fr <http;//www.renater.fr>
	RENATER
263 Avenue du Gal Leclerc
35042 Rennes Cedex

--
A bug in Sympa? Quick! To the bug tracker!

David Verdin Études et projets applicatifs
Tél : +33 2 23 23 69 71 Fax : +33 2 23 23 71 21   www.renater.fr	RENATER 263 Avenue du Gal Leclerc 35042 Rennes Cedex

Attachment: smime.p7s
Description: Signature cryptographique S/MIME