Developers of Sympa

[IKEDA Soji <address@concealed>]

David and all,

On Thu, 07 Nov 2013 15:55:00 +0100
David Verdin <address@concealed> wrote:

> Hi Soji,
> 
> Le 07/11/13 01:07, IKEDA Soji a écrit :
> > On Wed, 06 Nov 2013 16:45:05 +0100
> > David Verdin <address@concealed> wrote:
> >
> >> Hi,
> >>
> >> Le 04/11/13 17:21, Guillaume Rousse a écrit :
> >>> Hello.
> >>>
> >>> Le 03/11/2013 03:50, IKEDA Soji a écrit :
> >>>> Hi,
> >>>>
> >>>> Guillaume's new directory layout seems to have a bit more room for
> >>>> improvement.  For example subdirectories under src/ may be moved to
> >>>> top level.
> >> Are you talking about the trunk or the 6.2 branch?
> >>>> Also expecting registration to CPAN, I drafted a modified layout.
> >>> The desirability of such registration has never been discussed sofar.
> >>> Altough I don't have any actual objection, I don't see much interest:
> >>> - sympa does not belong to the primary target of CPAN (it is an
> >>> application, not a reusable perl library)
> >>> - sympa doesn't use ExtUtils::MakeMaker or Module::Build build system,
> >>> meaning cpan clients won't be able to install it
> >>>
> >>> And even if desirable, there is no constraint on internal archive
> >>> structure.
> >> To sum up: we need to have a correct modularization and get rid of
> >> circular references.
> >> If this leads to porting some modules to CPAN, it is potentially
> >> interesting. But this is not our current priority.
> > "_Also_ expecting registration to CPAN", I wrote.  Expecting it
> > just results the name of a few directories.
> I think we agree on this. I just wanted to catch the occasion to precise 
> my exact thought about CPAN. I'm all for moving modules to CPAN if it is 
> udeful to the community.
> >
> > My proposal _also_ moved some files and directories.  Do you say
> > they are unnecessary (or misled) changes?  Please look at table in
> > the quote below.
> I may have andswered to quickly.
> I like your repartition which looks like the one Guillaume did in the 
> sympa-cleanup branch, except that it is not contained in a single 
> src/Sympa directory.
> 
> Let's have a look a the repartition Guillaume did for sympa-cleanup 
> (simplified version):
> 
> sympa-cleanup/
> |-- several files to be read: AUTHORS, COPYING, ChangeLog, INSTALL, 
> NEWS, README, README.charset
> |-- some scripts: important_changes.pl, sympa.spec.in, tap-driver.sh
> |-- *doc/*: doc
> |-- *ext/*: external modules
> |-- *mail_tt2/*: mail templates
> |-- *t**/*: tests
> |-- *web_tt2/*: web templates
> |-- *po/*: user interface localization
> |-- *po-wwsympa/*: web help localization
> |-- *src**/*: source code
> |   |-- *bin/*: ponctual usage scripts
> |   |   |-- arc2webarc.pl.in
> |   |   |-- arcrepair.pl
> |   |   |-- create_db.Oracle
> |   |   |-- create_db.Pg
> |   |   |-- create_db.SQLite
> |   |   |-- create_db.Sybase
> |   |   |-- create_db.mysql
> |   |   |-- crypt_passwd.pl.in
> |   |   |-- find_missing_messages.pl
> |   |   |-- init_comment.pl.in
> |   |   |-- ldap_alias_manager.pl.in
> |   |   |-- logrotate
> |   |   |-- mod2html.pl.in
> |   |   |-- mysql_alias_manager.pl.in
> |   |   |-- p12topem.pl.in
> |   |   |-- sympa.generic
> |   |   |-- sympa.in
> |   |   |-- testldap.pl.in
> |   |   |-- testlogs.pl.in
> |   |   `-- tpl2tt2.pl.in
> |   |-- *cgi/*: web daemon and associated files
> |   |   |-- INSTALL
> |   |   |-- Makefile.am
> |   |   |-- README
> |   |   |-- mime.types
> |   |   |-- wwsympa-wrapper.fcgi.c
> |   |   `-- wwsympa.fcgi.in
> |   |-- *etc/*: configuration defaults and examples
> |   |   |-- create_list_templates/
> |   |   |-- families/
> |   |   |-- global_task_models/
> |   |   |-- list_task_models/
> |   |   |-- scenari/
> |   |   |-- auth.conf
> |   |   |-- ca-bundle.crt
> |   |   |-- charset.conf
> |   |   |-- crawlers_detection.conf
> |   |   |-- create_list.conf
> |   |   |-- edit_list.conf
> |   |   |-- ldap_alias_entry.tt2
> |   |   |-- ldap_alias_manager.conf
> |   |   |-- list_aliases.tt2.in
> |   |   |-- mhonarc-ressources.tt2
> |   |   |-- nrcpt_by_domain.conf
> |   |   `-- topics.conf
> |   |-- *lib/*: all modules (.pm files) in the Sympa namespace
> |   |   |-- Makefile.am
> |   |   `-- Sympa
> |   |-- *sbin/*: all excutable files needed for daily Sympa functionning 
> (daemons and queue programs) maybe sympa_wizard.pl.in should move, the, 
> as it is used at install time only?
> |   |   |-- alias_manager.pl.in
> |   |   |-- aliaswrapper.c
> |   |   |-- archived.pl.in
> |   |   |-- bounced.pl.in
> |   |   |-- bouncequeue.c
> |   |   |-- bulk.pl.in
> |   |   |-- familyqueue.c
> |   |   |-- queue.c
> |   |   |-- sympa.pl.in
> |   |   |-- sympa_wizard.pl.in
> |   |   |-- task_manager.pl.in
> |   |   `-- virtualwrapper.c
> |   |-- *soap/*: the SOAP server and associated files
> |   |   |-- Makefile.am
> |   |   |-- sampleClient.php
> |   |   |-- sympa.wsdl
> |   |   |-- sympa_soap_client.pl.in
> |   |   |-- sympa_soap_server-wrapper.fcgi.c
> |   |   `-- sympa_soap_server.fcgi.in
> |   `-- *www**/*: static files needed for the web
> |       |-- Makefile.am
> |       |-- icons
> |       `-- js
> 
> I comment below your own proposal and compare it with the organization 
> Guillaume proposes:
> *Warning: **I dont take into account the localization of the src 
> directory in the Sympa namespace that exists in the sympa-cleanup 
> branch, but not in the current trunk.*
> > -------------------------------- ------------------------
> > Current layout                    Proposed
> > --------------------------------  ------------------------
> > doc/                              doc/
> Identical
> > ext/                              ext/ --- Be separate modules in the
> > future.
> Identical
> > mail_tt2/                         default/mail_tt2/
> Interesting. You propose to have a "default/" directory containing all 
> that will be later located in the "default" directory. I like it because 
> it will probably simplify understanding for people willing to modify 
> this content: They will probably start from what is installed, so if the 
> organization is the same as in the install, it is probably simpler to 
> locate which file to modify in the sources.
> However, in the context of sources, the term "default" is a bit 
> confusing. Id' rather call it "default_src". We keep the "default" 
> naming pattern and add "src" to precise it is the sources for the future 
> "default" directory.
> 
> In addition, I like the dichotomy Guillaume added. We should therefore 
> have the following tree:
> 
> default_src
> |--templates
> |   |-- mail
> |   |-- web
> |-- create_list_templates/
> |-- families/
> |-- global_task_models/
> |-- list_task_models/
> |-- scenari/
> |-- auth.conf
> |-- ca-bundle.crt
> |-- charset.conf
> |-- crawlers_detection.conf
> |-- create_list.conf
> |-- edit_list.conf
> |-- ldap_alias_entry.tt2
> |-- ldap_alias_manager.conf
> |-- list_aliases.tt2.in
> |-- mhonarc-ressources.tt2
> |-- nrcpt_by_domain.conf
> `-- topics.conf

I don't have objection to /mail/ and /web/ subdirectories.

However, we have to keep in mind that current mail_tt2/ and web_tt2/
subdirectories shall be migrated into templates/mail/ and
templates/web/ subdirectories, not only under $defaultdir but also
under:
  $confdir, $confdir/VirtualDomain, $confdir/families, $expldir,
  $expldir/VirtualDomain.

Besides, I'll write about default_src/ and src/ at below.


> > po-wwsympa/                       po-wwsympa/
> > po/                               po/
> I'd like to follow Guillaume's suggestion:
> po/
> |--user_interface (new name for what was "po/")
> |--web_help (new name for what was "po-wwsympa/")

If the principle is that "sources should be located at similar places
to installed binaries", such naming does not have inevitability.

For example, source tree of GNU Mailman does:
    messages/LOCALE/LC_MESSAGES/mailman.po
This will be installed as:
    PREFIX/messages/LOCALE/LC_MESSAGES/mailman.mo


> > src/bin/sympa.generic             sympa.generic
> > src/bin/sympa.in                  sympa.in
> 
> > src/bin/            [OTHERS]      bin/  or  script/ [std]
> "script" looks better as the programs what Guillaume put in 
> "src/Sympa/bin" are more scripts to execute once or only once in a while.
> > src/cgi/mime.types                default/mime.types
> This is consistent as mime.types will be in "default" after install.
> > src/cgi/            [OTHERS]      libexec/
> I prefer "cgi". After all, this directory contains only the perl program 
> responsible to run the Sympa web interface.

I misled.  Sympa (and configure script) distinguishes between
$libexecdir and $cgidir.  I agree to placing sources of FCGI scripts
and wrappers in cgi/.

Additionally, I wish to install other wrappers including setUID
executables into single place, $libexecdir (this is not so strong
opinion).


> > src/etc/                          default/
> OK, for the reasons mentionned above;
> > src/lib/                          lib/ [std]
> > src/sbin/                         sbin/  or bin/ [std]
> I disagree here. the reason is that I like the idea of having the whole 
> perl code in a separated "src/Sympa/" directory. All the perl is in the 
> same place. I think that for new developpers, it is easier to find the 
> code (in addition to Guillaume's remarks regarding grep usage, which are 
> more our concerns).
> Consequently, I'd like to keep it this way.

I'll write about it at below.

> > src/soap/sympa.wsdl               default/sympa.wsdl
> This is consistent as sympa.wsdl will be in "default" after install.
> > src/soap/sympa_soap_client.pl.in  bin/  or /srcipt/ [std]
> script/ is pereferable here; As it is only a test script to check that 
> the SOAP server works.
> > src/soap/sampleClient.php         ? --- May be moved to contrib.
> Well, this is kind of an awkward script. Useful, but certainly not 
> essential.
> We could have a general "sample" directory in which to put this script 
> and the other examples such as the family example.
> > src/soap/           [OTHERS]      libexec/
> Interesting proposal (though I don't agree with the name, I prefer 
> "cgi"): After all, the SOAP server is a cgi script. It should therefore 
> go to the same directory as wwsympa.fcgi.
> If we don't want to mix SOAP and web, we can make sub directories : 
> cgi/web and cgi/soap

I agree to cgi/.  I feel subdirectories may not be needed.

> > src/www/                          www/
> Agreed. It is neither code nor future default directory. It should have 
> its own directory.
> > t/                                t/ [std]
> Agreed. You both know that better than me.
> > web_tt2/                          default/web_tt2/
> Wee comment above about mail_tt2

Now, I'll write about src/ directory: What means the "source"?

- If it means "source of executable", lib/, default/ (etc/) and so on
  _should not_ be placed under src/.

- If it means "source of compiled binary", lib/, default/ (etc/) and
  so on _should not_ be placed under src/, while po/ (they are compiled!)
  and so on _should_ be placed under src/.

- If it means "source of file to be installed at last", default_src/,
  www/, ... and so on _should_ be placed under src/.

- etc.

In the "least" case, only a few directories will be placed under src/.

In the "most" case, almost all directories will be placed under src/.

After all, I suppose that src/ directory is not necessary.



> > --------------------------------  ------------------------
> >
> > Note: "[std]" indicates (approximately) standard source layout with
> > ExtUtils::MakeMaker.
> OK. Shall we use it some day, we would take that into account. It is not 
> a pressing matter.
> >
> > Resulting layout will become:
> >
> > --------------------------------
> > bin/  or  script/ [std]
> > doc/
> > ext/
> > default/
> > lib/ [std]
> > libexec/
> > po-wwsympa/
> > po/
> > sbin/  or  bin/ [std]
> > t/ [std]
> > www/
> > --------------------------------
> Here is my advice:
> 
>  1. keep a "src/Sympa" directory to store our perl code
>  2. keep your proposals of renaming / reorganization that look good indeed.
> 
> We would therefore have the following mix of your proposal and what 
> guillaume had done in the sympa-cleanup branch:
> (in green, directories that were moved / renamed, in red, files that 
> were moved, according to Soji's proposals, Guillaume answers and my own 
> advice)
> (please note the new "samples" directory, in blue)
> 
> |-- several files to be read: AUTHORS, COPYING, ChangeLog, INSTALL, 
> NEWS, README, README.charset
> |-- some scripts: important_changes.pl, sympa.spec.in, tap-driver.sh
> |-- *doc/*: doc
> |-- *ext/*: external modules
> |-- *t**/*: tests
> |-- *po/*: localization
> |-- *user_interface**/*: uesr interface localization
>      `-- *web_help**/*: web help localization
> |-- *default_src/*: configuration and templates defaults
> |   |-- create_list_templates/
> |   |-- families/
> |   |-- global_task_models/
> |   |-- list_task_models/
> |   |-- scenari/
> |   |-- *templates/*: mail templates
> |   | |-- *mail/*: mail templates
> |   |   `-- *web/*: web templates
> |   |-- auth.conf
> |   |-- ca-bundle.crt
> |   |-- charset.conf
> |   |-- crawlers_detection.conf
> |   |-- create_list.conf
> |   |-- edit_list.conf
> |   |-- ldap_alias_entry.tt2
> |   |-- ldap_alias_manager.conf
> |   |-- list_aliases.tt2.in
> |   |-- mime.types
> |   |-- mhonarc-ressources.tt2
> |   |-- nrcpt_by_domain.conf
> |   |-- sympa.wsdl
> |   `-- topics.conf
> |-- *www**/*: static files needed for the web
> |       |-- Makefile.am
> |       |-- icons
> |       `-- js
> |-- *samples**/*: static files needed for the web
> |   |-- *families*/
> |   `-- sampleClient.php
> |-- *src**/*: source code
> |   |-- *script**/*: ponctual usage scripts
> |   |   |-- arc2webarc.pl.in
> |   |   |-- arcrepair.pl
> |   |   |-- create_db.Oracle
> |   |   |-- create_db.Pg
> |   |   |-- create_db.SQLite
> |   |   |-- create_db.Sybase
> |   |   |-- create_db.mysql
> |   |   |-- crypt_passwd.pl.in
> |   |   |-- find_missing_messages.pl
> |   |   |-- init_comment.pl.in
> |   |   |-- ldap_alias_manager.pl.in
> |   |   |-- logrotate
> |   |   |-- mod2html.pl.in
> |   |   |-- mysql_alias_manager.pl.in
> |   |   |-- p12topem.pl.in
> |   |   |-- sympa.generic
> |   |   |-- sympa.in
> |   |   |-- sympa_soap_client.pl.in
> |   |   |-- testldap.pl.in
> |   |   |-- testlogs.pl.in
> |   |   `-- tpl2tt2.pl.in
> |   |-- *cgi/*: daemons that run under a web server
> | |   |-- *web**/*: web daemon and associated files
> | |   |   |-- INSTALL
> | |   |   |-- Makefile.am
> | |   |   |-- README
> | |   |   |-- wwsympa-wrapper.fcgi.c
> | |   |   `-- wwsympa.fcgi.in
> | |   |-- *soap/*: the SOAP server and associated files
> | |   |   |-- Makefile.am
> | |   |   |-- sympa_soap_server-wrapper.fcgi.c
> | |   |   `-- sympa_soap_server.fcgi.in
> |   |-- *lib/*: all modules (.pm files) in the Sympa namespace
> |   |   |-- Makefile.am
> |   |   `-- Sympa
> |   |-- *sbin/*: all excutable files needed for daily Sympa functionning 
> (daemons and queue programs) maybe sympa_wizard.pl.in should move, the, 
> as it is used at install time only?
> |   |   |-- alias_manager.pl.in
> |   |   |-- aliaswrapper.c
> |   |   |-- archived.pl.in
> |   |   |-- bounced.pl.in
> |   |   |-- bouncequeue.c
> |   |   |-- bulk.pl.in
> |   |   |-- familyqueue.c
> |   |   |-- queue.c
> |   |   |-- sympa.pl.in
> |   |   |-- sympa_wizard.pl.in
> |   |   |-- task_manager.pl.in
> |   |   `-- virtualwrapper.c
> 
> >> What did you think about what Guillaume had done in the sympa-cleanup
> >> branch? It looked good: every perl code in the src/Sympa directory, then
> >> more or less the same organization as the one you proposed?
> > My proposal covers where he had not changed.
> Well, my proposal tries to cover both you proposals and my own advice. 
> It looks like a reasonable proposal to me.
> Do you agree?
> 
> Best regards,
> 
> David

Best regards,


--- Soji


-- 
株式会社 コンバージョン  セキュリティ&OSSソリューション部   池田荘児
〒231-0004　神奈川県横浜市中区元浜町3-21-2 ヘリオス関内ビル7F
e-mail address@concealed  TEL 045-640-3550
http://www.conversion.co.jp/