Developers of Sympa

[Marc Chantreux <address@concealed>]

On Fri, Mar 21, 2014 at 10:25:10AM +0100, David Verdin wrote:
> >>I must agree with Soji.
> >i wasn't disagreed.
> Sorry, I misunderstood. I read too quickly sometimes. ;-) 

don't: we all know how hard is it to share ideas and feelings over
emails. also, you know i'm not strongly opiniated when i contribute to
projects so i just share ideas without any other expectation than
sharing.

> >>param1
> >>     subparam1
> >>         sub-subparam1 value
> >param1/subparam1/sub-subparam:1 value
> >param1/subparam1/sub-subparam:2 value
> >
> >is a valid solution. simple to edit and parse even un shell …
> Yes, I thought about it, but as you need to repeat param names, it
> is more error-prone and, if you mistype a parameter, you must change
> its name as many times as you mistyped it. 

yes but from my experience, those kind of format are not there to be
editable directly. and i have to admit i don't really thought about it
because for me, YAML is the perfect default format (and i think we're
all agreed on this point)

note that i have lied a bit about shell admins because having the list
of editors emails would just be:

perl -0 -MYAML -lnE '
    say $$_{email} for @{YAML::Load($_)->{editors}}
' config 

but you know … habits … lot of us are unconfortable kissing awk goodbye :)

> - Sympa is not likely to have to transfer its config to another web
> app. Only data (groups, subscription and such). The same is valid
> for incoming data: other apps won't transfer config elements, only
> data. So the web app aspect in not discriminant for the
> configuration formalism choice.  

we *need* this feature at Strasbourg (because we have a unique bot
shared with many organizational units and what we want to give to them
is SAAS: they will use sympa features and infrastructures from their
intranet pages). So if there is no hook in sympa, we'll have to
patch it roughly which would be a pain to maintain.

> - Sympa is not only a web app. It is, at its core, a message
> broadcasting system which for now mainly use mail.  

sure. and i really would like the web interface to be completly
optionnal during the installation process.

> s-expressions have the very good point to be extremely flexible and
> likely to express anything. So why not?

because we already agree with YAML ;) 

> >And all of those have pro and cons (even xml which can be validated with
> >a schema).
> At the very important difference that XML is not humanly manipulable
> easily.

well. ask a java/eclipse fanatic and you'll have a difference answer
(and it hurts me to admit i saw cool stuff on it like onfly validation
with decent error messages)

> There is no good free XML editor (one that can do
> autocompletion while typing, based on the schema, for example). 

good xml editor don't show you tags: they rely on schema to make you
edit a graphical representation of the tree. you don't even have to know
what xml is and that's the good part when you think about the audience.

but yes: it's not our cup of tea. What i was trying to say is: the more
sympa is flexible, the more it will please the university world and its
large spectrum of (mainly twisted?) minds.

giving the hability to overload the $list->get_config will not be
difficult and the benefit could be huge. so it seems to me that it is a
quick-win we can't afford to miss.

> is very painful for human edition. I see it on a daily basis with
> Shibboleth.

the problem of Shibboleth is the targeted audience:

* administrators are pleased with simple, sed compliants formats. they
  try to edit xml with vim without even an xml autoclose or something.
  so yes: it's error prone and painfull that way.
* developpers are pleased with overdesigned montains of java/xml stuff
  (see ESUP and CAS), they use CPU burners called IDE with pleasant
  integrated editors: they really don't see the problem.

they don't share their visions, their tools and both camps are convinced
that the other are morrons.

But all of them will be aware that the underlaying structure is finally
the same in memory and everything written in yaml can be written in XML.
What we just need is telling people:

* the final datastructure in memory must obey the following rules:
  * blah 
  * blah
  * blah 
* the prefered representation of the in memory format is YAML (as it is
  the closer and easier way to write it without using Perl). (About
  using Perl: that's what RT does and it's very nice if you trust your
  admin :)).

represented as YAML structure. What the shibboleth project needs is
just a bridge between 2 data structures. and that's what sympa also need.

current situation:

   +------+ parser  +------+ 
   |      | ------->|      | 
   | old  |         | sympa|
   |format| writer  |      | 
   |      | <------ |      | 
   +------+         +------+ 

what i'm fearing we are ending up:

   +------+ parser  +------+ 
   |      | ------->|      | 
   | YAML |         | sympa|
   |      | writer  |      | 
   |      | <------ |      | 
   +------+         +------+ 

what i think is the better way.

   +------+ parser  +------+ parser   +------+ 
   |      | ------->|      | <------  |      | 
   | YAML |         | in   |          | XML  | 
   |      | writer  |memory| writer   |      | 
   |      | <------ |      | ------>  |      | 
   +------+         +------+          +------+         
                      |  |
                    +------+ 
                    |sympa |
                    +------+ 

but from the sympa code side, it's basically the same (and JSON and YAML
would be proably nothing but YAML::LoadFile).

> >so the community would be free ot discuss/implement new things.
> the idea to let administrators have choices. But configuration
> formalism is not the space where I want to let them choices. 

not configuration formalism: configuration providing :)  

> to give them expressivity and freedom in the core Sympa
> functionnalities: List configuration, communication with other
> applications, message broadcasting, list creation industrialisation,
> functionnality cusomization, etc. 

i'm a bit confused because i think we share the same idea so i'm afraid
not explaining myself clearly enough.

> For this, we need a common language: If a user experienced in, say
> XML, wants to share a receipe or help another debug, it will do it
> by providineg XML examples. If other users are used to JSON
> configuration, that forces them to understand another formalism
> before trying to understand how the receipe works. 

YAML should be the choosen convention, not a technical restriction.

* want to share ideas ? please dump your config file in yaml before
  posting it in a mailing list 
* you want to know if your webservice provides a valid configuration? 
  use the yaml writer to test it without havning a full sympa
  installation. 
* you're a bit freak? use Sereal to share your configurations :)  

all of those tools have only one thing in common: they all provide a
datastructure in memory that can be used by sympa. 

   +------+ parser  +------+ parser   +------+ 
   |      | ------->|      | <------  |      | 
   | YAML |         | in   |          | XML  | 
   |      | writer  |memory| writer   |      | 
   |      | <------ |      | ------>  |      | 
   +------+         +------+          +------+         
 
If you think about it: this kind of design made the modern web
developpement what it is in the dynamic languages (plack, rack, wsgi,
…): what we want is just an array of arrays which are representations of
requests and structures.

it's simple, easy to understand, degug, extend. You don't even neeed a
sympa instance to develop and test a new connector.

> Remember that most listmasters have not Sympa as their mùain
> occupation. This is rather a side task that, most of the time, they
> have very few time to invest in. Giving them several config
> formalism will only add to their confusion.  

so they will not even know that there is another way than yaml … until
their manager will comes back from a new IT managers congres where the
buzz word would be this new EDI system enable manage mailing lists to
satisfy some workflow scenari. and we would be ready for that too.

> open until we start working on 7.1. Except for XML. We will switch
> to XML over my dead body. ;) 

i used to think this way i changed: what i want is people to use sympa.
how to do that? telling us they can use it *their* way (as long as it
doesn't cost so much to us). if your way is talking to sympa with a huge
java servlets generating tons of xml: go ahead! if it doesn't work, we
can always fall back our way which is simple yaml editing :) 

regards


-- 
Marc Chantreux
Université de Strasbourg, Direction Informatique
14 Rue René Descartes,
67084  STRASBOURG CEDEX
☎: 03.68.85.57.40
http://unistra.fr
"Don't believe everything you read on the Internet"
    -- Abraham Lincoln