The mailing list for listmasters using Sympa

[Mark Valiukas <address@concealed>]

Hi all,

I've cleaned up and expanded on the code I posted a
very rough early version of, and I'm just about 
ready to go into production with it

This implements a new delivery type - html_summary -
which is designed to be pretty, have clickable links
and generally be less intimidating to people who are
unfamiliar with mailing list services. We'll be using
it in place of a custom summary.tpl for our University
General Announcements mailing list.

The attached diffs for List.pm and wwsympa.lib are against
the versions included with 3.4.4.3 - sorry, haven't looked
seriously at it in relation to 4.x yet, that's a project
for this week.

A MIME Multipart/Alternative message is created, based on
html_summary.tpl and html_summary.plain.tpl for the html
and plain parts respectively. The included versions are very
similar to what we're planning on going into production with.

There's a little problem I'm still having with regard to the
mailto: links - depending on how many % signs are in the
message, their positioning, and what follows them, the Subject:
part of the mailto: link can get mangled. I know this is
something to do with how the parser interprets what's fed to it,
but I haven't been able to pin down exactly how to escape the
subject line to make it display as sent when someone sends a
message with a subject like "99% of people never send %26 in an
email subject line"

If anyone has any ideas on how anything could be done
better, I'd be interested to hear them.

Regards,

--
Mark Valiukas
304c304
< 	    'available_user_options' => {'format' => {'reception' => {'format' => ['mail','notice','digest','summary','nomail','txt','html','urlize','not_me'],
---
> 	    'available_user_options' => {'format' => {'reception' => {'format' => ['mail','notice','digest','summary','nomail','txt','html','urlize','not_me','html_summary'],
307c307
< 								      'default' => 'mail,notice,digest,summary,nomail,txt,html,urlize,not_me',
---
> 								      'default' => 'mail,notice,digest,summary,html_summary,nomail,txt,html,urlize,not_me',
376c376
< 	    'default_user_options' => {'format' => {'reception' => {'format' => ['digest','mail','nomail','summary','notice','txt','html','urlize','not_me'],
---
> 	    'default_user_options' => {'format' => {'reception' => {'format' => ['digest','mail','nomail','summary','notice','txt','html','urlize','not_me','html_summary'],
2038c2038
< 	if ($user->{'reception'} =~ /^digest|summary|nomail$/i) {
---
> 	if ($user->{'reception'} =~ /^digest|summary|html_summary|nomail$/i) {
2304c2304
< ## Send a digest message to the subscribers with reception digest or summary
---
> ## Send a digest message to the subscribers with reception digest or summary or html_summary
2326a2327,2328
>     my @tabrcpthtmlsummary;
>     
2346c2348,2355
<     return if (($#tabrcptsummary == -1) and ($#tabrcpt == -1));
---
> 
>     ## Create the list of subscribers in html_summary mode
>     for (my $user = $self->get_first_user(); $user; $user = $self->get_next_user()) {
>         push @tabrcpthtmlsummary, $user->{'email'}
>              if $user->{'reception'} eq "html_summary";
>     }
> 
>     return if (($#tabrcpthtmlsummary == -1) and ($#tabrcptsummary == -1) and ($#tabrcpt == -1));
2406a2416,2451
> 	## need html-friendly and mailto-friendly subject lines in param hash
> 	## must escape ", <, and >
> 	## must replace & and ? in mailto as both are delimiters
> 	my $subject_temp = $subject;
> 	my $subject_mailto_temp = $subject;
> 	##
> 	#$subject_temp =~ s/&/%26/gs;   # do first, so we don't kill "&lt" etc...
> 	$subject_temp =~ s/&/&amp;/gs;   # do first, so we don't kill "&lt" etc...
> 	$subject_temp =~ s/#/&#35;/gs;
> 	$subject_temp =~ s/%/&#37;/gs;
> 	$subject_temp =~ s/\</&#60;/gs;
> 	$subject_temp =~ s/\>/&#62;/gs;
> 	$subject_temp =~ s/"/&#34;/gs;
> 	$subject_temp =~ s/!/&#33;/gs;
> 	$subject_temp =~ s/@/&#64;/gs;
>         ##
>         $subject_mailto_temp =~ s/&/\[and]/gs; # do first, so doesn't kill "&lt" etc...
>         $subject_mailto_temp =~ s/#/&#35;/gs; # do first, so we don't kill "&lt" etc...
>         $subject_mailto_temp =~ s/\</&#60;/gs;
>         $subject_mailto_temp =~ s/\>/&#62;/gs;
>         $subject_mailto_temp =~ s/"/&#34;/gs;
>         $subject_mailto_temp =~ s/%/%25/gs;
>         #$subject_mailto_temp =~ s/%/%25;/gs;
>         #$subject_mailto_temp =~ s/\?/\[question\]/gs; # no question marks either.
>         ##
>         $msg->{'htmlised_subject'} = $subject_temp;
>         $msg->{'mailto_subject'} = $subject_mailto_temp;
> 
>         ## Some mail clients might baulk at 'mailto:"Joe Blow" <address@concealed>?subject:Foo"
> 	## so extract an email address only to use as bare_from in param hash
> 	 my $sender_line = $msg->{'from'};
> 	my @sender_hdr = Mail::Address->parse($sender_line);
> 	my $extracted_email = @sender_hdr[0];
> 	$msg->{'bare_from'} = $extracted_email->address;
> 	
> 	
2472a2518,2651
>     
>     
>     ## send html summary
>     ## Note: multi-language template support may make it necessary to 
>     ## move all of this into send_file and do this the same way as "send summary", 
>     ## or make this a lot more complicated. This version will only work for
>     ## one language. - mjv 
>     ##
>     
>     if (@tabrcpthtmlsummary) {
>         $param->{'subject'} = sprintf Msg(8, 31, 'Summary of list %s'), $self->{'name'};
> 	
> 	## not supported in this version - no code in send_msg to do this.
> 	## see above - put in as placeholder/reminder...
> 	#$self->send_file('html_summary', \@tabrcpthtmlsummary, $robot, $param);
>         
> 	## from here on is the present means of generating and sending
> 	## html multi-part summaries.
> 	my $html_msg = MIME::Entity->build (
> 	    To         => $param->{'to'},
> 	    From       => $param->{'from'},
> 	    'Reply-to' => $param->{'reply'},
> 	    Type       => 'multipart/alternative',
> 	    Subject    => MIME::Words::encode_mimewords(sprintf "HTML Summary of list %s",$listname)
> 	    #Subject    => MIME::Words::encode_mimewords(sprintf "%s",$param->{subject})
> 	);                                                                               
> 	
>         my $charset = sprintf 'us-ascii';
> 
>         local $/ = undef;
> 
>         my $related = MIME::Entity->build (
> 	    Type    => "multipart/related"
> 	);
> 	  
>   
> 	my $html_template;
> 	my $plain_template;
> 	
> 	##Do the plain text templates exist anywhere? Borrowed from send_file
> 	##If there's no suitable plaintext template, defaults to 
> 	##summary.tpl using the same kind of precedence rules
> 	foreach my $f ("$self->{'dir'}/html_summary.plain.tpl",
> 	               "$Conf{'etc'}/$robot/templates/html_summary.plain.tpl",
> 	               "$Conf{'etc'}/templates/html_summary.plain.tpl",
> 	               "$self->{'dir'}/summary.tpl",
> 	               "$Conf{'etc'}/$robot/templates/summary.tpl",
> 	               "$Conf{'etc'}/templates/summary.tpl") {
> 	    if (-r $f) {
> 	        $plain_template = $f;
> 		last;
>             }
> 	}
> 																														  
> 	unless ($plain_template) {
> 	        do_log('err',"Unable to find a suitable template for plain text part of a HTML message anywhere");
> 	}
> 		    
> 	
>         ##Do the html templates exist anywhere? Borrowed from send_file
> 	##If there's no suitable plaintext template, defaults to
> 	##summary.tpl using same kind of precedence rules
> 	foreach my $f1 ("$self->{'dir'}/html_summary.tpl",
>                         "$Conf{'etc'}/$robot/templates/html_summary.tpl",
>                         "$Conf{'etc'}/templates/html_summary.tpl",
>                         "$self->{'dir'}/summary.tpl",
>                         "$Conf{'etc'}/$robot/templates/summary.tpl",
>                         "$Conf{'etc'}/templates/summary.tpl") {
>             if (-r $f1) {
> 	        $html_template = $f1;
> 	        last;
>             }
> 	}
> 	
> 	unless ($html_template) {
> 	    do_log('err',"Unable to find a suitable template for html part of a HTML message anywhere");
>         }                                                                                                                                                                                                                                                                                                                                                                                     
> 	my $name = $self->{'name'};
> 
>         foreach my $p ('email','host','sympa','request','listmaster','wwsympa_url','title') {
> 	    $param->{'conf'}{$p} = &Conf::get_robot_conf($robot, $p);
> 	}
> 
> 	$param->{'list'}{'lang'} = $self->{'admin'}{'lang'};
> 	$param->{'list'}{'name'} = $name;
> 	$param->{'robot_domain'} = $robot;
> 	$param->{'list'}{'host'} = $self->{'admin'}{'host'};
> 	$param->{'list'}{'subject'} = $self->{'admin'}{'subject'};
> 	$param->{'list'}{'owner'} = $self->{'admin'}{'owner'};
> 	$param->{'list'}{'dir'} = $self->{'dir'};
> 
> 
>         my @text_data;
>         my $pass_text;
> 	$pass_text = \@text_data;
> 	
> 	&parser::parse_tpl($param,$plain_template,$pass_text);
> 			    
>         my $text_content = \@text_data;
> 			
>         my $text_content = MIME::Entity->build (
> 	    Type        => "text/plain; charset=$charset",
> 	    Description => 'TEXT summary',
> 	    Data        => $text_content
> 	);
> 	
> 
> 	my @html_data;
> 	my $pass_html;
> 	$pass_html = \@html_data;
> 	 
> 	&parser::parse_tpl($param,$html_template,$pass_html);
> 				    
>         my $html_content = \@html_data;
> 
> 	my $html_content = MIME::Entity->build (
> 	    Type        =>"text/html; charset=$charset",
> 	    Description => 'HTML Summary',
> 	    Data        => $html_content
>         );
> 														 
> 
>         $html_msg->add_part($related);
>         $html_msg->add_part($text_content);
> 	$html_msg->add_part($html_content);
> 		
>         my $html_new_msg = $self->add_parts($html_msg);
>         if (defined $html_new_msg) {
> 	     $html_msg = $html_new_msg;
> 	}
> 	
> 	&smtp::mailto($html_msg, $param->{'return_path'}, 'none', '_ALTERED_', @tabrcpthtmlsummary);
>     }
>     
5018c5197
< 	$user{'reception'} = $1 if (/^\s*reception\s+(digest|nomail|summary|notice|txt|html|urlize|not_me)\s*$/o);
---
> 	$user{'reception'} = $1 if (/^\s*reception\s+(digest|nomail|summary|html_summary|notice|txt|html|urlize|not_me)\s*$/o);
5090c5269
< 	$user{'reception'} = $1 if ($line =~ /^\s*reception\s+(digest|nomail|summary|notice|txt|html|urlize|not_me)\s*$/o);
---
> 	$user{'reception'} = $1 if ($line =~ /^\s*reception\s+(digest|nomail|summary|html_summary|notice|txt|html|urlize|not_me)\s*$/o);

40c40,42
< 		   'not_me' => 'not_me');
---
> 		   'not_me' => 'not_me',
> 		   'html_summary' => 'html_summary')
> 		   ;

Title: [list->subject]

Here is a message summary for the [list->subject] mailing list. Please click on the relevant links to read the messages. If you are not connecting from within the VU network, please log in first.

[list->subject]

[IF conf->wwsympa_url]

	Log in to the groups server
	View archives for this group
	Change your subscription options
	Information regarding this list

[ENDIF]

[FOREACH m IN msg]



[IF conf->wwsympa_url]

[m->htmlised_subject]
  sent by [m->from]
[ENDIF]


[END]

For assistance please contact the ITS Helpdesk, x2777

Plain-text version:
You are seeing this because your email program is either
incapable of displaying HTML messages, or is set to only
display the plain-text part of the message.

Here is a message summary for the

    [list->subject]

mailing list. Please click on the relevant links to
read the messages. To access some features, you may
need to log in first.

Messages received today:

[FOREACH m IN msg]

    [m->subject]     [m->from]
[IF conf->wwsympa_url]
    [conf->wwsympa_url]/arcsearch_id/[list->name]/[m->month]/[m->message_id]
[ENDIF]

[END]
        

--------------------------------------------------------
You are receiving this because you are subscribed to the

    [list->subject]

mailing list.

For information regarding this list, please visit:
    [conf->wwsympa_url]/info/[list->name]

To modify your subscription options for this list, please visit:
    [conf->wwsympa_url]/suboptions/[list->name]

You may need to log in to access the messages or change your
subscription settings. To do so, please visit:
    [conf->wwsympa_url]/nomenu/loginrequest/home