The mailing list for listmasters using Sympa

[Steve Shipway <address@concealed>]

Using the -02 version, the $WINDOW seting is in the rate_limit.pm file (I 
think -02 defaults to per day anyway).

The CustomCondition line to limit to 60 per day for the list, and 6 per day 
per subscriber, would be as below Note you always use [listname] regardless 
of what the listname is, as this expands to the name!

  CustomCondition::rate_limit([listname],"",60,[header->Message-ID]) 
smtp,dkim,smime,md5 -> reject(tt2=rate_limit)
  CustomCondition::rate_limit([listname],[sender], 6,[header->Message-ID]) 
smtp,dkim,smime,md5 -> reject(tt2=rate_limit)

Stick these into the send scenario for the list you're testing on and all 
should work.

I don't know if it would work on 6.1.7, since it depends on the 
CustomCondition feature being available, and I don't know if that was present 
in 6.1.7

Steve

Steve Shipway
University of Auckland ITS
UNIX Systems Design Lead
address@concealed
Ph: +64 9 373 7599 ext 86487


________________________________________
From: e.c. [address@concealed]
Sent: Sunday, 29 June 2014 1:33 a.m.
To: Steve Shipway
Cc: David Verdin; address@concealed
Subject: Re: [sympa-users] Strict, per list, send limit...

This looks like exactly what I want, or at least one of the things I
want if the effects of these changes can be limited to one list. Is
that true? Even if not, I would like to test this on 6.1.17. If I
install sympa on a recent debian box can
I test this by myself (using a few different email addresses)? For a
limit of 6 per day per member and 60 per day per list is this what I
want (if listname is grex):

  CustomCondition::rate_limit([grex],[""],60,[header->Message-ID])
smtp,dkim,smime,md5 -> reject(tt2=rate_limit)

and

  CustomCondition::rate_limit([grex],[sender], 6,[header->Message-ID])
smtp,dkim,smime,md5 -> reject(tt2=rate_limit)

or do I lose the brackets?

Also, where do I set $WINDOW to 86400?

Thanks,

Ed



On Thu, Jun 26, 2014 at 6:39 PM, Steve Shipway <address@concealed> 
wrote:
> Here is my solution to provide message rate limiting on a per-list or
> per-subscriber-per-list basis.
>
>
>
> It works by adding a custom condition for send scenari that allows you to
> reject a message if a given subscriber has sent more than a given number of
> messages that hour.  Thanks to the suggestion from Giorgio, you can work
> around the issue of caching, though it may be more memory-efficient to
> simply disable the custom check caching by changing the verify_custom
> subroutine in the  lib/Scenario.pm file like this:
>
>
>
> ---line 1273, Scenario.pm----
>
>         return undef unless defined $res;
>
>
>
> # Comment out these three lines
>
> #        $persistent_cache{'named_filter'}{$condition}{$filter}{'value'} =
> ($res == 1 ? 1 : 0);
>
> #        $persistent_cache{'named_filter'}{$condition}{$filter}{'update'} =
> time;
>
> #        return
> $persistent_cache{'named_filter'}{$condition}{$filter}{'value'};
>
> # Add this one
>
>         return $res;
>
> }
>
>
>
> The custom condition creates a new table rate_limit in the Sympa database
> which is used to hold subscriber/list posting counts.  This should not need
> to be pruned except with public lists which could attract spam.
>
>
>
> You use a custom scenario line similar to this to make it work:
>
>
>
>   CustomCondition::rate_limit([listname],[sender],10,[header->Message-ID])
> smtp,dkim,smime,md5 -> reject(tt2=rate_limit)
>
>
>
> The first two parameters are list name and sender; they are used to form the
> key for counting.  To make a per-list count, use “” instead of [sender].  If
> you have multiple robots then replace [listname] with [listbame]@[robot].
> The third parameter is the maximum messages per clock hour (reset count at
> top of hour).  The fourth parameter is Giorgio’s trick to effectively
> disable the Custom Condition caching for this function; you do not need this
> if you modified Scenario.pm as suggested above.
>
>
>
> It will log information about the calling parameters, the count value found
> in the database, and the updated value at debug2 level in the logs.
>
>
>
> HOW TO INSTALL
>
> 1.       Copy rate_limit.pm into the $SYMPAETC/custom_conditions directory
> (create directory if you have to).  This makes the new function available to
> scenari.
>
> 2.       Copy rate_limit.tt2 into the $SYMPAETC/mail_tt2 directory (create
> directory if you have to).  This defines a custom notification message for
> messages rejected due to rate limiting.  (If this mod is merged into the
> main branch, I would expect this to be merged into authorization_reject.tt2)
>
> 3.       Create a Custom Scenario, such as $SYMPAETC/scenari/send.ratelimit
> which has the custom condition.  An example is provided.  Note how we try
> and exclude all non-posters before calling the Custom Condition in order to
> minimise table size and to prevent per-list counts from being incorrect.
>
> 4.       Configure your list to use this custom scenario, and all should be
> fine.
>
>
>
> Caveats:
>
> 1.       Only tested with Sympa 6.1.19
>
> 2.       Only tested with MySQL, likely doesn’t work with other databases
>
> 3.       The rate_limit table may grow large if scenario used on lists
> permitting public posting due to spam attacks.
>
> 4.       The condition limits messages per hour, resetting the count at top
> of hour (IE, at 12:00, 1:00, etc).  If you want per-day counts, modify the
> $WINDOW setting in the plugin to be 86400.  Note that if you make this
> change to daily, it will reset the count at midnight UCT regardless of your
> local timezone.
>
> 5.       Do not call the test twice with the same parameters in the same
> scenario.  Each time it is called it increments the count.  You can,
> however, call it once with listname/sender to enforce a per-sender limit and
> once with listname only to enforce a per-list limit.
>
>
>
> ***Please provide feedback.***
>
>
>
> If the Sympa team are interested in providing this functionality within
> Sympa, I’d probably expect it to be done either as a new internal function
> available to send scenari, or else as an option within list configuration
> that is tested before the send scenario (around the same time as the message
> size tests are run).  As it also requires a new database table to be added
> it might be a bit more complex to merge in smoothly.  Fortunately, Sympa has
> excellent hooks and API so it was not too hard to write a plugin.
>
>
>
> Steve
>
>
>
> Steve Shipway
>
> address@concealed
>
> University of Auckland, New Zealand