The mailing list for listmasters using Sympa

[Michael Heller <address@concealed>]

A post-cutover thank-you to Mark V. and the Sympa community!   We're (mostly) up and now just working through things that are different, changed.  

~Michael

On Wed, Mar 1, 2023 at 6:10 PM Mark Valiukas <SNIPPED> wrote:

Michael Heller asked:

> The (current) data in the new Sympa db will be out-of-sync with the production on
> go-live date so we need the latest data from the hosts to be retired. 

That's not all. Depending on how long it is since you brought the data across and active your lists are - and I'm guessing they're likely to be pretty active if you've had to scale out to separate app and db hosts - you're likely to need to bring across:

- Sympa_Aliases - in case lists have been added or retired
- spool directory, to cover anything awaiting moderation
- Archives (or at least new parts - to cover new messages and even entirely new lists)
- list_data directory (to cover new or retired lists, changes to list configuration, new custom scenari or templates etc)
- and, of course, the database.

Apologies if I'm stating the obvious and/or you've already thought of those. Or if I've missed anything else important.

> Is it possible to drop the Sympa tables from the Sympa database on the new db server and import once more?

Yes

> Or will the database structure be incorrect?

The database structure will be correct for 6.1.20, but you're upgrading to 6.2.70

>  That is, can we import the data into an existing Sympa db, or do we need to drop the Sympa db,
>(re)create, import data from soon-to-be-retired Sympa, and run the Sympa upgrade once more?

Cleanest is likely to be drop and import, followed by running sympa upgrade - which will change the database structure as part of the upgrade process, but for it to get that right you'll need to edit data_structure.version on the new host. That will currently contain 6.2.70, you'll need to edit that to 6.1.20 so the upgrade script will know what it's starting from.

That said, according to our notes and runsheet from last year's upgrade from 6.2.18 to 6.2.68, we just imported the dump taken from the old box to the new box, changed data_structure.version on the new box to reflect the db version we'd just applied, and ran sympa upgrade. That also seemed to work on a bunch of synchs over several months during testing as we verified and refined our go-live process.

> Thanks in advance for any suggestions and cautionary tales!

Oooh ... got one for you, maybe. Do your user workflows rely on one-time tickets in moderation messages? Some of ours do, but somewhere between 6.2.18 and 6.2.64 those were deprecated due to privacy/security concerns. I haven't looked at the 6.2.70 code yet, but I have some changes working on 6.2.68 to /home/sympa/etc/[robot_conf_dir]/mail_tt2/moderate.tt2, /home/sympa/bin/Sympa/List.pm, and /home/sympa/bin/Spindle/ToModeration.pm to turn that back on. They put a few things back in that were removed to drop the moderation ticket functionality. Tickets are still there for password changes and account provisioning if I remember correctly, so it wasn't too hard, but there's a risk a future version might change how those are done and remove functionality the moderation ticket code relies on.

Oh, and a post-import rebuild of the archives probably wouldn't go astray either. We're using summary mode for some lists with a nice custom multipart HTML/plain message template, I think we had some difficulties with existing links in old summaries to items in the archives until we did "Listmaster Admin -> Archives -> Rebuild all" on each robot we're running.

Kind regards,

Mark.