The mailing list for listmasters using Sympa

["K. Clair" <address@concealed>]

Hi List,

It turns out that sympa was trying to send >12GB of logs about its 
previous crashed process!

-Kristina

David Verdin wrote:
> Hi Kristina
>
> Le 07/07/2010 15:49, K. Clair a écrit :
> > Hi All,
> >
> > Our list server, running sympa 6.0.1, lost power yesterday and had to
> > be fscked when it came back online. Since then, when we start sympa,
> > the command sympa.pl process grows to using 12GB of memory and then dies.
> Is this growth quick or slow?
> What do the Sympa log say? Are there any frequently executed operations? 
> Too frequently for what you think would be right?
>
> Maybe there are some kind of weird state of you database directory that 
> could explain the problem.
>
> In the strace you give, Sympa tries to locate CPAN modules and then 
> finds the good one and then, there is this ioctl problem that I don' t 
> really understand. Some people had the same kind of troubles while 
> trying to printf something containing the "$!" variable (which 
> corresponds to the string of the last error). Problem is: we use "$!" in 
> a lot of places in Sympa, notably when trying to parse a template. All 
> the trace you showed us are related to the "Template" module.
>
> Maybe it was somewhat corrupted and you should try to reinstall it? I 
> think Sympa can open this module, otherwise it would die immediately.
> Do the other Sympa processes behave normally?
>
> Sorry I can't help you more than that.
>
> Regards,
>
> David
> > We're a bit baffled, and I'm wondering if there is anything anyone can
> > think of that would cause this, possibly related to hard drive or
> > filesystem errors?
> >
> > I managed to get an strace of the sympa.pl process before it dies, and
> > this is what I see. I fear it's not very informative - it's not
> > obvious to me what is eating the memory.
> >
> > First there is a lot of this:
> >
> > stat64("/home/sympa/bin/Template/Context.pmc", 0xbfde846c) = -1 ENOENT
> > (No such file or directory)
> > stat64("/home/sympa/bin/Template/Context.pm", 0xbfde83e4) = -1 ENOENT
> > (No such file or directory)
> > stat64("/usr/share/mhonarc/Template/Context.pmc", 0xbfde846c) = -1
> > ENOENT (No such file or directory)
> > stat64("/usr/share/mhonarc/Template/Context.pm", 0xbfde83e4) = -1
> > ENOENT (No such file or directory)
> > stat64("/etc/perl/Template/Context.pmc", 0xbfde846c) = -1 ENOENT (No
> > such file or directory)
> > stat64("/etc/perl/Template/Context.pm", 0xbfde83e4) = -1 ENOENT (No
> > such file or directory)
> > stat64("/usr/local/lib/perl/5.10.0/Template/Context.pmc", 0xbfde846c)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/local/lib/perl/5.10.0/Template/Context.pm", 0xbfde83e4) =
> > -1 ENOENT (No such file or directory)
> > stat64("/usr/local/share/perl/5.10.0/Template/Context.pmc",
> > 0xbfde846c) = -1 ENOENT (No such file or directory)
> > stat64("/usr/local/share/perl/5.10.0/Template/Context.pm", 0xbfde83e4)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/lib/perl5/Template/Context.pmc", 0xbfde846c) = -1 ENOENT
> > (No such file or directory)
> > stat64("/usr/lib/perl5/Template/Context.pm", {st_mode=S_IFREG|0644,
> > st_size=54043, ...}) = 0
> > open("/usr/lib/perl5/Template/Context.pm", O_RDONLY|O_LARGEFILE) = 6
> > ioctl(6, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfde81f8) = -1 ENOTTY
> > (Inappropriate ioctl for device)
> > _llseek(6, 0, [0], SEEK_CUR)            = 0
> >
> > I am able to open the file /usr/lib/perl5/Template/Context.pm without
> > a problem.
> >
> > It does that loop for some time for various modules in the Template
> > directory.
> >
> > Then the last thing seen before the memory output is:
> >
> > open("/usr/lib/perl5/Template/Iterator.pm", O_RDONLY|O_LARGEFILE) = 6
> > ioctl(6, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfde81f8) = -1 ENOTTY
> > (Inappropriate ioctl for device)
> > _llseek(6, 0, [0], SEEK_CUR)
> > read(6, " $self->{ _DATA };\n    my $size ="..., 4096) = 4096
> > _llseek(6, 8104, [8104], SEEK_SET)      = 0
> > _llseek(6, 0, [8104], SEEK_CUR)         = 0
> > close(6)                                = 0
> > brk(0x2a7c0000)                         = 0x2a7c0000
> > mmap2(NULL, 327680, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> > -1, 0) = 0xb74a6000
> > mremap(0xb74a6000, 327680, 331776, MREMAP_MAYMOVE) = 0xb731c000
> >
> > From there, it is mremap over and over again until it shows it's using
> > 12GB of memory, and then it dies with an out of memory error.
> >
> > Thanks for any input!
> >
> > Kristina