Recent outages - Dec 2015

[shona]

hi all,

Apologies for the recent outages.

We are spending some time today to look into the cause and hopefully stop it once and for all.

A few months ago, the issues were caused from the disk filling up, but once I had some time to sit down and go through everything I cleared it back to 50% full so we shouldn't have any issues on that front for a while.

The more recent issues have been because of the forum database, and there have been problems on and off for the last couple of months - usually during the early hours of the morning which wouldn't impact many/any users, but sometimes the issue has been spilling over into the morning.

Added to this, the company where our VPS server is hosted has just been rebooting the disk every time our database causes problems, rather than letting us know, so often a purge/repair/check is in process and gets killed by the reboot.

Thanks to all the people who email us when there is an issue, as I don't always have time to get on the site as soon as I wake up every morning and check everything is OK.

I'll post an update if we discover anything more.

thank you for your patience,
Shona

MASA Admin

[n0rk]

Where are we hosted and what sort of resource usage do we have, Shona?

[shona]

Digital Pacific VPS - 2GB RAM, 75GB disk space, 150GB traffic

Have found one possible problem - all backups were scheduled at the exact same time rather than staggered which could be crashing mysql, especially as the databases have grown over time. We've staggered them now and will monitor it tonight.