Disaster Recovery Plan

Problem

The organisation had a very rudimentary and minimal disaster recovery plan. Backups ran and services stayed up, but the questions a plan actually answers had never been written down: what do we do, in what order, who decides, and what can we actually get back.

Its public services also sit on the open internet, as a digital archive have to, and were taking the traffic that implies.

The finding

Working through what was genuinely recoverable rather than what was assumed turned up the thing worth finding: old data was not being maintained beyond 90 days. Three months, and then it was gone.

That is a short horizon for an archive whose entire purpose is keeping material readable over decades. It is also exactly the kind of gap that stays invisible while nothing goes wrong, because a backup regime that works perfectly at 30 days looks identical to one that works perfectly at 30 years — right up until somebody asks for something older.

Retention for most databases is now agreed at twelve months before deletion. That change came out of writing the document, not out of an incident, which is the entire argument for writing one.

What the plan covers

Comprehensive rather than a formality:

• Roles and responsibilities — a named DR team, with contacts, and who holds which responsibility when something is actually happening
• An escalation path diagram, so the answer to "who do I call" is not improvised at the worst possible moment
• Business impact analysis, with RTO and RPO defined per system, the critical digital assets and systems the ADS needs to function, and the dependencies between them
• Risk mitigation and prevention — patch management, MFA, TLS, password management, backups, data security, and the core security controls in place
• Staff training, both what is currently done and what should be
• Hardware — servers, workstations and peripherals
• DR scenarios, each with how to respond

Alongside the plan I tightened the Cloudflare rules protecting public services against hostile traffic, and aided in some TLS certificate renewal.

The training section led to work still in progress: a phishing simulation to test staff against realistic fake messages rather than assuming an annual slide deck has landed.

Outcome

A plan that describes the real recovery position rather than an optimistic one, a DR team that exists before it is needed, and a retention window four times longer than the one I found.

The general point is worth keeping. An organisation that knows it can only reach back 90 days can decide whether that is acceptable. One that has never checked cannot, and will find out at the worst possible time.