The internet is fragile

Dafydd Vaughan on 28 January 2013

At some point in the last few weeks a lot of the things I worked on back in Consumer Focus disappeared from the internet. The sites were turned off and the URLs redirected to the organisation homepage.

There is no explanation or notice explaining why you end up on that homepage and you would be forgiven for thinking that the sites and tools never existed. Every link ever made to any of those things are now broken.

It’s strange to see two years of your professional life just disappear. I can’t really blame Consumer Focus for this – they no longer have anyone with the skills or knowledge to keep the things running (and the organisation will close entirely very soon).

Unfortunately, this happens all too frequently online. Websites are rebuilt, things are moved, stuff is deleted without explanation or warning. Bits of our digital history are just lost forever. The internet is fragile.

Keeping everything operating and the same for the rest of eternity isn’t realistic.

Luckily, however, for UK government organisations The National Archives takes a copy of most government sites every couple of months (there are other similar projects aimed at non-gov sites). Even better, most of the things I worked on are sitting there in The National Archives (unfortunately, you have to know they are there because the redirection goes to the wrong place).

The Government Digital Service is currently migrating a large number of websites to GOV.UK. My colleague Paul Downey wrote about how they are putting lots of effort into stopping broken links and redirecting users as much as possible. Where a similar thing doesn’t exist on GOV.UK, they are notifying users and directing them to a copy in the archive.

From day 1, over 42 thousand URLs from Business Link were redirected. This number has grown continually since, most recently with the migration of the Department of Energy and Climate Change site to GOV.UK. GDS hasn’t always got it right and do occasionally find some URLs they’ve missed, but as soon as they realise, they are being added to the list and redirected properly.

If you turn off a website or radically change a site, it doesn’t take much to make sure it’s in The National Archives and redirect the URLs properly. That way you don’t break any links or bookmarks. Users don’t get lost or end up in dead ends. Redirecting isn’t difficult to do (particularly when just redirecting to the archives), it doesn’t cost much and doesn’t require any maintenance. It’s part of being a good citizen of the web.

Of course, this should just be the way things are done. For government sites, there are/were rules for handling URLs for website rebuilds and closures. This rule has been around for quite a while, but unfortunately tends to get ignored fairly frequently.

More redirects means fewer broken links, a less frustrating web for everyone and makes the web a little less fragile.

2 comments on “The internet is fragile

  1. This is a good post about an important and unsolved problem.

    The National Archives seem to have good coverage of content from public organisations but what about beyond that? It would be nice if The National Archives or somebody else could step in and save some of the third party content that risks being lost.

    Once a project or company or the like ends or is liquidated, the perceived value of the legacy content falls close to zero. You and I know that there IS value in archived content. Unfortunately there is no incentive to being a ‘good citizen of the web’, even for the originator of the content!

Comments are closed.

Any views stated here are my own and not those of my employer unless otherwise stated.