How Townsville's Public Art Archive Ended Up Full of Duplicates — and What's Being Done About ItUpdated

Townsville City Council's digital heritage archive — a repository holding thousands of photographs, maps and cultural records spanning more than a century — is currently undergoing a systematic duplicate-image replacement program after an internal audit identified significant data-quality problems that had built up over at least a decade.

The audit, completed in early 2026, found that a substantial portion of image records held across the council's library services platform contained either exact or near-identical duplicates, many of which had been entered multiple times during successive software migration projects. The practical effect is that researchers, First Nations community members seeking historical records, and members of Townsville's Pacific Island community using the archive for genealogical work have been pulling up redundant files — sometimes dozens of versions of the same image — instead of distinct records.

How the Problem Built Up Over Years

The roots of the problem trace back to at least 2013, when the council migrated its holdings from an older catalogue system into the first version of its current content management platform. A second migration occurred around 2018, and staff at the Townsville City Libraries network — which operates the central branch on Flinders Street as well as branches at Aitkenvale, Thuringowa Central and elsewhere across the local government area — have since identified that each transition introduced a new batch of duplicates rather than cleaning up existing ones.

Budget pressure is part of the explanation. Digitisation and archival work has historically competed for funding against more visible council priorities. The 2019 flood recovery consumed enormous administrative resources across North Queensland, and library services, like many non-emergency functions, absorbed cuts to backfill disaster response costs. By the time a dedicated digital collections officer position was properly staffed again at the Flinders Street branch, the duplicate problem had grown substantially.

The situation is not unique to Townsville. Libraries and local government archives across Queensland have grappled with similar legacy issues as digitisation programs from the 2000s and 2010s — many of them done quickly and without rigorous deduplication protocols — aged into problems rather than solutions. State Library of Queensland has published guidance on best-practice image metadata standards, but applying those standards retrospectively to large existing collections is labour-intensive work that requires both skilled staff and time.

What Replacement Actually Involves

Duplicate-image replacement in an archive context is not simply deleting extra files. Each image record carries its own metadata trail — accession numbers, donor attribution, condition notes, links to associated finding aids — and collapsing duplicates without destroying that provenance information requires careful manual review. Automating the detection step is now standard; resolving each flagged pair or cluster is still largely a human task.

Council library staff are currently working through the backlog using a combination of automated hash-matching software to flag exact duplicates and a secondary visual-review process for near-duplicates — images that were rescanned at different resolutions or cropped differently at different points in time. The Townsville Museum and Cultural Services team at the Pinnacles museum site on Bundock Street, which holds physical originals for a portion of the digital collection, is involved where questions about the authoritative version of an image require physical verification.

For users of the archive, the immediate practical reality is that some records are temporarily marked as under review and may not be publicly accessible during the resolution process. Community members — particularly those engaged with the First Nations treaty consultation process or researching Pacific Islander migration histories into North Queensland — have been advised to contact library staff directly if they need access to specific records that are currently flagged.

The council has not yet published a public timeline for completing the program, but library services staff have indicated the bulk of the highest-priority records — those most frequently accessed — are expected to be resolved before the end of 2026. The longer tail of less-accessed historical material will take longer. The lesson drawn from the experience is a practical one: migration projects need deduplication baked in from the start, not treated as a problem to be cleaned up later.