Tag: QA & Launch Readiness

  • Does it do what it’s supposed to do?

    Does it do what it’s supposed to do?

    Cogo was a carbon-footprint app being redeveloped, with a lot of moving parts and a lot of ways for things to go quietly wrong. I provided quality assurance throughout the redevelopment, combining methodical testing with the kind of curiosity that finds bugs nobody planned for.

    Curiosity first, then method

    I’ve always been good at breaking software. I get curious about “what happens if…” or sense that something’s off, and a few minutes later there’s a bug that needs fixing!

    I paired that instinct with structured testing of how the app behaved across different scenarios, so we got both the surprising bugs and the systematic coverage.

    I also kept an eye on live user data for signs of problems, especially after each update, since real users find things no test plan does. How I reported depended on where the bug was: for features still in development, I discussed each bug with developer responsible in the format they preferred. For the live app I wrote formal bug reports.

    Does it do what it’s supposed to do?

    Not every problem is a bug. A feature can work perfectly and still not be right. When testing a new feature, I started from the original wireframe and specifications from the product and behavioural science teams. I then checked the developers’ work against this. The question was never just “does it work?” but “does it do what it was designed to do?”

    That caught a different kind of problem: places where the build had quietly drifted from the intent or where the specification itself left room for interpretation. I took these back to the right people, so each one got resolved before the users ever saw the fearure.

    Testing journeys, not just screens

    Some of the trickiest problems only appear across a sequence of actions. One example was a feature letting users commit to changing their diet. They could swap out beef, go vegetarian or go plant-based for a period, and the app would show a reduced carbon footprint on their food transactions plus a total saving. Users could also change their minds, perhaps avoiding beef usually, joining in with Veganuary, then swapping back.

    That meant every transaction had to show the right footprint, saving and graphic for whichever diet period it fell into, and link to the right prompt for an even better option. I built and tested user journeys for each of these scenarios, then did the same for swapping fuel for public transport, planes for trains, and new clothes for second-hand. With hundreds of possible combinations, I found several bugs that a quick check would never have caught.

    Checking the data worked before launch

    The app also had to collect data on user behaviour so that the behavioural science team could test their research hypotheses. Before each feature launched, I tested that every tracked action was recording correctly in Amplitude and appearing in the charts correctly. This meant that the data would be trustworthy from day one rather than something we discovered was broken weeks later.

    Being “the person who knew how it works“

    The project had a complicated structure, so I spent a lot of time coordinating between departments. I worked within engineering but talked to the product and research teams daily, which made me the go-to person for how the app actually behaved and what it needed to do also. I pushed for documentation at every stage, so that knowledge was written down and not just in my head.

    Across a year of testing, a combination of curiosity, structured scenarios and checking against the original intent meant a wide range of issues were found and settled before they reached users, not after.

  • Finding and moving 1,500 pieces of content nobody could locate

    Finding and moving 1,500 pieces of content nobody could locate

    Thy Kingdom Come had a new Drupal website, and everything on the old proprietary site needed to move over: pages, stories, resources and videos, in English and six other languages.

    The catch was that nobody had a complete picture of what was actually on the old site. Much of the content was hard to find and not indexed or searchable on the back end, and a move this size can’t be done reliably if you don’t know what you’re moving.

    Start with what exists

    Initially worked from a large data dump, then filled the gaps going throge the old site “line-by-line” manual downloading content so nothing was missed. I organised the content into folders and made a plain-text copy of every item in every language. Each file also listed the images and PDFs it depended on, so nothing had to be hunted for at upload time.

    Prepping the move

    I held off starting the move until the new site had been thoroughly tested. Spending a month moving content onto a build that might still need changes would have meant doing some work twice. The number of times each item had to be edited was kept low to make the move as cost effective as possible.

    Keep track of everything

    As each item went onto the new site, I logged it in one large spreadsheet covering every translation. I recorded any errors, which were either fixed on the spot or reported back to the developers then tracked and marked as fixed later. At any moment in time, I could pinpoint what had moved, what hadn’t, and what needed attention.

    The result

    Over about a month, I moved around 1,500 items: the main pages, stories, resources and video resources in English, around 300 translated pages in French, Portuguese, Spanish, Japanese, Korean and Swahili, and 870 translated snippets, form items and buttons. I also created 130 new pages that had never been translated before, using a translation tool. Also I got to subtitle the Pope!

  • Getting a website and app ready for a ten-day global campaign

    Getting a website and app ready for a ten-day global campaign

    Thy Kingdom Come is a global prayer campaign with a fixed ten-day window in May. The date can’t move, so whatever is built has to work, in seven languages, from day one. The campaign had commissioned a new Drupal website and an Android app to be built by an external development team. There was no one in-house with capacity to test the website and app, turn what they wanted into specifications or move the multi-lingual website content over to the new platform. I came in to fill those gaps.

    Seeing the site the way visitors do

    Before any content was moved over, I tested the new site on desktop and mobile and reported 75 issues to the development team. The build was strong, but every build has problems. In this case, early on, images that displayed fine when logged in as an admin were missing for everyone else, and the language switcher bounced back to English for anyone not logged in. Problems like these are easy to miss when everyone building and checking a site is signed in as an administrator.

    Each issue went into a shared log with steps to reproduce, a priority from one to three, and a named owner, so the developers could act without having to ask questions. I worked with them to prioritise, specified fixes where the right answer wasn’t obvious, and then re-tested every fix.

    Testing also meant asking whether pages worked for the people using them. For example, I recommended that resource pages carry descriptive text rather than just a PDF download. That helps search visibility, and it helps older visitors and people on limited data know what they’re downloading before they commit.

    Specifying what was missing

    Some problems weren’t bugs, but gaps. I specified new features to resolve them, including:

    • an automated Videos page that pulled in video from elsewhere on the site
    • a full-width, responsive video embed for videos placed within text or in groups
    • a clear “Translated by Google” notice on pages translated automatically

    Translating between the two sides

    I also acted as the go-between for the developers and the organisations staff, so the charity got the look and usability they wanted without taking up much of the team’s time during their busy pre-launch period.

    The hardest task was the map on the Light Up The World page, which shows people’s “pledges”. It needed enough stars to look good without becoming unusable, and it mattered because the campaign’s social media strategy depended on people sharing their pledge.

    The app

    The Android app was the tougher of the two builds. I tested it, reported multiple bugs, and tested the fixes. I spent time going back and forth with the developers until the issues were resolved ahead of the campaign.

    The result

    The whole project went live on time, on a deadline that couldn’t move. The website is still the one Thy Kingdom Come uses today. Also I got to subtitle the Pope!