Materiaalit TestausOSYn James Bach-sessiosta – Testaus ilman testausta

Liitteenä materiaali TestausOSYn ja Agile Finlandin sessiosta, jossa puhujana testauksen superstara James Bach ( Esitys ”Testing Without Testing”.


Maaret Pyhäjärvi opened the session by welcoming James to Finland, thanking Oliver Vilson from Estonia fulfilling a community dream to bring this guru to Finland, F-Secure for the great premises for the sessions and James for sparing his time for community work. The community work for the Finnish testing community includes two parts: 1) this open session (150 people enrolled to be there in person, 110 people with bambuser – without proper sounds, lesson taken), and 2) FiST#1 Finnish Testing Peer Conference, with a new acronym for the first time.

The open session – Testing without Testing

James started his presentation by discussing the appearance of testing without the substance of testing and a presentation he’s done earlier on How to fake testing. We learned, that while this was the intro part of it, faking testing, as James sees it, is a common thing happening around the world. We heard of examples of what James considers faking, the folklore of how testing should be done and as particular example he talked about equivalance partitioning.

Next James moved to the theme of the collaboration with FAST this time: Testing without testing, the substance of testing without the appearance of testing – to a novice / manager eye, that is. The part where he discussed what testing is and isn’t was particularly interesting: that a thing a testers may do, and often do, making the product better, is not testing but with those who do testing, it’s time away from testing. Designs for how things work better and fixes were discussed as specific examples. Testers can sometimes take the designer tasks too, but also should be aware that the owners of those tasks may not always welcome a less-experienced designer into the work.

Then he talked about requirements checking, which is a part, but only a part of testing. Testing includes a human element and learning about what is being tested.

Looking at the slides, you can see that he identified around the visible checks (testing at the keyboard, already knowing what to do) a lot of things that need to happen for testing. The list of things is organized roughly in three circles around the center, showing increasing distance from the center, but still being within what is testing. James talked about many of these. I recall Galumphing (such a funny word!), which is a way of testing in a way that overwhelms the product; Discovery of non-bugs is about identifying features while testing that no one knew about; Sympathetic testing is about understanding why anyone would want the product and understanding the value it would give if it worked by testing shallowly; Recruiting helpers is about building relations to people who notice but dismiss bugs, and making it easy for them to tell the testers what they know without putting much effort on reporting and followup.

The last slide we did not talk about, but I heard the story during a planning dinner we had for the sessions. It’s an example of a thing this particular tester had done – learning a complex algorithm that was core to the risks of a product, implemented a prototype version to learn more about how that is supposed to work and through this approach for learning finding a problem in the actual product implementation that would be hard to catch. And while doing it, getting to James’ favorite theme: building credibility by doing things that are right and useful, and not easy.

The peer conference – FiST#1 on Testing without Testing

After the open session a smaller group of about 15 people continued with a peer conference for Friday and Saturday. The idea with a peer conference is to collect together a group of people that share experience reports – lessons learned in actual work – within the theme. To organize for this, we had an open call for invitations, as we knew we don’t know the relevant and willing people to invite them without asking about who wants to join. We took also a late joiner from the open session, who let us know he could join in the end of the open session and spent the whole two days in the peer conference. James played the content owner role, meaning that his questions had priority whereas the others’ questions were queued and handled in discussion threads, and he chose who gets to go when supporting the unsure experience presenters in getting their message out. Ru Cindrea took the facilitator role, and needed to silently fight the will to participate in the discussions, as the facilitator focuses on the process.

Experience reports delivered and discussed:

  • Maaret Pyhäjärvi: Excel checklist to replace test cases & status report, supporting discussions with team’s developers on test scope
  • Arttu Haataja: Doodling, drawing quick pictures to move between the technical details and end-user view and visually model concepts quickly to support learning
  • Laura Ojala: Short test plan to fulfill regulatory requirements and enable testing to happen
  • Ari Alsi: Setting non-functional target values
  • Anssi Lehtelä: Failing with different approaches and what was learned from the failures
  • Petri Kuikka: Bug triage process change where you address things faster and don’t collect an inventory of bugs already known of
  • Tuula Pääkkönen: Protecting other people’s time when more is asked than the team can deliver

In end of day 1, we placed all experience reports on James’s slide of Testing without Testing, to notice we were heavily skewed with our reports on the left-hand side of it, and on the second circle. The stuff on the inner circle may be harder to talk of.

We summarized the peer conference days with enthusiasm towards doing more of this. James taught us things about how to organize the session (that is, finetuning our facilitation practices) and about what is an experience report: it teaches something relevant through your experience to the audience and it should also have taught you something, something you’re excited about. If you introduce an idea, other can disagree on the theory. But if you introduce your experience, we can talk about it to understand it better, but we can’t disagree on the fact that it’s your experience.

Post Navigation