Several years ago a colleague introduced me to a talk by J. B. Rainsberger called “Integrated tests are a scam”. I really like and resonate with the ideas presented in this talk. The dependencies should be isolated and pushed outwards the system one’s working on. In ideal scenario, everything inside the small world (system) one has control of is covered with a robust yet flexible set of isolated tests. External dependencies provide various uncontrollable inputs, which one’s system needs to consume gracefully and with lots of forgiveness (think of the worst case scenarios — an html parser, or some fresh software that works with a decades old legacy system).
Yes, but then I’m confused if this advice is applicable when we’re talking about UI-heavy applications which I tend to work on in the recent years. For me, the most dreadful scenario is logging a user into the app. User opens the app, and is presented with a plethora of choices to sign up (three social media buttons plus a good old email-password combination), and then… well, then follows a huge tree of one-off states and verification steps. “I as a user login with my Google Apps for Work account, and if I’ve logged in previously, but my employer in the meanwhile has signed our company for a team account, then I’m presented with an option to choose either a private workspace or my team’s workspace.” Things like that.
It’s entirely possible to spec this whole process out with isolated unit tests. But in the end it doesn’t give me any real confidence. Because in the end everything that matters is what user sees on the screen, and how do you test that?
In ideal world, I want a single command that would launch a dozen of parallel UI tests that would go through every single one of the login scenarios, with every deviation on every step, and would record at a minimum a slide show of what user has seen on every step. Then I want to see a huge gallery of screenshots to be able to assess with a human eye how did it all go.