"All testing is exploratory".
I saw the phrase pop up again in my good friend's bio, and stopped to think if we agree or disagree.
All testing I ever do is exploratory. All testing that qualifies as good testing for me is exploratory. But all testing that people ask from me definitely is not exploratory.
I worked at one product company on two separate batches of about 3 years each, and with 10 years in between. If there is a type of company that needs good, exploratory testing to survive, product companies are this. The whole concept originated 35 years ago from Silicon Valley product companies Cem Kaner was working in. Yet when I first joined, exploratory testing was something we did after test cases.
We wrote test cases, tracked running of those test cases, and had some of the better tooling in continuously moving test target version with our Word-Excel in-house tooling. What made the tooling better than any of the ones I have now in my use was the in-built idea of primarily needing to understand when you last verified a particular test idea, since every change effectively cancels out results of all the things you've done before.
On top of test cases, we were asked to do exploratory testing. We were guided to test our tests so that we did not obey the steps, but introduced variance. We were guided to take a moment after each test to think what we would test because of what we had just learned, and do it. We were guided in taking half-a-day every week in just putting the test cases aside and testing freely.
It was clear that all testing was not exploratory testing.
10 years later, there was no test cases. There was one person who would do "exploratory testing" meaning they would follow user flows to confirm things without documentation, without any ability to explain what they were doing other than rare bugs they might run into, missing out a lot of problems. And then there was test automation that codified lessons of how to keep testing continuously, discovered through detailed exploration finding problems.
It was clear that testing they called now exploratory testing was not exploratory testing. It was test automation avoidance. And the testing they called test automation was the real exploratory testing.
I get to go around companies and inside companies, and I can confirm that we are far from all testing being exploratory. We have still tyranny of test cases in many places. We will have test automation done without exploring while doing it, taking away some of its value.
We have managers asking for a "planned" approach to testing, asking for "requirements coverage". They ask those as a proxy to good testing, and yet in asking those, often end up getting worse testing, testing that is not exploratory.
Opposite to exploratory is confirmatory. It does not seek the holes between the things you realized to ask for, but only things you realized to ask for. And you want more than you know to ask for.
So I keep going to companies, to introduce exploratory testing.
I bring down test cases, I move documenting tests to automation.
I convince people to not care so much for who does what, but work together to learn. Developers are great exploratory testers if only they let go of the recipe where they do what they were told, and own up to creating a worthwhile solution.
I break ideas of exploratory testing as something you do on top of other things testing. I introduce foundations of confirming what we know being true and learning, and then building figuring out unknown on top of it.
We read requirements, we track requirements, we use them as testing starters. But instead of confirming what is there, we seek to find what is not.
All testing will not be exploratory testing while we write and execute tests cases. All testing may have an exploratory angle. But we can do better.
