No excuses...

This post was originally intended to catalog a few tools to aide in viewing the internals of a system with an added chastisement to those who complain about not having enough information. In fact, I never got to CLR profiler or the array of performance counters made available in .NET. Instead, it turned into a brain dump of the ways in which I ask questions and seek out their answers. The journey took me somewhere I didn't expect. You'll see what I mean...

I've been reading a lot of posts lately talking about how testers cannot test lower level functionality, structures, etc. because they don't have access to the code.

If you're working with a .NET application, my response is "Pish-posh".

Download Reflector. Nefariously point it in the direction of those impenetrable .NET assemblies. Behold the fruit of your efforts:



This is an actual disassembly of a program I was writing using the WebAii automation framework. Though my example is incomplete, you can still see what's going on under the hood.

"But Zach", you protest, "I can't read all that code-y stuff! What kind of bugs can I find doing that?"

Well, using this tool, I can think of one possible bug that doesn't require intimate knowledge of C#, .NET, CLR, or IL disassembly techniques. How about the fact that you can go out and download a tool that can expose your stakeholder's intellectual property after a few clicks of a button? If the company is concerned about protecting its secrets, they should be concerned about the relative ease with which one could reverse-engineer their system.

There are also plugins available that allow you to view differences between two versions of the same assembly. If a bug-fix is delivered, you may be able to ascertain where that change was made. What do you find there?

"But our application is web-based! I can't just look at assemblies on my local machine!" More pish-posh! Your browser has a "view source" function. Use it to peer in behind the curtain. Want some more power than that? Download FireBug. More power than that? Download Wireshark.

But tools are not the answer to the question. In fact, they are distracting if you don't understand the question you're asking. Put another way, I'm not persuaded by the argument, "I don't know what all this code stuff is all about. I just mash buttons." It is time to take a lesson from the economy and learn something new. The internet has now made this a lot easier than it was when I was trying to learn Raster Interrupts on my Commodore 64.

Your Mission:
Make a Windows application containing one button. When you click this button, a dialog box appears that reads, "Hello, World!". I won't tell you HOW to do this, that is YOUR job. But a few points are hidden in this exercise.

Learn Developer Empathy.
Many communities like to push the idea that testing and development should be at odds. I think this is a dangerous idea. To me, developer empathy is NOT the same thing as BUG empathy because developers are not the problem. Sure, they may have coded it in wrong, but you're not going to be able to catch all the bugs in the system ANYWAY. So none of us are doing are jobs perfectly, cool?

As the saying goes, you catch more flies (which are bugs) with Honey than with Vinegar. The 'honey' here is your willingness to learn about life on their side of the table. I guarantee that if you approach a group of developers with the problems you encounter while doing this exercise, you'll receive no end of advice. You can also use your experience to frame questions about the system under test: "I notice this alert comes up in the browser when I enter in invalid information. How is that different from my little application?" These are the kinds of questions I asked developers. Just remember to ask when it is convenient for them. I always walk away with more knowledge and more questions. Which leads me to...

Mind Map your journey.
Create a mind map. In the middle, enter in the task of creating that Windows application. Before you start trying to figure out coding, try to articulate your confusion and frustration. Create nodes that encapsulate your questions: "What is a Windows application?", "Why do I hear so much about, 'Hello World'?" As you find answers, add them as nodes of the question itself. But you're not done because that answer should have created several new questions for you. Add them and repeat. Download this mind mapping program if you must.

As testers, ours is not only a journey of questioning things, but in finding out the answers to those questions. If someone plops down a huge system for you to test, how are you going to tackle it? The question is always antecedent to the approach. So if you don't know, find out and track the way(s) in which you found out. Like watering holes in the desert, there will be places you perennially come back to for answers. Where are they? You won't find them unless you go looking for them.

Brush up on your Google-Fu
Learn to ask questions in different ways, including how to ask them in a Google query. Remove extraneous words. Group the related terms together to eliminate false positive results. Reduce the results to a certain domain, a certain file type, within a certain period of time. This ability can greatly improve the signal-to-noise ratio.

Don't brush up on your Google-Fu
Send that query out without all the constraints mentioned above. Misspell a word. Ungroup terms. Expand the time frame. Select a result from page 10, not page 1. Select the LAST result. You will be surprised by how much you uncover in the journey. Bob Ross called them happy little accidents. I cannot tell you how many discoveries I made on the way to answer a completely DIFFERENT question. In fact, most of the downloads on this page are result of another train of thought.

Now, you've downloaded Microsoft C# 2008 Express and written your program. Go ahead and set up a new, small project for yourself and see it through to completion. Use TDD,BDD, NUnit or MbUnit. (What do all those terms mean? You don't seriously expect me to answer it for you, do you?) You will have gained more knowledge of how to view the internals of the system. These internals are not only the constituent bits that comprise the application, but the human processes around their creation. Knowing these processes can guide your questions later on. No, you're not developer level but you are CONVERSANT. Being conversant means being able to have a conversation, and conversations lead to questions which lead to discoveries which leads to...

Have fun playing with the tools. Just don't start/stop there.