I recently made the argument (successfully?) that computers automate by nature; that anyone who uses a computer is by definition an automation engineer. Couple that with the observation that humans are always "testing something", and one could suggest that anyone who uses a computer is a test automation engineer.
For example, the Internet is automation of data retrieval, one that is engineered based on how we use it. My parents would not immediately identify weather widgets on Google as test automation. They just want to know if they need an umbrella when they go out. I, however, see a dashboard that allows them to interpret important data relevant to their needs and allows them to make decisions based upon their interpretation of that data.
Did they program it? Well, yes and no. While they did not sit down and write the statements that brought this convenience into their lives, they did set its preferences, position where they deemed fit, and ensured that it welcomes them with each new visit. Without these interfaces, the tool is of less use to them.
Another example: spreadsheets help automate data manipulation and presentation. My wife would not immediately identify a budget workbook as test automation. She would see it more of a way to track income and expenses, and how to identify if we're on target this month, can we go on vacation, etc. I, however, see it as a tool that allows us to repeatedly take all our disparate financial activity and produce a cohesive picture that allows us to manage our money better - to make better decisions (or to know the precise value of regret).
Did we program it? Well, yes and no. While we did not sit down and write Excel, we did set set up its inputs and its formulas using the interfaces exposed to us.
The differences between my definition and what is typically identified as test automation engineering have to do with complexity and intention - the scope of our solutions and how aware we are that testing/automating is going on in the first place. Like regular users, we manipulate the interfaces available to us in order to obtain data, arrange it into meaningful statements, test those statements for validity, and make decisions based upon those tests. Where we differ is how aware we are of the interfaces available to us, and how extensively we use them in our tasks.
One side note: neither of the tools used in the my examples actually automate information. That is the sole purview of people - to identify patterns and meanings. These tools may re-organize the underlying data, but we are the ones who are burdened with determining if this stuff makes sense or not; if it is valuable or not.
Hopefully you get my gist: we all use computers to automate tasks that are important to us. We all use more automation tools than we realize. We know more than we think we do, and do ourselves a disservice if we believe that the high-priests of automation are the only people worthy enough to discuss its doctrine.
Wednesday, May 30, 2012
Tuesday, May 22, 2012
Test Automation
Note: I'm writing this in the true hope that, within my lifetime, time travel will become a reality. And that future me, given to indulge in the kinds of preoccupations beset upon the nouevau riche, will visit past me with this post firmly in hand. I look forward to the chastisement that I gave myself 3 years ago.
Warning: It is a bit complain-y.
Example #1: I watched my niece fumble through some math problems. In her concentration, she utilized humanity's greatest aide to the mind addled with numerical perplexity: she counted on her fingers.
We have these fingers. But we also have the abacus, calculators, Babbage's adding machine, personal computers, cell phones, etc. We have a lot of ways to solve the problem of '2+2', and yet we still count.
We've automated the means, but the need remains.
Example #2: From papyrus to the the Gutenberg press - we like getting our message out. In recent years, the effort it takes to get a message out has become almost laughable. I am writing this blog post at home and will publish it to a medium that goes around the world instantly. Whether it is read....
We have these papers and pencils. But we also have blogs, Twitter, Facebook. We have lots of ways to solve the problem of 'how do I communicate this idea to others'; and, as evidenced by this post, we're in no shortage of opinions or things to say..
We've automated the means, but the need remains.
The point: the computer is a tool that automates tasks we've programmed it to do. Be it math problems or publishing daily drivel, it exists to make things easier by abstracting away all the gritty mechanics of it all. But we use it as a tool towards a greater end than just itself. What's left after our tools do their work is the thing we sought in the first place. My neice found her answer. My voice found its audience.
If you accept that point as valid, then you may see why I now think the obsession with test automation is ridiculous, especially when it wears itself as a badge of honor. We don't hear writers extol their prowess at automating words. We don't hear scientists happily exclaim that while they have not solved the problem at hand, they have several thousand lines of framework ready to log the stack trace of all their functional programming. It may be shocking for some to consider that computers are now so ubiquitous that everyone is a test automation engineer.
Perhaps I'm jaded, but this is partially why I stopped blogging about my journey in software testing. It is why I no longer darken the virtual door posts of the online watering holes wherein gather my former compatriots. The software testing I was enamoured with got stuck in a rut where the same functional test automation question repeated over and over ad infinitum. Don't get me wrong - I'm eternally grateful for all the help given to me when my voice rose in chorus with the din of noob-posts. I just fear that many newcomers now (continue to?) see this brand of "testing" as a means to be amazed at their own brilliance.
Experience taught me things ambituion could not. Software testing is not the problem that needs to be solved. Who hasn't felt the narcotic effect of basking in the glow of perceived efficiency? However, my current role requires me to manage the testing I used to automate, and to do so from the customer persepctive. It is here that I hear the echoes of my naivete resonating back across the years. Decisions I once thought dripping with the kind of genius that would cause the world to beat a path to my door now seem to be unnecessary complications. The fun was in writing all that code and dreaming of the time it would save. The hell is now in maintaining it, training to it, supporting it. Ugh.
Example #3: As an adolescent learning to play the drums, I gazed upon the massive arsenal of drums and cymbals that MTV served up daily. "This," I thought to myself, "is what playing the drums IS!". And so I asked Dad for a new cymbal. His response was, "learn to play what you've got". And so, 2 days later, I returned exclaiming that I had indeed learned to play what I had and was ready to receive my reward. He laughed and ignored my request. Now, years later, I know why.
Playing drums is hard. It's demanding. It's exhausting. It requires discipline and experience. Mistakes are mandatory avenues towards greater mastery. And so it is with software. So the next time I read someone wanting to know how to automate testing, I'm will fight the urge to respond, "learn to play what you've got".
But more than likely, I'll just ignore it. Before you think me cruel, you should hear me play drums.
Warning: It is a bit complain-y.
Example #1: I watched my niece fumble through some math problems. In her concentration, she utilized humanity's greatest aide to the mind addled with numerical perplexity: she counted on her fingers.
We have these fingers. But we also have the abacus, calculators, Babbage's adding machine, personal computers, cell phones, etc. We have a lot of ways to solve the problem of '2+2', and yet we still count.
We've automated the means, but the need remains.
Example #2: From papyrus to the the Gutenberg press - we like getting our message out. In recent years, the effort it takes to get a message out has become almost laughable. I am writing this blog post at home and will publish it to a medium that goes around the world instantly. Whether it is read....
We have these papers and pencils. But we also have blogs, Twitter, Facebook. We have lots of ways to solve the problem of 'how do I communicate this idea to others'; and, as evidenced by this post, we're in no shortage of opinions or things to say..
We've automated the means, but the need remains.
The point: the computer is a tool that automates tasks we've programmed it to do. Be it math problems or publishing daily drivel, it exists to make things easier by abstracting away all the gritty mechanics of it all. But we use it as a tool towards a greater end than just itself. What's left after our tools do their work is the thing we sought in the first place. My neice found her answer. My voice found its audience.
If you accept that point as valid, then you may see why I now think the obsession with test automation is ridiculous, especially when it wears itself as a badge of honor. We don't hear writers extol their prowess at automating words. We don't hear scientists happily exclaim that while they have not solved the problem at hand, they have several thousand lines of framework ready to log the stack trace of all their functional programming. It may be shocking for some to consider that computers are now so ubiquitous that everyone is a test automation engineer.
Perhaps I'm jaded, but this is partially why I stopped blogging about my journey in software testing. It is why I no longer darken the virtual door posts of the online watering holes wherein gather my former compatriots. The software testing I was enamoured with got stuck in a rut where the same functional test automation question repeated over and over ad infinitum. Don't get me wrong - I'm eternally grateful for all the help given to me when my voice rose in chorus with the din of noob-posts. I just fear that many newcomers now (continue to?) see this brand of "testing" as a means to be amazed at their own brilliance.
Experience taught me things ambituion could not. Software testing is not the problem that needs to be solved. Who hasn't felt the narcotic effect of basking in the glow of perceived efficiency? However, my current role requires me to manage the testing I used to automate, and to do so from the customer persepctive. It is here that I hear the echoes of my naivete resonating back across the years. Decisions I once thought dripping with the kind of genius that would cause the world to beat a path to my door now seem to be unnecessary complications. The fun was in writing all that code and dreaming of the time it would save. The hell is now in maintaining it, training to it, supporting it. Ugh.
Example #3: As an adolescent learning to play the drums, I gazed upon the massive arsenal of drums and cymbals that MTV served up daily. "This," I thought to myself, "is what playing the drums IS!". And so I asked Dad for a new cymbal. His response was, "learn to play what you've got". And so, 2 days later, I returned exclaiming that I had indeed learned to play what I had and was ready to receive my reward. He laughed and ignored my request. Now, years later, I know why.
Playing drums is hard. It's demanding. It's exhausting. It requires discipline and experience. Mistakes are mandatory avenues towards greater mastery. And so it is with software. So the next time I read someone wanting to know how to automate testing, I'm will fight the urge to respond, "learn to play what you've got".
But more than likely, I'll just ignore it. Before you think me cruel, you should hear me play drums.
Tuesday, December 22, 2009
Business-Facing Tester Helps Developer; Film at 11
These days find me sitting squarely on the customer side of the development cycle. I haven't let that limit my approach, though.
I still use automation, APIs, logging, checklists - basically all the tools I used when I was on the other side. Perhaps this delineation of 'sides' is unnecessary when both are striving to work together. This is just a unique configuration, one where a technical "expert" is collocated with the customers instead of SMEs being collocated with the development team.
One day, our software vendor set up a WebEx to walkthrough the installation of their software. I shared my desktop and began to follow the developer's instructions. The first one hour session ended when we encountered an anti-virus policy that was not accounted for in the install package. Simply put, our company disallows "autorun.inf" files to be created on the file system, but the software has a dependency whose installer unpacks just such a file. This looked like it would be a job for our internal IT department, who would ultimately be responsible for deploying this software anyway. One session down, one to go.
In our second session, we were able to continue the installation by temporarily bypassing the blocking process. Since this is what the IT department would have to do, I felt it was not masking any potential issues. The dependency installation was taking more time than the developer expected. We tracked it down to another internal IT process that was pushed out recently. Search indexing was working hard to account for the deluge of new files being created on the file system. Once we disabled that process for the sake of brevity, we continued on to the last issue.
The package was finally installed on my system and all that remained was to update it. It was at this time that my co-worker Shane asked what would happen if we attempted to update while a session was active. We explored this scenario with the developer and were able to confirm that the update did not occur - as we had expected. However, the developer was perplexed as to why the active session window did not close during the update attempt.
The developer mentioned that the recent buyout of their company required them to update all their company branding, including their window titles. It was possible that the routine responsible for closing the session window was looking for something with Company X in the title, but was actually Company Y. The developer began to search through their code to determine if this was the case.
I thought I could prove his point rather easily during our WebEx. I fired up AutoIt and compiled the following code:
WinSetTitle( "Company Y", "Company X" )
I ran this on the remote box where we had an open session, observed that the window title changed, then re-ran the update utility. Sure enough, the window closed and we were able to move on. The developer gave us a great compliment, exclaiming that what we had just done was, "a pretty cool hack". We were able to confirm his suspicions in real time, and thus confirmed where his fixes needed to occur.
I later looked back on that moment with much fondness. It reminded me that there are still some technical activities we can engage in that provide value for both sides of the effort. After all, we all just want to deliver good stuff. Who cares if some customers understand those lower-level details, provided the testing represents the customer's interest?
I still use automation, APIs, logging, checklists - basically all the tools I used when I was on the other side. Perhaps this delineation of 'sides' is unnecessary when both are striving to work together. This is just a unique configuration, one where a technical "expert" is collocated with the customers instead of SMEs being collocated with the development team.
One day, our software vendor set up a WebEx to walkthrough the installation of their software. I shared my desktop and began to follow the developer's instructions. The first one hour session ended when we encountered an anti-virus policy that was not accounted for in the install package. Simply put, our company disallows "autorun.inf" files to be created on the file system, but the software has a dependency whose installer unpacks just such a file. This looked like it would be a job for our internal IT department, who would ultimately be responsible for deploying this software anyway. One session down, one to go.
In our second session, we were able to continue the installation by temporarily bypassing the blocking process. Since this is what the IT department would have to do, I felt it was not masking any potential issues. The dependency installation was taking more time than the developer expected. We tracked it down to another internal IT process that was pushed out recently. Search indexing was working hard to account for the deluge of new files being created on the file system. Once we disabled that process for the sake of brevity, we continued on to the last issue.
The package was finally installed on my system and all that remained was to update it. It was at this time that my co-worker Shane asked what would happen if we attempted to update while a session was active. We explored this scenario with the developer and were able to confirm that the update did not occur - as we had expected. However, the developer was perplexed as to why the active session window did not close during the update attempt.
The developer mentioned that the recent buyout of their company required them to update all their company branding, including their window titles. It was possible that the routine responsible for closing the session window was looking for something with Company X in the title, but was actually Company Y. The developer began to search through their code to determine if this was the case.
I thought I could prove his point rather easily during our WebEx. I fired up AutoIt and compiled the following code:
WinSetTitle( "Company Y", "Company X" )
I ran this on the remote box where we had an open session, observed that the window title changed, then re-ran the update utility. Sure enough, the window closed and we were able to move on. The developer gave us a great compliment, exclaiming that what we had just done was, "a pretty cool hack". We were able to confirm his suspicions in real time, and thus confirmed where his fixes needed to occur.
I later looked back on that moment with much fondness. It reminded me that there are still some technical activities we can engage in that provide value for both sides of the effort. After all, we all just want to deliver good stuff. Who cares if some customers understand those lower-level details, provided the testing represents the customer's interest?
Friday, August 7, 2009
Facebook's Hidden Bug
Facebook allows me to gather friends ad infinitum. Currently, I have 279.
As I add friends, I offer up chunks of Facebook real estate for their posts, thoughts, comments, etc. But keeping up with 279 friends causes me to spend a lot of time reading these little nuggets of personality.
So I limit my interaction, as I've noticed that I tend to get sucked in. Some of my friends notice the same tendency. I know this because I read their comments. Just not all of them.
The bottom line is this: I am a user of Facebook whose sole purpose is to visit the site briefly to find out what people of varying importance to me are saying.
That is the User. The problem here is that everyone has something to say, but there is only one method of consuming that information: reading. Reading takes time and reading takes space. While having to read is not the problem - I'm not that lazy - the volume of items to read has become insufferable. As more and more people post their little gems with great rapidity (e.g., Twitter), my home page becomes a teenage-ADD-brain dump of middle-age technical professionals, musicians, and old high school chums; most of whom are being rudely run off the page by Twitter-bullies.
I think Facebook should provide an option where users are provided posts grouped within a certain time frame. Only the latest entry by a friend is shown with an option to see more. Think of something like a tree view where the root is "Posts within the last 8 hours." The child nodes would be unique friend avatars with the most recent item and a count of how many posts they have made within that time frame. Collapsed under that would be children nodes with the actual posts, comments being children of THAT node. Something like this:
"Posts within the last 8 hours"
--->Bill Clinton "Taking a nap!" (3)
------>"Taking a nap!"
------>"Eating a ham sandwich."
--------->Al Gore
------------>"Again?"
--------->Hillary Clinton
------------>"What's new with that?"
------>"Rescuing reporters."
--->George W. Bush "Poking Cattle" (2)
------>"Poking Cattle"
------>"Tipping Cattle"
etc...
In addition, I should be able to specify classes of friends. Friends from high school, friends from work, good friends, acquaintances, friends I have accepted because I didn't want to be rude but really can't remember them at all.
I should be able to rank these classes such that the higher the rank, the higher up on the page. Yes, older posts may appear before newer ones, but that's okay with me because I know that it is time boxed to my requested window.
Currently, Facebook just blithely assumes I want to read it all at once and treats the posts with a flat hierarchy, which is really nothing more than LIFO. I would love to have an option like this.
As I add friends, I offer up chunks of Facebook real estate for their posts, thoughts, comments, etc. But keeping up with 279 friends causes me to spend a lot of time reading these little nuggets of personality.
So I limit my interaction, as I've noticed that I tend to get sucked in. Some of my friends notice the same tendency. I know this because I read their comments. Just not all of them.
The bottom line is this: I am a user of Facebook whose sole purpose is to visit the site briefly to find out what people of varying importance to me are saying.
That is the User. The problem here is that everyone has something to say, but there is only one method of consuming that information: reading. Reading takes time and reading takes space. While having to read is not the problem - I'm not that lazy - the volume of items to read has become insufferable. As more and more people post their little gems with great rapidity (e.g., Twitter), my home page becomes a teenage-ADD-brain dump of middle-age technical professionals, musicians, and old high school chums; most of whom are being rudely run off the page by Twitter-bullies.
I think Facebook should provide an option where users are provided posts grouped within a certain time frame. Only the latest entry by a friend is shown with an option to see more. Think of something like a tree view where the root is "Posts within the last 8 hours." The child nodes would be unique friend avatars with the most recent item and a count of how many posts they have made within that time frame. Collapsed under that would be children nodes with the actual posts, comments being children of THAT node. Something like this:
"Posts within the last 8 hours"
--->Bill Clinton "Taking a nap!" (3)
------>"Taking a nap!"
------>"Eating a ham sandwich."
--------->Al Gore
------------>"Again?"
--------->Hillary Clinton
------------>"What's new with that?"
------>"Rescuing reporters."
--->George W. Bush "Poking Cattle" (2)
------>"Poking Cattle"
------>"Tipping Cattle"
etc...
In addition, I should be able to specify classes of friends. Friends from high school, friends from work, good friends, acquaintances, friends I have accepted because I didn't want to be rude but really can't remember them at all.
I should be able to rank these classes such that the higher the rank, the higher up on the page. Yes, older posts may appear before newer ones, but that's okay with me because I know that it is time boxed to my requested window.
Currently, Facebook just blithely assumes I want to read it all at once and treats the posts with a flat hierarchy, which is really nothing more than LIFO. I would love to have an option like this.
Wednesday, August 5, 2009
Letters to a future me about...automation
The point of the product is not cool automation frameworks. Automation is a byproduct of the testing effort. The end-user may benefit from automation much the same way that my cereal benefits from automatic milking. Which is to say it may be helpful, but is putting the cart before...um...a quadrupedal animal of some variety.
Saturday, August 1, 2009
No excuses...
This post was originally intended to catalog a few tools to aide in viewing the internals of a system with an added chastisement to those who complain about not having enough information. In fact, I never got to CLR profiler or the array of performance counters made available in .NET. Instead, it turned into a brain dump of the ways in which I ask questions and seek out their answers. The journey took me somewhere I didn't expect. You'll see what I mean...
I've been reading a lot of posts lately talking about how testers cannot test lower level functionality, structures, etc. because they don't have access to the code.
If you're working with a .NET application, my response is "Pish-posh".
Download Reflector. Nefariously point it in the direction of those impenetrable .NET assemblies. Behold the fruit of your efforts:
This is an actual disassembly of a program I was writing using the WebAii automation framework. Though my example is incomplete, you can still see what's going on under the hood.
"But Zach", you protest, "I can't read all that code-y stuff! What kind of bugs can I find doing that?"
Well, using this tool, I can think of one possible bug that doesn't require intimate knowledge of C#, .NET, CLR, or IL disassembly techniques. How about the fact that you can go out and download a tool that can expose your stakeholder's intellectual property after a few clicks of a button? If the company is concerned about protecting its secrets, they should be concerned about the relative ease with which one could reverse-engineer their system.
There are also plugins available that allow you to view differences between two versions of the same assembly. If a bug-fix is delivered, you may be able to ascertain where that change was made. What do you find there?
"But our application is web-based! I can't just look at assemblies on my local machine!" More pish-posh! Your browser has a "view source" function. Use it to peer in behind the curtain. Want some more power than that? Download FireBug. More power than that? Download Wireshark.
But tools are not the answer to the question. In fact, they are distracting if you don't understand the question you're asking. Put another way, I'm not persuaded by the argument, "I don't know what all this code stuff is all about. I just mash buttons." It is time to take a lesson from the economy and learn something new. The internet has now made this a lot easier than it was when I was trying to learn Raster Interrupts on my Commodore 64.
Your Mission:
Make a Windows application containing one button. When you click this button, a dialog box appears that reads, "Hello, World!". I won't tell you HOW to do this, that is YOUR job. But a few points are hidden in this exercise.
Learn Developer Empathy.
Many communities like to push the idea that testing and development should be at odds. I think this is a dangerous idea. To me, developer empathy is NOT the same thing as BUG empathy because developers are not the problem. Sure, they may have coded it in wrong, but you're not going to be able to catch all the bugs in the system ANYWAY. So none of us are doing are jobs perfectly, cool?
As the saying goes, you catch more flies (which are bugs) with Honey than with Vinegar. The 'honey' here is your willingness to learn about life on their side of the table. I guarantee that if you approach a group of developers with the problems you encounter while doing this exercise, you'll receive no end of advice. You can also use your experience to frame questions about the system under test: "I notice this alert comes up in the browser when I enter in invalid information. How is that different from my little application?" These are the kinds of questions I asked developers. Just remember to ask when it is convenient for them. I always walk away with more knowledge and more questions. Which leads me to...
Mind Map your journey.
Create a mind map. In the middle, enter in the task of creating that Windows application. Before you start trying to figure out coding, try to articulate your confusion and frustration. Create nodes that encapsulate your questions: "What is a Windows application?", "Why do I hear so much about, 'Hello World'?" As you find answers, add them as nodes of the question itself. But you're not done because that answer should have created several new questions for you. Add them and repeat. Download this mind mapping program if you must.
As testers, ours is not only a journey of questioning things, but in finding out the answers to those questions. If someone plops down a huge system for you to test, how are you going to tackle it? The question is always antecedent to the approach. So if you don't know, find out and track the way(s) in which you found out. Like watering holes in the desert, there will be places you perennially come back to for answers. Where are they? You won't find them unless you go looking for them.
Brush up on your Google-Fu
Learn to ask questions in different ways, including how to ask them in a Google query. Remove extraneous words. Group the related terms together to eliminate false positive results. Reduce the results to a certain domain, a certain file type, within a certain period of time. This ability can greatly improve the signal-to-noise ratio.
Don't brush up on your Google-Fu
Send that query out without all the constraints mentioned above. Misspell a word. Ungroup terms. Expand the time frame. Select a result from page 10, not page 1. Select the LAST result. You will be surprised by how much you uncover in the journey. Bob Ross called them happy little accidents. I cannot tell you how many discoveries I made on the way to answer a completely DIFFERENT question. In fact, most of the downloads on this page are result of another train of thought.
Now, you've downloaded Microsoft C# 2008 Express and written your program. Go ahead and set up a new, small project for yourself and see it through to completion. Use TDD,BDD, NUnit or MbUnit. (What do all those terms mean? You don't seriously expect me to answer it for you, do you?) You will have gained more knowledge of how to view the internals of the system. These internals are not only the constituent bits that comprise the application, but the human processes around their creation. Knowing these processes can guide your questions later on. No, you're not developer level but you are CONVERSANT. Being conversant means being able to have a conversation, and conversations lead to questions which lead to discoveries which leads to...
Have fun playing with the tools. Just don't start/stop there.
I've been reading a lot of posts lately talking about how testers cannot test lower level functionality, structures, etc. because they don't have access to the code.
If you're working with a .NET application, my response is "Pish-posh".
Download Reflector. Nefariously point it in the direction of those impenetrable .NET assemblies. Behold the fruit of your efforts:
This is an actual disassembly of a program I was writing using the WebAii automation framework. Though my example is incomplete, you can still see what's going on under the hood.
"But Zach", you protest, "I can't read all that code-y stuff! What kind of bugs can I find doing that?"
Well, using this tool, I can think of one possible bug that doesn't require intimate knowledge of C#, .NET, CLR, or IL disassembly techniques. How about the fact that you can go out and download a tool that can expose your stakeholder's intellectual property after a few clicks of a button? If the company is concerned about protecting its secrets, they should be concerned about the relative ease with which one could reverse-engineer their system.
There are also plugins available that allow you to view differences between two versions of the same assembly. If a bug-fix is delivered, you may be able to ascertain where that change was made. What do you find there?
"But our application is web-based! I can't just look at assemblies on my local machine!" More pish-posh! Your browser has a "view source" function. Use it to peer in behind the curtain. Want some more power than that? Download FireBug. More power than that? Download Wireshark.
But tools are not the answer to the question. In fact, they are distracting if you don't understand the question you're asking. Put another way, I'm not persuaded by the argument, "I don't know what all this code stuff is all about. I just mash buttons." It is time to take a lesson from the economy and learn something new. The internet has now made this a lot easier than it was when I was trying to learn Raster Interrupts on my Commodore 64.
Your Mission:
Make a Windows application containing one button. When you click this button, a dialog box appears that reads, "Hello, World!". I won't tell you HOW to do this, that is YOUR job. But a few points are hidden in this exercise.
Learn Developer Empathy.
Many communities like to push the idea that testing and development should be at odds. I think this is a dangerous idea. To me, developer empathy is NOT the same thing as BUG empathy because developers are not the problem. Sure, they may have coded it in wrong, but you're not going to be able to catch all the bugs in the system ANYWAY. So none of us are doing are jobs perfectly, cool?
As the saying goes, you catch more flies (which are bugs) with Honey than with Vinegar. The 'honey' here is your willingness to learn about life on their side of the table. I guarantee that if you approach a group of developers with the problems you encounter while doing this exercise, you'll receive no end of advice. You can also use your experience to frame questions about the system under test: "I notice this alert comes up in the browser when I enter in invalid information. How is that different from my little application?" These are the kinds of questions I asked developers. Just remember to ask when it is convenient for them. I always walk away with more knowledge and more questions. Which leads me to...
Mind Map your journey.
Create a mind map. In the middle, enter in the task of creating that Windows application. Before you start trying to figure out coding, try to articulate your confusion and frustration. Create nodes that encapsulate your questions: "What is a Windows application?", "Why do I hear so much about, 'Hello World'?" As you find answers, add them as nodes of the question itself. But you're not done because that answer should have created several new questions for you. Add them and repeat. Download this mind mapping program if you must.
As testers, ours is not only a journey of questioning things, but in finding out the answers to those questions. If someone plops down a huge system for you to test, how are you going to tackle it? The question is always antecedent to the approach. So if you don't know, find out and track the way(s) in which you found out. Like watering holes in the desert, there will be places you perennially come back to for answers. Where are they? You won't find them unless you go looking for them.
Brush up on your Google-Fu
Learn to ask questions in different ways, including how to ask them in a Google query. Remove extraneous words. Group the related terms together to eliminate false positive results. Reduce the results to a certain domain, a certain file type, within a certain period of time. This ability can greatly improve the signal-to-noise ratio.
Don't brush up on your Google-Fu
Send that query out without all the constraints mentioned above. Misspell a word. Ungroup terms. Expand the time frame. Select a result from page 10, not page 1. Select the LAST result. You will be surprised by how much you uncover in the journey. Bob Ross called them happy little accidents. I cannot tell you how many discoveries I made on the way to answer a completely DIFFERENT question. In fact, most of the downloads on this page are result of another train of thought.
Now, you've downloaded Microsoft C# 2008 Express and written your program. Go ahead and set up a new, small project for yourself and see it through to completion. Use TDD,BDD, NUnit or MbUnit. (What do all those terms mean? You don't seriously expect me to answer it for you, do you?) You will have gained more knowledge of how to view the internals of the system. These internals are not only the constituent bits that comprise the application, but the human processes around their creation. Knowing these processes can guide your questions later on. No, you're not developer level but you are CONVERSANT. Being conversant means being able to have a conversation, and conversations lead to questions which lead to discoveries which leads to...
Have fun playing with the tools. Just don't start/stop there.
Saturday, June 6, 2009
Testing Programming Interview Questions...
I was cruising around the internets today and stumbled upon one of those sites that posts programming interview questions. You know the ones: archetypal algorithmic queries requesting the shortest way to move manhole covers from Mt. Fiji while weighing eight balls of 3 different colors (choose only two). Those kind of rudimentary questions.
I suppose the allure is that if one were able to successfully answer all the questions, they'd be a shoe in for the illustrious programming gig. Or perhaps to validate themselves; either they haven't been discovered (yet!) or are simply too brilliant to be burdened with the trivial job search (gimme a raise!). Only narcissists - insecure in their feeble abilities - answer these out-dated questions to puff up their sense of self-importance with a strapped-down, vein-tapped quick-fix of techno-trivia devoid of context.
So by the time I got to question number 6, I was feeling pretty good about myself. It read as follows:
You’ve got someone working for you for seven days and a gold bar to pay them. The gold bar is segmented into seven connected pieces. You must give them a piece of gold at the end of every day. If you are only allowed to make two breaks in the gold bar, how do you pay your worker?
***SPOILER ALERT***
I pondered for a bit, acquiesced to my plebeian logic, and moved ahead to the answer. However, when I read the answer I realized that it required that the worker be willing to trade in yesterday's pay for today's pay. Ummm???!!!!(Note I cannot spell the word I want to type here, so I'll simply link to a wav sound of it in use). I would never have arrived at that conclusion. There appeared to be a great many assumptions about what actions could/could not be taken.
And that is why I'm writing this. I'm not so much fascinated by the answers as I am the meta-questions. What questions would you ask before you would feel satisfied?
Some of mine include:
-Can anyone else cut the gold instead of/in addition to me?
--If Yes: Are they subject to the same restriction as me?
---Do we both get 2 cuts each?
---Must I pay extra for their services?
---Can we renegotiate the terms if the worker is the cutter?
-With what are we cutting the gold?
--What if the cut cannot be made in a 24-hour period?
What questions would you ask?
I suppose the allure is that if one were able to successfully answer all the questions, they'd be a shoe in for the illustrious programming gig. Or perhaps to validate themselves; either they haven't been discovered (yet!) or are simply too brilliant to be burdened with the trivial job search (gimme a raise!). Only narcissists - insecure in their feeble abilities - answer these out-dated questions to puff up their sense of self-importance with a strapped-down, vein-tapped quick-fix of techno-trivia devoid of context.
So by the time I got to question number 6, I was feeling pretty good about myself. It read as follows:
You’ve got someone working for you for seven days and a gold bar to pay them. The gold bar is segmented into seven connected pieces. You must give them a piece of gold at the end of every day. If you are only allowed to make two breaks in the gold bar, how do you pay your worker?
***SPOILER ALERT***
I pondered for a bit, acquiesced to my plebeian logic, and moved ahead to the answer. However, when I read the answer I realized that it required that the worker be willing to trade in yesterday's pay for today's pay. Ummm???!!!!(Note I cannot spell the word I want to type here, so I'll simply link to a wav sound of it in use). I would never have arrived at that conclusion. There appeared to be a great many assumptions about what actions could/could not be taken.
And that is why I'm writing this. I'm not so much fascinated by the answers as I am the meta-questions. What questions would you ask before you would feel satisfied?
Some of mine include:
-Can anyone else cut the gold instead of/in addition to me?
--If Yes: Are they subject to the same restriction as me?
---Do we both get 2 cuts each?
---Must I pay extra for their services?
---Can we renegotiate the terms if the worker is the cutter?
-With what are we cutting the gold?
--What if the cut cannot be made in a 24-hour period?
What questions would you ask?
Subscribe to:
Posts (Atom)
Posts
