tag:blogger.com,1999:blog-16039353470194443862024-02-20T00:57:47.105-08:00ITestStuff.caPassionate software tester turned leader.
All rants are my own and may not reflect those who employ me.Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-1603935347019444386.post-29528353966899354422018-09-11T17:27:00.002-07:002018-09-11T17:27:38.518-07:00Tips for programmers who want to build testing skills<span style="font-family: Arial, Helvetica, sans-serif;">I previously wrote a post sharing tips for <a href="https://www.iteststuff.ca/2018/06/5-steps-for-testers-who-want-be-more.html" target="_blank">testers who wanted to get more comfortable with code</a>. In response, Twitter user </span><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://twitter.com/ssmusoke">@ssmusoke</a> asked me, </span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">"What advice would you give for getting <a href="https://twitter.com/hashtag/developers?src=hash">#developers</a> <a href="https://twitter.com/hashtag/TestInfected?src=hash">#TestInfected</a> and grow their <a href="https://twitter.com/hashtag/QA?src=hash">#QA</a> chops …" </span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">So here we are. I'm going to share my thoughts on how developers can grow their QA chops, to use Stephen's words.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">If you're fortunate enough to be working with an Agile</span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: x-small;">™</span><span style="font-family: Arial, Helvetica, sans-serif;"> team, then you've already got an advantage because if you're <i>doing it right</i>, your team will be made up of specializing generalists whose collective goal is to complete all the stories committed to within each sprint. This sets the stage nicely for any team member to feel empowered or even encouraged to perform tasks beyond their defined specialty - in this case, testing even though they're a programmer. So there's a little motivation/reasoning.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Let me start with what I believe is the most important piece of advice I can give on this topic: <b>Programmer's can still use their primary skill set, coding, in the pursuit of assisting with testing.</b> Sure, to catch up on a QA backlog we can teach testing skills and set programmers loose on exploratory testing of features and fixes. But that would be a lot like handing engineers buckets and telling them to bail out the sinking boat, instead of fixing the broken bilge pumps.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<b><span style="font-family: Arial, Helvetica, sans-serif;">1. Please, please, </span><span style="font-family: Arial, Helvetica, sans-serif;">(and one more for importance) </span><span style="font-family: Arial, Helvetica, sans-serif;">PLEASE write unit tests.</span></b><br />
<b><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></b>
<span style="font-family: Arial, Helvetica, sans-serif;">If you're not already doing this then I'm not convinced you're looking to grow your QA chops. This is easily the quickest win for getting developers to help with testing. Now, it's not just that easy. I've seen many instances where unit tests get added for the sake of having unit tests without much thought being put into what they test. Test the happy path, then test the negative cases, and any edge cases you can identify that seem valuable. Then grab a tester, show them the scenarios you've got and see if they have any more suggestions about what to test. Ok, now you've got good, useful unit test coverage. But that is likely not enough on it's own.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>2. Pair with a tester, find out what cases they would test, and discuss how much of that can be tested in code. Now, do it</b>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">If a tester says "Well I'd do x,y,z and then check that the expected value was written to the database"...you've got a candidate for a database test, or an integration test. The tests you come up with don't have to cover the *exact* steps that the tester would take. The goal here is to answer the same questions they were seeking to answer in their described testing scenarios.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Maybe you don't have the infrastructure in place to create and execute these kinds of tests. That happens a lot. In that case, you can discuss whether now is the right time to add them or not (make a value assessment around what info you'd be getting from the tests). If it's not the right time, log a tech debt ticket or a story and work with your product and engineering managers to get them prioritized appropriately.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>3. Pull branches when doing others' code reviews, and exercise the code locally.</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Here's where we diverge from just testing in code.<a href="https://techbeacon.com/qa-necessary-or-should-developers-do-their-own-testing" target="_blank"> A Tech Beacon article</a> on the subject states: </span><br />
<blockquote class="tr_bq">
<span style="background-color: white; color: #454545;"><span style="font-family: Arial, Helvetica, sans-serif;">"Pure code-based testing fails because it lacks the human factor. Humans do interesting things to applications in ways that are often surprising. [Testers] enhance the success of coded tests by providing a human eye, and a human element, to help anticipate that."</span></span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Want to help get a jump start on early testing coverage? Well those code reviews you're doing (<i>you ARE doing code reviews...aren't you?</i>) is a great time to execute some high level exploratory testing. Don't just read the code changes in your friendly neighborhood diff viewer or code review tool, but actually pull the code and dive in through your IDE. Experience the code the way you would if you were the one writing it. Then click that run button and actually give the feature a test drive armed with your knowledge of the code change. The more you discuss with testers about how they'd explore through features during their testing, the more efficient you will be at doing this on your own at code review time...likely catching important bugs early in the process.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>4. Start by having debriefs done. Then move to participating in debriefs.</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I covered this a little bit above. If you can't pair with a tester when you're starting out to help you learn to think the way testers think, then at least do debriefs with them. After you've done some testing, grab a tester and walk them through what you did, <i>and why</i>. This can foster a healthy discussion around what kinds of workflows need to, or do not need to, be exercised. For instance, if you can prove edge cases were covered in other automated tests, then great! No need to run them manually. Perhaps the tester will point out some additional things they'd have tried, and you can go back and do it... and in the process, sharpen your ability to identify those types of scenarios in future.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>5. Seat time</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In past, my spare time has involved racing cars. At many driving schools and speaking with many experienced drivers, they'd almost unanimously agree on the single most important thing to improve one's driving skills.</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">"Seat time."</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">What they meant was that the single best thing you can do is practice. Now the caveat to this is to have advisers, teachers, mentors or whatever you want to call them ensure you're practicing the right behaviors and not bad ones. The same is true for testing. With enough practice and the right guidance, you can teach your brain to think the way a tester does. This is why I keep encouraging "talk to your testers!" Ask questions, discuss, and listen with an open mind to how they think, and why they do the things they do.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>For those REALLY interested</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There are other ways if you truly want to keep growing your QA chops. For instance, a couple times a year, our local testing meetup <a href="http://kwsqa.org/" target="_blank">KWSQA</a> hosts <a href="https://kwsqa.org/testing-games/" target="_blank">Testing Games</a> nights. Anyone is invited. At each event, a company offers up their app to be put through its paces. Attendees are then encouraged to test and find bugs in the app in any way they feel is appropriate. Bugs get logged, and often there are prizes for most creative test, biggest security flaw, and other interesting things that attendees find. This is a great way to practice testing in a fun and open environment against an application not in your usual domain.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Good luck, and happy testing!</span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-1103288789307175032018-06-14T21:21:00.000-07:002018-06-14T21:21:04.540-07:00I created the SDET role at my company and I wish that I hadn't...<span style="font-family: Arial, Helvetica, sans-serif;">Hindsight is 20/20, as "they" say.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">"They" also say you have to make mistakes to learn from them. I say, learn from MY mistake so you don't have to make this one. Because some mistakes are really hard to recover from.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I was hired to develop automation frameworks and had been given</span><span style="font-family: Arial, Helvetica, sans-serif;"> the title "Software Engineer". To be honest, I didn't have the confidence to believe I deserved it. I even appended my own qualifier to the title in my online profiles, calling myself "Software Engineer (in Test)".</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Fast forward a few months and I've been asked to lead a team of people doing the same job, developing and maintaining automation frameworks. So I set to work developing role descriptions and career growth plans - for the position I'd like to give people on my team: <i>Software Development Engineer in Test (SDET)</i>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I had used this title for automation related positions I helped create in the past, as well as having the title of Quality Assurance Developer earlier in my career. So when it came time to decide how to title the team members of my new automation focused team, I didn't put much thought to it. I completely glossed over the benefit I was being extended when I myself was given a Software Engineer title. Ooops.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In retrospect, after many conversations with industry professionals I respect, I realize that development is development regardless of what your focus area is. By tacking on a qualifier such as "in test", it opens up the discussion that these team members are not as skilled or deserving as team members with a "real" developer title. How often do we hear about developers being "good enough" to be in an SDET role? Or how about when development teams pass on junior applicants but recommend them for the automation role? And then you have to engage in discussions like where the SDET role fits in the pay scale of the organization (hint: in my experience, it's almost always been below developer, but above QA).</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Simon Stewart made the claim (in a talk of his I attended at STPCon Spring 2018...and I hope he doesn't mind me attributing this to him) that finding good Automation Engineers (or SDETs, or SETIs, or Test Engineers, etc...) is actually very hard. Anyone who's been a hiring manager for this role will more than likely agree. And if this role is truly a hard one to fill, would that not warrant that position being worth more? Simon's claim was that there's plenty of developers in the candidate pool looking for positions, but we have to actively go out and hunt for months to find good SDETs. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So, back to my 20/20 hindsight. I've been reflecting on this a lot lately and I've come to realize that my new opinion on the matter is that anyone developing in code should have a developer role and be treated like a developer. That is, held to the same standards, paid the same wages, etc. The bonus to this? The people focused on your test frameworks can float between automation development and feature development as required. You're finding the people with the test and automation skills and putting them on the development team. This doesn't mean other developers can't contribute to automation, and vice versa. It's about getting right skills in the door and making them available to assist the wider team.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I never gave much focus to titles in past. But take it from me: take the time and think carefully about it. Once in place, they can be quite hard to change.</span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-72843778391157715532018-06-14T12:12:00.001-07:002018-06-14T12:12:21.832-07:005 steps for testers who want be more comfortable with code<span style="font-family: "arial" , "helvetica" , sans-serif;">In a <a href="https://www.iteststuff.ca/2018/05/discussing-testers-value-beyond-echo.html" target="_blank">previous post</a>, I made mention of some tips I have for testers who lack confidence working with raw code. Here they are, in the order I think most easily executed. Of course, they could be done in any order, you can pick and choose which you even want to do, or you can do them in parallel. The idea is to just work towards being more comfortable talking about and writing code.</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Look at code reviews.</b></span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Seriously. Just look at them. This is step one. You don't need to be added as an assignee or a reviewer, but start looking at them. When you do:</span><br />
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Look at what files change - how do they relate back to this particular feature/bug fix?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Look at what comments other reviewers leave - what kinds of things are they focusing on?</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Try and understand what the code change is doing - read through and start to identify what kinds of functions live in the different files.</span> </li>
</ul>
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Set up your laptop with a development environment.</b></span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">You are part of the development team, after all - aren't you? So if your on-boarding as a tester didn't include setting up your environment the way a developer's is, find the docs and do it yourself. Or ask for it. Either way, I encourage that you get set up to pull code and run local builds. Not only does this give you the opportunity to see the code and spend time in the environment that the developers do, but it also allows you to be able to pull branches, run local builds, and get early eyes on features and code changes.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Use the terminal at every opportunity.</b></span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Just to get used to writing commands, and understanding what the terminal can do for you. There's so much power available to you. Once you learn some of the shortcuts, I promise you'll wish you had explored the environment sooner. This will also help you set a foundation for the skills needed to write bash scripts. Once you have that foundation, you can start to...</span><br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Experiment with writing scripts.</b></span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">There are many opportunities for scripts to help you do tasks related to your job much more quickly and efficiently. Once you understand how to do things like cURL from a terminal window, you can start to script things like API calls for testing and validation. Consider any task you spend time doing more than once a candidate for a script. Once you've written a few of these, you'll be on your way to building a library of tools that help you complete menial and tedious actions with minimal effort - freeing you up to spend more time on the most impactful work.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Pair with developers.</b></span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">Get face time with developers whenever you can. In my experience, they often more than willing to let you sit with them, shadow them, and work through problems together. By doing this, you'll see how they tackle problems and if they're really helpful, they'll talk through their process of building the feature with you. You can hear where they start, what kinds of things need to be added, modified or removed, any many other insights into the code base. You can also watch for and discuss coding conventions. Every organizations have their own conventions and standards and you'll want to be familiar with yours so as you learn you ensure that you're learning habits that align with the company. This sets you up to have your first few attempts at code be well received by your friendly neighborhood developers.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><i>Feel free to comment below if you have any other useful tips for testers who may be nervous about participating in the code side of software engineering.</i></span></div>
<div>
<blockquote class="tr_bq">
</blockquote>
</div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-47685508069795030932018-05-24T23:18:00.000-07:002018-06-02T15:19:06.955-07:00Discussing Tester's Value, Beyond the Echo Chamber<span style="font-family: "arial" , "helvetica" , sans-serif;">I've finally come to a conclusion. I spend a lot of time thinking about why I feel like a lot of the advice to testers feels common sense, and why I don't feel like my thoughts are novel or unique. Here's the problem: the testing community is in some senses an echo chamber. The same thoughts get shared within the community, and people hear, agree and repeat. What I've personally neglected to do is reach <b>outside </b>the tight testing community to listen, learn and share. What do people in other engineering roles think the value of testers is? Or maybe they think there's no value, and I should seek to understand why. Sharing my thoughts on the value of the tester role and engaging in discussions when opposition is raised is only going to serve to improve bring clarity to my points.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here's a couple things I've noticed in discussions with people in non-testing roles who think testing isn't valuable:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">"Testers don't understand code. They rarely have technical understanding of how software works."</span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Admittedly, this one pisses me off more than most. Understanding how software works and understanding code don't go hand in hand. And being able to understand what a snippet of code is doing doesn't necessarily involve knowing how to code in that specific language either. Having said that, I've seen many organizations add fuel to this fire by hiring large numbers into testing roles who truly do not understand software. Like at all. We're talking applications-as-a-black-box, tell-me-what-to-click-to-test mentality. Check the box, move on. I've got plenty of tips and tricks I'll share with you (in person, over social media, or <a href="https://www.iteststuff.ca/2018/06/tips-for-testers-who-cant-code.html" target="_blank">in a later blog post</a>) if you are a software tester who is afraid of code, or doesn't know how to grow beyond the role of a pure blackbox tester. Having a blackbox hat on your rack is a great skill...but don't let it be your only skill.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">"Testing is slow. We should automate it all so it's fast and we won't need human testers."</span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Well we're actually partially agreeing here. We should automate regression coverage so that devs can be sure their feature continues to work as expected going forward, without relying on testers having to revisit the feature and all valuable scenarios after every change. That would be slow, tedious, and not a good use of anyone's time. But if we can automate all that away, it's a <i>good</i> use of skilled testers time to explore the feature - exercise workflows as a user would, look for edge conditions and risky areas, think about the <i>feel </i>of the feature, consider performance, accessibility, security....</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Good testers don't want to do the boring, repetitive shit. So let's not make them.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">"Testers are only needed to execute testing steps. Therefore they're a cheap safety net for organizations."</span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Have I ever mentioned how much I hate seeing testers referred to as "cheap"? I read a response to a LinkedIn post the other day, and someone said: they don't want their developers responsible for doing any testing because "why pay a skilled developer $120/hr to test their feature when a tester could do it for less than half that"...or something to that effect. Wow. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This is akin to the whole "anyone can test" or "testing isn't a real skill" types of arguments. You also hear a lot of people repeating things like "this is why companies like Google or Facebook don't have testers". (Newsflash by the way...they DO have testers, despite publishing all kinds of things saying they don't. Is it just cool to say you're hip and modern and don't have testers?)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I think this argument is perpetuated by people that have only come into contact with bad testers, or been at organizations that foster poor testing practices. Testing requires skills in functional decomposition, risk assessment, specific types of communication and technical writing. But we as testers need to prove that by fighting back against bad testing.</span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-40544287563367863292018-05-09T08:57:00.003-07:002018-05-09T08:57:42.903-07:00The value testers bring...<div class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">It started with a simple question that I posed in the Ministry of Test #general slack channel:</span></div>
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Hi all. Does anyone have any sort of resources to point to for “proving the value of testers”? I’ll try to explain: I’m looking for resources that would help when having a conversation trying to explain that testers do more than just sit in a seat and execute test steps. Along with that, I’d like to try and justify reasonable pay for testers. I know how I <i>feel</i> about it all, but having something to point to beyond my own opinion would likely go a long way.</span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Well, fortunately for me, Mr. Michael Bolton (someone who's lessons I've followed closely throughout my career as a tester) was watching. He pushed back on me to provide my thoughts on what value testers add. I typed what came to my head first, with no censoring or editing:</span><br />
<blockquote class="tr_bq">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Personally I believe testers’ value comes at many stages of development, which is why I’ve always been a proponent of them being involved in every stage. Customer advocates, connectors (of people and information), informants (informing the org of their observations on quality, risk, …), analysts (analyzing change, risk areas, issues from the field…and distilling that data to inform changes in process/development in hopes of improving efficiency of development with quality)…and <strike>likely</strike> definitely more</span></blockquote>
<span style="font-family: "arial" , "helvetica" , sans-serif;">At this point, Michael offered to role play with me some of the arguments I could expect to encounter when typically having this discussion. I'll paste the majority of the conversation here. It's a long read but those who have read it have expressed interest in the content. I'll summarize with a TL;DR for now, and then likely provide updates on my follow up thoughts soon.</span><br />
<br />
<blockquote>
<blockquote>
<b><i>michaelabolton</i></b></blockquote>
<blockquote>
(In character:) Don't you think the product owner is a customer advocate? Isn't the business analyst a customer advocate? Heck, aren't the programmers customer advocates?</blockquote>
<blockquote>
I don't see programmers say "screw the customer", except as a joke sometimes. Why do we need testers?</blockquote>
<blockquote>
</blockquote>
<blockquote>
(Be prepared; these are perfectly legitimate questions. AND there are good answers for them.)</blockquote>
</blockquote>
<br />
<blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Sure. And I believe programmers often do advocate for the customer and weigh in on discussions to that end. But programmer's primary skill sets lie in finding solutions to specifically defined problems, and implementing those solutions. We could take some of their time away to investigate how user's use the software and features, and analyze the types of bugs coming in from the field…but that's a lot less time they'd be spending in their primary skillset.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Does that seem valid?</blockquote>
</blockquote>
<br />
<blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
That's pretty good, but not as strong as it could be.</blockquote>
</blockquote>
<br />
<blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
I won't argue that. This is my first attempt at writing my thoughts on the matter beyond having them in my mind or rambling them verbally to an ally</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(In character) The programmers have a lot of knowledge about the product already. AND they know how to code. And the testers don't, by and large.</blockquote>
<blockquote>
</blockquote>
<blockquote>
(C) So the programmers will find most of the important problems anyway.</blockquote>
<blockquote>
</blockquote>
<blockquote>
(C) The programmer's primary skill set is building a quality product. Who would know more about how to do that than they would?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
The programmers have a lot of knowledge about their specific domain of the product. In our case, programmers are focused on the windows app, the iOS app, the Android app, the Web app…the APIs, the worker services…all specifically. Testers are actually one of the only roles in the engineering org that understand the customer's usage and interaction with the apps from end to end. </blockquote>
<blockquote>
A programmer mindset is typically one that is able to prove there aren't problems with the solution <i>the way they decided to implement and use it</i>.</blockquote>
<blockquote>
</blockquote>
<blockquote>
I don't know how to say this not anecdotally, but if I received demos of features from programmers where we didn't encounter bugs during the demos, I'd be more inclined to believe they're good at finding the important problems.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(NOT in character) Demos are funny like that. They're intended to be demonstrations, but they turn out to be experiments. :slightly_smiling_face:</blockquote>
<blockquote>
</blockquote>
<blockquote>
But here's the thing: it's important to acknowledge that developers are really good at finding most of the problems. <i>Almost</i> all of them. What kinds don't they find, and why not?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Speaking generally, it seems as though developers find important problems in the primary, happy path workflow. But typically not edge cases, or at integration points (especially when integrating with a feature outside of their primary focus). I don't think most developers are mindful in a way that allows them to understand how the user might use their feature beyond the way they <i>expect</i> a user to use it.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(C) So... we'll get product managers and BAs to test that stuff. </blockquote>
<blockquote>
(C) And after we've done that, we'll ship the product and if there are problems, we'll fix them right away?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Product managers don't have the same curious mindset that testers bring, and thus don't know how to experiment and investigate in risk areas of the product. In my experience, while testers may not always know how to write code, they do often know how to analyze it and determine technical risk areas. So we could have PMs and BAs try to test the primary workflows…but again that's time that they're not spending thinking about future roadmap features and products, or working to understand customer needs better.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(C) Why not just tell them to do that?</blockquote>
<blockquote>
</blockquote>
<blockquote>
(C) Who says that testers are the only curious people?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
As for the ship and fix problems…we do roll out most of our features in a way that allow us to catch issues before the entire customer base encounters them. However, for the issues to be found to be fixed, <i>someone</i> must encounter them. Hard to ask for money for a product where you expect the users to be your line of QA.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Who are the people teaching PMs how to experiment and assess risk areas/find issues at the integration points? Surely that's the job of a tester, right? Informing about those areas and providing that information back to people who can consume that information?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<i><b>michaelabolton</b></i> </blockquote>
<blockquote>
(C) Hey, it's not the job of a tester to tell the PMs how to do their jobs!</blockquote>
<blockquote>
</blockquote>
<blockquote>
(C) As for the release-to-production strategy: not much can go wrong with our product. We're really good at this stuff. We've been doing it for years.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
I mean…we have data to prove we're not really good at this stuff.</blockquote>
<blockquote>
</blockquote>
<blockquote>
We can ask any one role to go do all of the things. We wouldn't need PMs if we just told developers to go figure out what the problems our customers are having that need to be solved. But I can't imagine a lot of actual development would get done. In the same vein, while PMs can go do that, asking them to also do the work of a tester is time spent that the PM isn't building roadmaps and deciding product direction to keep making the company money.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(NC) Very good.</blockquote>
<blockquote>
</blockquote>
<blockquote>
(C) We don't have roles here. Everybody does everything. Why not just get rid of roles?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
(NC) In <b>this</b> case, we are so much the other direction of having specific developers from different stacks, focus areas, etc…and product having specific focus areas…I'm not concerned about this argument. But will address anyways since its good to know how to.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
:slightly_smiling_face:</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
(C) Again it comes back to mindsets and skillsets. We can either have specialists do what they're best at to build the strongest product possible, or we can have teams of people doing a mediocre job at everything, make a mediocre product, have a mediocre customer base…and make a mediocre amount of money…</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(C) All right. So what is it that testers do that's <i>special</i>? <i>Distinct</i>? </blockquote>
<blockquote>
(C) And why can't people just switch from one mindset to the other? We've got really smart people here.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
testers will experiment and test (both in manual and automated fashions) to gather and analyze data, distill that data and provide direct recommendations on issues to fix, not just in the product, but in the processes of development. This will allow us to ship quality products to customers, and do so with even more speed and efficiency in the future.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(C) I still don't see why the programmers and the PMs can't do that.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Hmm…I think this is where I could use some Michael Bolton wisdom and insight.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
(NC) OK. Not in character unless explicitly tagged that way. (edited)</blockquote>
<blockquote>
</blockquote>
<blockquote>
The answer is that they <i>can</i> do that.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Anyone can switch roles and mindsets to some degree. Anyone can learn to program. Anyone can learn how to use tools.</blockquote>
<blockquote>
</blockquote>
<blockquote>
The issue, it seems to me, is this: <i>it's hard</i>. And it's probably slow, too.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Shallow work is easy to do. Deep work is harder.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Going through the motions and rituals is easy. Skilled, expert, focused work is harder.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Awesome, that's what I was trying to reach for mentally…but couldn't articulate the way you are</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Recognizing many of the problems in something you wrote just now is easy. Recognizing other problems takes time, or distance, or both.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
It's much more about providing T shaped value to the teams than about just doing a bit of everything with little focus and expertise</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
That whole T-shaped business sounded tired to me on the first day.</blockquote>
<blockquote>
I'm not sure why.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
I guess since the org is talking about building "T-shaped" developers anyway…its common language I can use</blockquote>
<blockquote>
Even if its not ideal language in general</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Yes. It feels kind of naff to me, but it is what it is. Nonetheless: it's reasonable to believe that everyone should have some degree of general skills outside their speciality.</blockquote>
<blockquote>
</blockquote>
<blockquote>
The testers' speciality, it seems, is this: living and working at close <i>social</i> distance, but farther <i>critical</i> distance than other people on the team.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
We often have programmers "test" each other's work by pulling branches in code review and doing shallow testing. Yet bugs still get out, so obviously there's evidence there that having a non-expert do a shallow test doesn't solve the quality problem.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Exactly.</blockquote>
<blockquote>
</blockquote>
<blockquote>
It helps to have someone at some distance from the work to evaluate it IF you need serious evaluation.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Right</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Rather than having someone adopt that mindset, it's a powerful heuristic to have someone <i>inhabiting</i> that mindset.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
So…to the people who say "engineers" deserve more money because their coding skills are hard, measurable skills…how is the monetary value proven in the tester's skills where you can't look at code output and "grade" it directly?</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Coding skills are not hard and measurable... <i>without testing</i>.</blockquote>
<blockquote>
</blockquote>
<blockquote>
(They're not even measurable <i>with</i> testing, but testing can tell us something about the quality of the product that non-empirical approaches can't.) </blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
interesting</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
You can't <i>measure</i> quality. Measurement <i>might</i> inform <i>some aspects</i> or <i>some attributes</i> of quality.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>Graeme</i></b> </blockquote>
<blockquote>
Yes, that I am aware of and believe.</blockquote>
<blockquote>
<br /></blockquote>
<blockquote>
<b><i>michaelabolton</i></b> </blockquote>
<blockquote>
Quality is not about measurement. It's about assessment and evaluation.</blockquote>
<blockquote>
</blockquote>
<blockquote>
Literally, about <i>how we value</i> something; that's the value in "evaluation".</blockquote>
<blockquote>
</blockquote>
<blockquote>
And that takes us right to the tester's role.</blockquote>
<blockquote>
</blockquote>
<blockquote>
The tester's role is to focus on <i>threats</i> to the value of the product.</blockquote>
<blockquote>
</blockquote>
<blockquote>
The tester's role is to focus on <i>investigating</i> the product to <i>discover</i> threats to its value.</blockquote>
<blockquote>
</blockquote>
<blockquote>
The tester's role is to bring <i>expertise</i> and <i>focus</i> to that task.</blockquote>
</blockquote>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">For me, the TL;DR of this conversation boils down to two (or potentially combined to one) tweet-sized quotes from Michael.</span><br />
<blockquote>
The testers' speciality, it seems, is this: living and working at close <i>social</i> distance, but farther <i>critical</i> distance than other people on the team. </blockquote>
<blockquote>
The tester's role is to focus on <i>threats</i> to the value of the product.</blockquote>
<br />
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-14283618060155594342018-05-08T22:33:00.000-07:002018-06-03T23:08:46.787-07:00It's been a minute...and a lot has changed<span style="font-family: "arial" , "helvetica" , sans-serif;">Well, it's been a minute since I blogged here. To be more exact, it's been almost 3 years.</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In those 3 years, a lot has changed. I've grown in my career, from Test Specialist to Test Strategist to SDET to Engineering Manager. I'll touch briefly on what I spent my time doing in each of these roles.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Test Specialist</span></h4>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">My time as a Test Specialist was as one on a team of 4 as part of an engineering team of less than 20 people. These were times of a rapidly growing team (and company!) who was trying to find their way into more modern agile development practices with shorter release cycles and more automated processes. The team was just ramping up on things like CI, unit testing, and more automated tooling and scripting.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">After learning the product, I helped the team move away from documented test scripts and test cases to a more context-driven model. We worked to shorten time needed for testing by performing tests in more isolated builds and environments. I also had a large hand in standing up the first automated UI tests for the product. As the team continued to grow and we broke development down into smaller feature focused agile teams, I began to recognize a need for the testing across teams to share good patterns, lessons learned and work towards and larger goal of a quality product offering, not just individual quality products. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This is when I proposed the role of Test Strategist...</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Test Strategist</span></h4>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I spent the next 2.5 years at the company in this position. While I continue to be an individual contributor, I was also tasked with attempting to bring some order and consistency to the company's testing practices across the teams. During this time, I wore many hats where I performed manual functional testing for multiple teams, wrote automation scripts, designed and built automation frameworks and tools, acted as a release manager, and "unofficially" product managed the STS team's work (Software Tooling and Support - essentially our version of an automation and release operations team). I also mentored junior testers, provided coaching and direction on good unit testing practices, advised on quality metrics provided to Execs, and much, much more. In this time, the company grew from 2 feature focused agile teams to 8+ agile development teams - most of which having embedded functional testers, and all of which employing CI processes and incorporating some level of automated testing. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">6 months ago I left that role to move across the continent from Waterloo, Ontario to San Francisco, California...</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">SDET (Software Development Engineer in Test)</span></h4>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">My official title here was "Software Engineer", but I was asked to focus first on building/improving the Windows UI Automation framework (since I had experience in such things - my last company's main offerings being primarily Windows Desktop based products). I quickly went to work not only implementing this, but also working to reorganize the way QA was performed and structured at the company (essentially putting my Strategist hat back on, in a way). My biggest regret in this time is creating the SDET role officially at the company. I now believe the correct approach was to just continue calling the role Software Engineer, but giving them automation specific tasks to work on (more on this on another blog post - OMG I have content worth sharing again...after my 3 year funk! *squeals with glee*).</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A little context - our company is a cloud based offering, which allows customers to access their projects via web browser, iOS app, Android app or Windows Desktop clients. The means we have teams building features in 4 platforms, plus APIs, workers, services, databases and all the other fun stuff that comes along with large cloud-based offerings. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">My proposal was to move QA back under one "umbrella", with a QA Manager at the top. "Why?" you might ask. With most companies moving to embedding testers into agile, or "no specific role", feature based teams - the "QA Team" model seems obsolete. Well, a year ago I would have agreed with you. However, as it turns out, there are some issues with that. In our case, we had a consistency problem. Features implemented across multiple platforms were developed and tested in isolation from each other, essentially causing the platforms to feel like different products. On top of this, testers often lack direction and career growth when not exposed to a manager that actually knows how to manage them and help the grow in QA. The outcome of this are things like stagnant testing practices, and often a very high attrition rate - either losing testers to other companies, or to other roles within the company (this is not inherently a bad thing, as people should be able to move to various roles but ideally this isn't happening because people feel a lack of growth in their current role after a relatively short period of time).</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Oh, by the way, testers are on the QA team as far as reporting and feeling like they belog to a team, but they are each given a feature set to own, and are embedded in the respective feature teams to be the testing lead on that team. There is no old school, traditional dev - test wall that work gets thrown over. We still follow the model of testers being involved as early as design and planning of features, through development, deployment and monitoring in production together with the rest of the development team that owns that feature.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Part of this move to a QA team was the establishment of SDETs, who build and maintain frameworks that the development teams can leverage for various levels of automated testing. They also help implement quality of life tools and other offering to help teams deliver products of higher quality at greater speed. That's always the goal, enable the teams to deliver faster without sacrificing quality. Arguably the SDETs could go either way with respect to reporting to the agile teams' engineering managers or the QA manager. We chose the latter. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">It was decided there should be a manager with strong automation knowledge for the SDET team...and thus my seemingly quick move to...</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Engineering Manager (Platform Automation Team)</span></h4>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I now currently manage 2 SDETs plus a part-time contractor (long story, but it's working out wonderfully for us - hmmm, I smell even more blog content), and have positions open for at least two more SDETs. Besides the team management responsibilities, I still contribute to the building and maintenance of automation frameworks and CI tooling (though sometimes I look at the open instance of VS Code on my monitor, realize that it has been days since I typed anything into it and cry a little inside), as well as advise the QA team in a strategist-like capacity on process and mentorship, and unofficially "product manage" the growing automation backlog. </span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Because I get asked this a lot, I'll touch on this here for anyone interested in following up - here are the tech stacks we're currently using to automate our UI/E2E testing:</span></div>
<div>
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">XCUITest for iOS</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Espresso for Android</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">An in-house wrapper around FlaUi Automation Library for Windows (WPF)</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Cypress.io for Web</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">...with more to come as the team grows.</span></li>
</ul>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
</div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Wow it's been a crazy 3 years! And the adventure still feels like it is just beginning. Feel free to reach out to me with any questions, thoughts or challenges you have for me in regards to any of this. I'm always happy to share my thought process, experiences and to learn more from others' experiences as well.</span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com2tag:blogger.com,1999:blog-1603935347019444386.post-48544215740039626742015-06-07T19:02:00.002-07:002015-06-09T10:31:40.585-07:00Please, stop telling testers how to test!<div class="separator" style="clear: both; text-align: center;">
<a href="https://ibiologystephen.files.wordpress.com/2014/05/avoid_injury_ict_job_ibiologystephen.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://ibiologystephen.files.wordpress.com/2014/05/avoid_injury_ict_job_ibiologystephen.png" width="253" /></a></div>
<div style="text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Let me start with a scenario.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">A programmer works away on a new feature or a bug fix. They finally feel like they're finished and it's time to move the user story over to a tester. They update the user story with an explanation of the work they did, any assumptions they made and one final thing: </span></div>
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">"Testing notes: Try exercising feature X by using entry point A as well as entry point B and assure that the output is the same."</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">Now let me explain my problem with this. I have a gut-wrench reaction when something like this occurs, because my first thought is, "Wait! If these scenarios have been identified as important, why didn't you try them?"</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">That's right. I'm proposing that if a programmer can identify a risk area, wouldn't it make sense that they ensure their code works under those conditions? Even better - shouldn't there be coded tests for these scenarios, if at all possible?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I don't mean to lay blame here. As far as I'm concerned, in development there is no "us" vs "them". "We" are a team. So if testing is playing out like this, it is up to everyone to correct it. But, I AM advocating that it gets corrected. </span><span style="font-family: Arial, Helvetica, sans-serif;">Allowing testers to be told how to test is basically boiling down their job to executing steps that others don't feel like doing. It is removing what I believe is the core expertise of testers - to think critically and creatively about how features have been implemented, how they integrate, and how to stress the shit out of them!</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So please, stop telling testers <i>how </i>to test - and start collaborating with testers on <i>what </i>to test! In an Agile environment, we have the luxury as programmers and testers to sit together and chat about a feature or a bug fix <i>any time we need to</i>. Theres a few times when these talks can be mutually beneficial and help us build the best products as quick as possible.</span><br />
<br />
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><b>Before programming begins: </b>This is an excellent time to discuss what the problem being tackled is (new feature or bug fix), and how both parties see it being solved. The programmer can explain how they are going to approach the implementation of the solution and the tester can talk about how they plan to test it. This gives the programmer the chance to keep those scenarios in mind during development. It also gives the tester the chance the develop a test plan based on the intended implementation.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><b>Demoing before testing:</b> When the programmer believes they're done, a brief demo to the tester can be extremely helpful. (In fact, you wouldn't believe how many times we've caught a glaring bug during these demos...BEFORE any formal testing has begun). This is an opportunity for the tester to ask about integration points, and think about how to exercise those.<br />The other thing that could be discussed at this time...UNIT TESTS! Talk about what unit tests have been written and if any more coverage would be beneficial. If certain things aren't unit testable, the programmer can explain why that is the case and the tester can plan to focus on that area (since we know there's less coverage there than other areas).</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><b>During development:</b> And of course, in true Agile fashion, any point in time is actually a great time to talk about roadblocks that arise, and possible solutions! Keep the communication up and leverage eachothers' skills early and often! </span></li>
</ol>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Tell me what to test. </span><span style="font-family: Arial, Helvetica, sans-serif;">Tell me where to test. But please, don't tell me how to test!</span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-21552577629800698092015-01-14T19:57:00.000-08:002015-01-15T07:22:31.784-08:00Looking beyond the titles (and an explanation of my QA CAN'T CODE movement)<span style="font-family: Arial, Helvetica, sans-serif;">In every facet of my life, I'm a self-proclaimed "jack of all trades, master of none". </span><span style="font-family: Arial, Helvetica, sans-serif;">And being a jack of all trades often goes hand in hand with being a fast learner. </span><span style="font-family: Arial, Helvetica, sans-serif;">While there are many people out there who identify their skills and abilities, and hone them to be a master of their craft, I am not one of them. However, I have always believed that this has actually been a key factor of my success in software testing. I have had experiences in software and hardware, script-directed manual testing and self-directed exploratory testing, planning testing and executing testing, reading and yes, even writing code. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I have written at least some code in over 12 programming languages. Couple that with exposure to various frameworks at work, attending workshops and doing projects in my spare time and I'd say I have at least a basic level of understanding of some programming concepts. But I bet most of my coworkers don't know this about me. And why should they? - I am, by current title, a "Software Quality Assurance Specialist" and to most people that means I am a blackbox, manual software tester.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">That is not their fault. It's a labelling issue. It's a perspective issue. Dare I say, it is a stereotype issue. I almost feel as though the industry has done a disservice by adding modifiers and variations to the title. Quality Assurance Developer, Software Developer in Test, Test Engineer - what do all these titles have in common? They perpetuate the belief that "typical QA can't code". And herein, I believe, lies the problem. </span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">If I am a Software Quality Assurance Specialist*, then I believe my specialty is anything that can help contribute to the quality of the product. </span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">This spans a far wider range than functional testing of features and products. If you look upstream in the development process, this means advocating quality in design meetings and helping build testability into new features. If you look in the "middle" of the development process, this could mean <i><b>smart</b></i> functional testing of the product or tool development to ease the workload with scripts or other tools. To me, smart functional testing means testing complex scenarios and applying automation where automation is best suited. Looking further downstream, the responsibilities could include being involved in the support process to analyze escaped defects, close the feedback loop and iterating over the whole testing process for constant process improvement.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I'm not advocating that every member of the test team should have all of these skills - that would be a little too Agile Utopian. But if you're a tester, building up your skills toolbox never hurt. So it would be wise to keep in mind the full range of areas where testers can contribute, and seek to improve skills across those areas. Then work your ass off to apply the skills within your current team, so that the rest of the team is aware you possess these awesome skills! I'm sure it will net you new and exciting challenges, more respect from your team members and leaders, and a bump in product quality (which surely no one is going to complain about).</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">I once volunteered to write an in-house tool that plugged into another system being used, in a language I had zero experience with. It was desperately wanted, but no developer time could be spared on it. I whipped up the tool, learned a new language in the process and in the eyes of management, "saved the day". Suddenly, that "Quality Assurance Developer" title I had been asking for was granted, and I was being given more time to work on tools, automation initiatives and more!</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">Anyways, the point is that titles mean nothing - so don't let the lack of a "developer" title stop you from learning to code and finding ways to apply that. In a <a href="http://www.iteststuff.ca/2015/01/a-rant-qa-cant-code.html" target="_blank">previous post</a>, I revealed that I've chosen to use the phrase "QA CAN'T CODE" (ironically, of course) - because QA <i>can</i> and <i>should</i> code, among many other things.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;">*Despite being a Software Quality Assurance Specialist, I typically don't associate with the term "QA". I usually try to refer to myself as a Software Tester.</span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-20398796273906353502015-01-12T21:48:00.002-08:002015-01-14T12:12:18.782-08:00A rant - QA CAN'T CODE<div style="text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;">So...this is happening:</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigLasCrsQffW_3lVWOLoP7VVnCmcFoQCzGQuQkZJNi1cr2_oMY_n1piyc1WXmhC-gXsMnz9WgCdIbHGnsykf4zqbDmzQAyNzHxKVAEWovvS8xEHEpb_68JNbqZaTo8cG3OjEsJNGtZubqV/s1600/Screen+Shot+2015-01-13+at+8.44.53+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigLasCrsQffW_3lVWOLoP7VVnCmcFoQCzGQuQkZJNi1cr2_oMY_n1piyc1WXmhC-gXsMnz9WgCdIbHGnsykf4zqbDmzQAyNzHxKVAEWovvS8xEHEpb_68JNbqZaTo8cG3OjEsJNGtZubqV/s1600/Screen+Shot+2015-01-13+at+8.44.53+PM.png" height="292" width="320" /></a></div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjW-ErMAWKKpW8uqdkYUlgtm0E50nPkKqMLrmFTNnDi2X0JRa4bbsSHqZRbMGA-2KI_iITTHcpTj3QsNW2b3RKnQapi6-ifjN2d7VV8r7w89wFJc6jecj6tz_EVfG3rsJyhvGGz0UV8wGI/s1600/Screen+Shot+2015-01-13+at+8.45.13+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjW-ErMAWKKpW8uqdkYUlgtm0E50nPkKqMLrmFTNnDi2X0JRa4bbsSHqZRbMGA-2KI_iITTHcpTj3QsNW2b3RKnQapi6-ifjN2d7VV8r7w89wFJc6jecj6tz_EVfG3rsJyhvGGz0UV8wGI/s1600/Screen+Shot+2015-01-13+at+8.45.13+PM.png" height="292" width="320" /></a></div>
<div style="text-align: center;">
<br />
<br /></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">I've heard it too many times. So now I plan to wear it in irony. There's this notion that lingers and hasn't been dispelled yet - one that perpetuates the belief that "QA can't code". </span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">I'd like to send two messages with this simple statement.</span></div>
<div style="text-align: left;">
</div>
<ol>
<li><span style="font-family: Arial, Helvetica, sans-serif;">This is a push for a re-branding. I am not a fan of the job title QA (aka "Quality Assurance"). I've said it before - "I don't assure quality; I test software." Maybe quality inspectors can't infact write code, which is fine. But then I don't want to be a quality inspector.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">This is also meant to be an ironic statement to spark a discussion. Why can't QA code? I'm a QA Specialist by title..and I can write code. So why then do I often hear amongst the industry that writing code is reserved for developers? In the modern "Agile" era, where every member is responsible for every stage of the development process in one capacity or another, QA <b>can</b> code. Infact, QA <b>should</b> code!</span></li>
</ol>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">There's a larger and more well thought out blog post to follow on this topic. For now, I'm just going to mention that these shirts will be available for sale (at no profit to me) for anyone that agrees, and feels like making this bold statement with me.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">*EDIT*: Design has changed thanks to a suggestion from Jim Hazen!</span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-34307577399422384562014-11-06T15:06:00.001-08:002014-11-08T17:58:32.035-08:00Famous first words - the moments leading to defect discovery<span style="font-family: Arial, Helvetica, sans-serif;">Just a fun little thought of the day from today...</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There's a well-known phrase that people use - "Famous last words". Often it's in this context:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.kappit.com/img/pics/39940364cfgef.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.kappit.com/img/pics/39940364cfgef.png" height="318" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">I was thinking about my inner dialogue today during testing, and I realized that every moment just before finding a bug, I'd catch myself saying one of two phrases. I'm going to call them my <b>Famous First Words</b>, because they define the first moment when I know I'm on to a defect. I'm sure many testers can relate to these moments.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">"Hey! I wonder what happens when..."</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">I'd say this is the more common one. It usually happens during exploratory test sessions. I'll be working through testing the feature as normal, and out of nowhere this thought occurs. It's like being able to tell ahead of time that an area is just asking for there to be a bug. "I wonder what happens when I enter this value..." BOOM! - Exception occurs.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The other phrase happens a little more out of my control:</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">"Hey! That was weird..."</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">This one happens after catching a glimpse of some action (or lack of action) that occurs. It's when a dark corner of my brain lights up and says "I've seen this before" or "Did that just...?". This one is neat to me because it's those little details that the untrained tester may miss. When just a flick of uncertainty pops up for a brief second. This phrase has led me to find countless threading/asynchronous issues and things that are just subtle enough they weren't caught in your average, functional testing.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">These are just to two I seem to notice most commonly in my day-to-day testing activities. Are there any other internal phrases that people find are precursors to finding defects?</span><br />
<blockquote class="tr_bq">
</blockquote>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div style="text-align: center;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-25703077899495666632014-10-01T17:44:00.000-07:002014-10-02T08:18:50.404-07:00Using Blackbox & Whitebox Analysis Methods to Build Strong & Efficient New Feature Test Plans<span style="font-family: Arial, Helvetica, sans-serif;">My most recent position has me doing a significant amount of new feature testing, with the freedom to do it in an exploratory manner. This is great - a dream come true! So I see a testing task come through for the newest developed feature and the excitement begins. (EDIT - <i>I feel the need to clarify here. I work hard to ensure testers are included early in the design and development process, so I'm well aware of the feature before it comes into the test queue. My excitement here is that I get to begin physically testing the feature.</i>) But then the reality of the situation sets in: I'm responsible for testing this entire new feature and confirming that it works to a certain standard.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">While a new feature can be intimidating, with the right planning, nothing should be too much to handle. Without really realizing it, I have grown into a process for how to handle the decomposition of a new feature. It pulls from a few different concepts to a formula that seems to have worked for me so far.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">Now is a good time for this disclaimer: I'm a big fan of mind maps. Growing up, I had introductions to mind maps in a variety of classes and scenarios. I never really applied them to testing until meeting my (now) testing mentor. He's also a big fan of mind maps and the driving force for my continued use of them.</span></blockquote>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I think you know where this is going. Yes, I use mind maps for my test planning. This is not a new concept, and there's at least a couple good reasons why I like using them. They're lightweight. They're quick to whip up. They're easily modified. And most importantly, they're easy for anyone to understand. They serve as sort of a living document of the testing being performed. And once the testing is complete, they serve as documentation of what was considered during the initial testing phase of the feature.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">In a <a href="http://www.iteststuff.ca/2014/07/being-agile-tester-youre-going-to-need.html" target="_blank">previous blog post</a>, I refer to Agile testers needing to wear many hats. I employ two of those hats when planning for new feature testing. One of the blackbox tester followed by one of the whitebox tester. And I do so in that specific order.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Step 1: Blackbox Analysis</b></span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">First things first - I mind map the usage of the feature. </span><br />
<br />
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Starting with any provided documentation (assuming there is any requirements or specifications), nodes are created in the map based on actions I'm told I can perform. These are the expected/documented actions of the feature. </span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Then nodes are created based on my assumptions about what I think I should or should not be able to do. As far as I'm concerned, I'm playing the role of a user who has not read any documentation. So if I *think* I can or can't do something, that is a scenario worth testing. These are the assumed actions of the feature.</span></li>
</ul>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Step 2: Whitebox Analysis</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br />Now that I have a mind map that shows some semblance of what I'm going to do during my testing, I remove my blackbox testing hat and replace it with a white one. There are many ways to get insights into the feature code itself. I use a combination of looking at the code reviews for the code submission for that feature and speaking to the developer face-to-face. </span></div>
<div>
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Understanding what code paths exist allows for some nodes on the mind map to be de-prioritized. There may be a node for a negative path that we can clearly see (or have been told by the developer) that the code prevents. For me, I'd still like to try that scenario in the product to ensure it is handled well, but it requires less focus because it's a scenario the code was designed to handle.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">This may also reveal execution paths we didn't think about in our initial assessment. Maybe a requirement wasn't stated, but the developer decided it was implied and coded for it. Or maybe there was just a potential path that we missed during our blackbo</span><span style="font-family: Arial, Helvetica, sans-serif;">x assessment.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Look at which methods have been unit tested, and what the unit tests actually test for. If there's good unit test coverage, there's a good chance we don't need to test for things like basic inputs and outputs, because that has already been covered (our unit tests run before the code is merged into the code base...assuring us that the unit tested scenarios are good before the feature makes it into any product build).</span></li>
</ul>
<span style="font-family: Arial, Helvetica, sans-serif;"></span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span></div>
<span style="font-family: Arial, Helvetica, sans-serif;">
<b>TL;DR:</b><br /><br />The intention here is that the blackbox analysis of a new feature is performed before the whitebox analysis. By not having the code in front of us while we think about how to test a feature, it prevents tunnel vision which could cause us to not think creatively about how that feature might perform. The whitebox analysis allows us to hone that creative thinking down using facts. We can focus more on the areas that are risky, and have confidence that specific scenarios have been unit or integration tested.</span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com1tag:blogger.com,1999:blog-1603935347019444386.post-26960754723089400042014-08-13T15:56:00.001-07:002014-08-13T21:57:37.770-07:00Why "We've always done it that way" is not such a bad phrase after all<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-0MjSTiRFVky-NgKONkMDJ4wQT5HAFyfj6_w0_u2P3um_ny4NNxAPUJurJupwtuLRIoU7o38I0WGpw1CmUtd8Un7jMIRYAyTpavhHZ6Enx2gppLBXChdqZun09MrR-13ZpctLn-69MFjC/s1600/124139a0-114f-11e4-9c37-22000a98b2af-original.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-0MjSTiRFVky-NgKONkMDJ4wQT5HAFyfj6_w0_u2P3um_ny4NNxAPUJurJupwtuLRIoU7o38I0WGpw1CmUtd8Un7jMIRYAyTpavhHZ6Enx2gppLBXChdqZun09MrR-13ZpctLn-69MFjC/s1600/124139a0-114f-11e4-9c37-22000a98b2af-original.png" height="238" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">This isn't a new topic. I've heard many people before talk about how this phrase is so detrimental. I want to propose a different take on it, instead of the classic negative view.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">There's a running joke at my office about this phrase. In a sprint retrospective, as a relatively new member of the team I mentioned that I was sick of people answering my questions with "We've always done it that way". As a result, my coworkers now like to toss the phrase out at me in playful office jest.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">But all jesting aside, it did spark some very interesting discussion. I realized that I wasn't tired of hearing that phrase specifically, but <b>I was tired of hearing it used as a way to explain away a questionable practice without any responsibility or ownership</b>. As the newbie on the team and getting frustrated with the painfulness of process X, when I asked "why do we perform action X this way?", it was easy for a more experienced team member to reply with "we've always done it that way". This avoids having to explain the "why" portion of my question and halts all discussion about improvement. I don't think anyone's doing in intentionally, and that is what makes the phrase so detrimental.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">A year ago, I was listening to a talk by Adam Goucher (<a href="https://twitter.com/adamgoucher" target="_blank">@adamgoucher</a>) entitled "It's our job to collect stories" (Title may not be exact). One of his points was regarding this phrase. Adam pointed out that if "we've always done it that way", then at some point a decision was made to do it that way, and that decision must have been based on certain factors at the time. </span><span style="font-family: Arial, Helvetica, sans-serif;"> </span><span style="font-family: Arial, Helvetica, sans-serif;">Make sense?</span><span style="font-family: Arial, Helvetica, sans-serif;"> Furthermore, if we trust our team, then we should also trust that it was the RIGHT decision at the time. So perhaps </span><b style="font-family: Arial, Helvetica, sans-serif;">the phrase is actually an indicator of a previous decision that needs to be reassessed</b><span style="font-family: Arial, Helvetica, sans-serif;">. We should be able to justify WHY a particular choice was made. I believe this blanket statement is so popular because it allows us to skip the justification altogether, thus not requiring us to think about the reasons behind the initial choice. I liken it to the classic scenario of a child asking an adult "why" repeatedly. Once we run out of the ability to provide a reasonable explanation, we revert to "because". But children often don't accept "because" as a response. They continue to prod. </span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Consider the following scenario:</span></div>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Child: Why do we wait until 8:00pm to eat dinner every day?</span></blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Father: Because that's the time our family has always eaten dinner.</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">It's easy to imagine that the father thinks this answer should suffice for the child. But what if the child is getting hungry at 6pm every day? This answer would probably frustrate him. </span>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Here's the magic part that transforms this phrase from detrimental to something that can be used to focus improvement.</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Same scenario, but the child responds to his father's answer:</span>
<br />
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Child: Why do we wait until 8:00pm to eat dinner every day?</span> </blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Father: Because that's the time our family has always eaten dinner.</span> </blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Child: Well, why so late?</span> </blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Father: Because my dad didn't used to get home from work until after 7pm.</span> </blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Child: But you get home from work at 5pm, and I'm usually hungry by 6pm.</span> </blockquote>
<blockquote class="tr_bq" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Father: You're right - I never thought about that before. Let's make it 6pm then.</span> </blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">It's a silly example, but it shows the point clear enough. By pushing to get to the reason why the family eats so late, they were able to recognize that they could change dinner time with no negative effects, while improving the scenario for the kid (he's not starving for hours each evening).</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>By pushing back and refusing to accept a blanket justification, we can dig down to the underlying reasons a decision was made.</b> Perhaps a decision was made 2 years ago based on a time crunch, but now we have time to address tech debt and this particular feature would be a prime candidate. In fact, <b>any time the question "why do we do X?" gets asked, it should be a flag to investigate further.</b> You may find there's a reason that is still valid, in which case, no harm to have asked. But I'm guessing that fairly often you will find a decision was reached based on factors that have now changed.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Sometimes it just takes that one stubborn person to point it out. </span><span style="font-family: Arial, Helvetica, sans-serif;">So the next time you're asking "why" and you get that dreaded response: push back. Encourage everyone involved to dig deeper, and find out the real reasons. It has worked for us and begun to help foster discussions of iteration and improvement on things we've been doing for a long time.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Just because it has always been done that way doesn't prove that it's the best way anymore. </b></span><br />
<blockquote class="tr_bq" style="clear: both; text-align: left;">
</blockquote>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-43054948261155471862014-07-20T11:23:00.004-07:002014-07-20T18:39:08.369-07:00Being an Agile Tester - You're going to need a bigger hat rack<div style="text-align: center;">
<a href="https://yy1.staticflickr.com/8514/8370875254_ba508feac6_z.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://yy1.staticflickr.com/8514/8370875254_ba508feac6_z.jpg" width="265" /></a></div>
<div style="text-align: center;">
<br /></div>
<span style="font-family: Arial, Helvetica, sans-serif;">Software testers have traditionally worn multiple hats. We are frequently asked to switch context quickly, and adapt accordingly to get each and every job done.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">With the rise of Agile development, testers need to be prepared to perform a variety of testing activities - both traditional activities and some new, potentially unfamiliar ones. The general expectation is that these activities will be performed in less time, with even less structured requirements. If you're already a context-driven tester, this won't come as too much of a shock to you. If you come from a more traditional testing background, this could be a pretty sizeable change for you to grasp.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In the Agile world, all roles and contributors within the team are considered equal. That is, all are equally required for the team to be successful - there is no "rockstar". And from time-to-time, each and every member of the team WILL be called upon to perform actions outside of their standard responsibilities. I believe for some people, this is a scary concept. A classical developer is as likely to respond with "I'm expected to test?" as a classical tester is with "I'll have to develop things?". My answer to any of you with these questions is "yes". And I don't want you to simply grasp this concept - I want you to embrace it.</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">If you want to be a successful tester on an Agile team: you're going to need a bigger hat rack.</span> </blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">
There, I said it. Now let me explain why.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Throughout my early years in testing, I understood that as I grew, I would learn about more hats to include in my theoretical "hat rack" of testing skills. I began honing my <b>functional, blackbox testing skills</b>. I would follow test plans and test steps, keeping a keen eye out for things that were unexpected. Then I developed skills in <b>performing the test planning</b> - learning how to functionally decompose a feature and figure out how to test it against its expected behaviour (as provided by the dev team via the project manager). As I gained more credibility within my test team, I was able to work closer to the dev team and hone some <b>whitebox testing skills</b>, where I could see what the code was doing and test accordingly. This all happened throughout the course of a few years working with the same testing team.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Eventually I switched companies and joined an Agile test team. Within the first year, on any given work day, I could be expected to do any of the following:</span><br />
<br />
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Work with a developer and/or designer to talk about how a feature could potentially be tested when it was finished development</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Functionally test a new feature</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Accessibility test a new feature</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Usability test a new feature</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Performance test a new feature</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Security test a new feature</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Perform full system regression testing for all of the above</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Gather statistics from internal and external sources and analyze data (for the purposes of improving testing focus)</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Write code to support the test automation framework</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Write automated test scripts</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Develop test tools & scripts to assist testers wherever possible</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Mentor team members (testers and non-testers) in improved testing techniques</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Contribute to overall test strategies for Agile teams</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">And so much more...</span></li>
</ul>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Of course, not everyone has to do all of these things. But the opportunities for a purely functional, blackbox tester are diminishing. In Agile, they're virtually non-existent. As I said:</span></div>
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">You're going to need a bigger hat rack.</span></blockquote>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">We see Agile testing often go hand-in-hand with context driven testing. In context driven, you are the <b>test planner AND the test executor</b>. You are given a handful of requirements and expected to determine if they are met. You are also expected to <b>advocate for the customer</b>. Question when things don't <b>feel</b> right. And you have to do this all with limited time, using the most effective method. Is one exploratory pass good enough? Should this be automated and added to the regression suite? HOW should this be automated? Where does this fall within our testing priorities?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Hopefully you are prepared for this. It's a new and exciting world, where testers are being handed more responsibility than ever before. Testing is definitely not becoming obsolete - <i>but bad testing is</i>. And with that comes the need to constantly learn new things, continuously improve and find new and creative ways to contribute to the Agile team and keep quality improving. Just as programmers must continue to learn and keep up with the latest technologies and languages, testers must continue to learn new practices, new ways of thinking, and keep collecting those hats.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com0tag:blogger.com,1999:blog-1603935347019444386.post-13405563963062837492014-06-10T19:25:00.002-07:002014-07-20T10:40:27.349-07:00Student Testing Resumes<span style="font-family: Arial, Helvetica, sans-serif;">A friend and myself were discussing the pains of screening hundreds of resumes for Software QA co-op applications. </span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">If you're not familiar with the co-op program, students basically do classes for a term, then work for a term, on and off for a 5 year degree program. There's often lots of pressure to get good jobs (or jobs at all), and the competition is fierce. Students often apply for countless jobs and go through many interviews to land a job.</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Often as employers we have to whittle a list of 100+ resumes down to an achievable number to pursue for interview. If we're lucky, that's around 15 interviews. Reading that many resumes and attempting to make a good selection can be tedious.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So today, the idea was jokingly tossed around that we should have a script that does a text search of all the applications (such as a recursive grep search) for the word "test". Then we realized the word "test" exclusively would be too limiting, so we figured a search of the pattern "test*" with a wildcard behind it would be better (ignore the syntax, I know it's not correct). This would result in words like "testing", "tested", "tester", etc. We also realized there were some words that would sneak through the pattern, such as "contest". This called for a regex to ensure there were no letters before the "t", such as " test*" (again, ignoring the syntax please).</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">This experiment resulted in a *rough* number of approx 17% of applications resulting in a hit. Remember, an application can include resume AND cover letter. Let's allow that to sink in...</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<b><span style="font-family: Arial, Helvetica, sans-serif; font-size: large;">...17%!?!?</span></b><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><span style="font-size: large;"><br /></span></b>
This is absolutely ridiculous. I understand the time pressures of applying to lots of job postings, and I understand lots of people who want to eventually get a developer role often use testing as a stepping stone into dev. But come on! You're applying for a testing job - you should probably have something about testing on your resume. I guess the whittling down of applications to a few interview requests isn't so hard after all.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Let this be a lesson:</b> <i>To be considered for a testing position, your application should probably make mention of testing.</i></span>Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com1tag:blogger.com,1999:blog-1603935347019444386.post-7918822511178127072014-05-13T17:08:00.000-07:002014-05-28T20:38:19.111-07:00Reflecting on my Software Testing World Cup Experience<span style="font-family: Arial, Helvetica, sans-serif;"><b>Leading up the competition:</b></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">When I first heard about the Software Testing World Cup, I was extremely excited. What a concept! Having a challenge for people to showcase their skills as testers? - It's brilliant! Developers have development competitions, so I'm glad the testing community can embrace similar concepts.</span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">We quickly assembled our team (<a href="https://twitter.com/_eddegraaf" target="_blank">@_eddegraaf</a>, <a href="https://twitter.com/drewbrend" target="_blank">@drewbrend</a> and <a href="https://twitter.com/josh_assad" target="_blank">@josh_assad</a>. Unfortunately Josh ended up having a prior commitment and couldn't partake with us). We had a short initial meeting to talk about how we would develop our strategy before the competition start date. We developed a mind map of the things we needed to take into account, as well as a tracking strategy for all the information we'd need to track during the competition (besides bugs, which we knew we would be tracking in the provided HP software).</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<span style="font-family: Arial, Helvetica, sans-serif;">Unfortunately, once the date was announced I came to the realization that I would be away for the two weeks prior for vacation and a speaking slot at STPCon New Orleans. The week after my return was shaping up to be a busy one, with STWC being on the Friday at the tail end of that. If I'm honest, I had considered backing out because I was disappointed with the amount of pre-planning we'd be able to accomplish. I felt unprepared and this upset me because I had grand ideals built up in my mind about our execution of test strategies during the STWC. </span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">Then it struck me: in the field of software testing, we're often hit with unexpected events. How often are we thrown into a project last minute and asked to sign-off on a project or feature? What about when we switch Agile teams and have to quickly ramp up to the new team's processes? Part of being a good tester means tackling challenges as they come to us.</span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">With this in mind, I decided this was a challenge I needed to see through because it was a real-life representation of true software testing.</span><br />
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">With the day of the competition came another unexpected roadblock - none of us were able to co-locate for the competition. Enter: my first experience with Google Hangouts. Wow! If anyone has to work remotely, and needs a convenient solution for video chat & desktop sharing with more than one person, USE GOOGLE HANGOUTS! It worked unbelievably well. </span><span style="font-family: Arial, Helvetica, sans-serif;">I was also fortunate enough to have multiple machines at home (though both were OSX machines). This was super useful for me, as I used one for watching the YouTube live stream to listen for important info and ask questions as we needed, as well as for entering data into our Google shared docs/HP Agile Manager. I used the other for the software under test.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Our strategy:</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">We used shared Google docs to track the issues we found before entering them into HP Agile Manager. This allowed us to very quickly see the items the other team members had found, as well as what platforms they had been tried on. We also had a Google doc to track the "areas" of the tool that needed to be tested (broken down as we observed them - ie. Installer, individual features, mobile integration, etc). This allowed us to see what areas each member was working on so we didn't hammer at the same areas. It also allowed us to structure our Exploratory Test sessions into reasonable sizes.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">With about an hour left in the competition, one of our members began porting the list of issues over to a document for the beginnings of our Test Report. We also took that opportunity to organize them by priority (based on end-user impact) and charting the results. In the final half hour, we collectively decided on what info to include in the executive summary and we made a ship/no-ship decision on the product (by platform, since we had different versions on PC and OSX).</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Things to improve on next time:</b></span></div>
<div>
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">We inaccurately assumed the software under test would be a web application. We prepared by assembling lots of tools to use for performance/load testing, debugging proxies, accessibility, etc. In future, we should assemble a similar list, but be prepared for both online and offline applications.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Try and co-locate for the duration of the testing. Having access to whiteboards and a central network would have been far ideal to using the online solution.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Be more prepared for multiple platforms. We got lucky having both PCs and Macs but we ended up only having Android devices to test the mobile integration. We should have had a better way of tracking the testing performed and issues found on each platform.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Build a template of a test report ahead of time. We knew what types of info we wanted to include in the report, but we didn't actually have a document framework to plug the data into. This would have saved valuable time wasted on basic formatting.</span></li>
</ul>
</div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Final thoughts:</b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">As I stated above, I'm so happy that this competition took place, regardless of our final standings (EDIT: We ended up placing within the Top 10, and won special recognition for "Most Useful Test Report"). It was a really good learning experience which I feel will only further my skills as a tester, especially in the area of test strategy and design. I strongly encourage everyone to participate if the opportunity ever presents itself again.</span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div>
<span style="font-family: Arial, Helvetica, sans-serif;">Lastly, a huge thank you to the organizers, sponsors, judges and other participants. I know lots of people have put quite a lot of time into this event and I hope in future I can pay it back and volunteer my own time and experience to something. As always, engage me on Twitter (<a href="https://twitter.com/graemeRharvey" target="_blank">@graemeRharvey</a>) or email me (<a href="mailto:graeme@iteststuff.ca">graeme@iteststuff.ca</a>) to chat about all things testing.</span></div>
Anonymoushttp://www.blogger.com/profile/14794545778543666562noreply@blogger.com4