Is Testing a Profession?

Testing is a real profession, but it’s undergone significant upheaval and change in recent years.  Don’t expect that to change in the future.  If you like rapid change and a fresh challenge with every project, testing can be a challenging and exciting career.

Just as programming offers a wide range of career choices, so does testing.  You can do anything from traditional UI testing to working with medical devices or even wearable devices.  Depending on your skills, inclinations, and yes, job opportunities, there are many different directions to take your career in testing.  Whether or not it can become a true career depends on your ability to reinvent yourself on a regular basis.

The profession of testing, as well as its knowledge and skills requirements, has changed substantially in the last decade.  First, it has diversified.  As testers, we understand the language of requirements, test plans, test scripts, and defects.  We create automation frameworks and run performance tests.  Yet, the testing world has become increasingly more complex and fluid.

The testing profession offers so many new areas of specialization.  Big data,  security,  IoT,  wearables,  mobile and embedded as well as user experience, customer experience and even machine learning and artificial intelligence are just a few of the possibilities.  And there are certifications in many of these areas.

Testers don’t necessarily have to choose a specialization; in fact, because computing changes so rapidly, a single specialization may be too limiting to give you the breadth of experiences you need over a long career. However, there are some trends in technology and methodology which all testers much embrace in order to build a career in testing.

First, are the agile methodologies, which have largely become the mainstream of development methodologies.  In agile teams, the testers take on multiple roles, the most important of which is championing quality and helping the team to see quality as the responsibility of every team member.  Testers also play a critical role in test-driven development (TDD) where the tests are created first and code is developed to pass the test. Testers also have the opportunity to pair with developers in testing as the code is being developed.

DevOps is another trend that testers can’t ignore and must contribute.  DevOps principles are based on breaking down barriers between development, testing, preproduction, and operations, in order to get software into users’ hands as quickly as possible, and to find faults and fix them in real time.  In DevOps, testers’ primary responsibility is ensuring the quality process.

Work on your communications skills

Communication and collaboration skills have never been more important.  Testers are often the ones delivering the bad news, and have to do so in a way that doesn’t lay blame or responsibility.  Further, they typically work as part of a larger team, and have to establish respect while advocating sometimes-unpopular positions.

Although testing is still a viable profession, there can be disadvantages to a testing career.  While most organizations need developers and IT people, some methodologies have been trending away from formal testing as practiced in many waterfall-based methodologies.  In particular, agile and DevOps are focused primarily on automated build and delivery chains, so automation skills are a requirement.  Wearables, IoT, and other devices may require knowledge of electronics and hardware-in-the-loop testing.

Testing in general is no longer the gateway through which an application has to pass in order to be successively deployed.  Companies have become far more willing to deploy software that may fail, as long as it does so fast.  Yes, users may face obstacles, but companies figure (often correctly) that users will help them tune their applications for more widespread use.

The upshot is that testing has become much more ambiguous than it has been in the past.  To effectively pursue a career in testing, it is critical to develop skills related to defining and executing tests earlier and faster, and to be more attuned to the ambiguity of software releases and feedback.

Testers must manage their own careers by getting the experience and training they need to thrive and grow in the testing profession.  Creative approaches for obtaining some of these experiences include volunteering for other projects that may be using innovative new techniques, or by participating in, or even organizing, corporate learning activities such as workshops and lunch and learn functions. Testers will have to invest in their own careers, perhaps by taking agile or DevOps courses and certifications, or by working on independent open source projects.

Finally, to create a career in testing it is important to recognize testing as a profession.  Get involved in the testing community through conferences, online groups and meetups.  These are not only great learning opportunities, but they offer testers a chance to mentor each other and to build recognition of testing as a profession.

Advertisement

Understanding How Assistive Technologies Make Products Accessible

 

Hi Testers, welcome to the brave new world of assistive technology.  Assistive technology is a term that refers to all types of technological devices that enhance the quality of life and improve independent function for people living with disabilities.  Assistive technology is available for people with visual, hearing, mobility disabilities as well as cognitive impairments.  Assistive technologies range from low-tech devices that assist with daily living activities such as eating or showering to high-tech readers for the blind and listening devices for the deaf. 

How does this fit into the category of software or device testing?  In many cases, testers are responsible for accessibility testing of new products, including web applications, personal fitness devices, and hardware/software products for finance, transportation, and other areas.  Knowing how the assistive devices work, and understanding how they are tested, is an essential part of understanding the requirements of accessibility.

My friend Max introduced me to his world of assistive technologies when I visited him recently.  He is legally blind and has an impressive array of assistive technologies in home office.  His devices range from lighted magnifying glasses to the Optelec Reader and Zoomtext.  The Optelec reader will not only magnify pages of magazines but also scan and read them aloud.  Zoomtext is a software program that will enlarge, enhance and read aloud everything on a computer screen.  Using Zoomtext, Max is currently writing a book.  

Max is an avid reader and when the Library of Congress digitized books, Max was a beta tester for the various reading devices.  Max uses his phone to dial into the National Federation for the Blind’s Newsline to keep up with current events.  He has access to thousands of newspapers and he can listen to articles of his choice.  I was so impressed with how much these technologies improved Max’s quality of life.

After my assistive technologies demo, as a tester, of course I became interested in how and who would test these devices and software programs.  My initial thoughts centered on accessibility testers who can apply their knowledge of specialized accessibility test techniques that they use to determine levels of usability by people with disabilities.  Accessibility testing is critical for websites and software programs, yet actually testing assistive devices requires something more.  More than any other type of device or software, assistive devices and software must be designed and tested based on the needs of the user.

Since usability testing is so critical for assistive technology, I realized that human experience testing so as applicable here as it is to wearables.  I recently developed a framework for human experience testing that I’ve presented at several testing conferences.  Human experience testing goes beyond usability in scope, depth and approach.  The closer the device becomes to the human, the more important “Human” Testing becomes. 

The Human Experience testing framework uses personas and user value stories to test the interaction between the person and the device.  Personas are detailed descriptions of the archetypical users who represent the needs and motivations of the user group.  They represent the motivations, values, expectations and goals for their interaction with the device.  User Value stories describe the ways in which users interact with the device.  They are based on how the users go about their daily lives.  Since people with disabilities depend on assistive technologies in the daily lives, human experience testing is critical.

The use of personas in assistive technology design is happening today. The Assistive Technology Industry Association (ATIA) is currently working with Jeff Higginbotham, PhD,  professor at the University of Buffalo on promoting persona-based design for assistive technologies and Microsoft is pioneering the concept.  So it follows that testing should involve the human experience.

I believe that testing assistive technology requires not only special test techniques, but also, special testers.   The initial testing of the prototypes and human experience testing can be done by accessibility testers; however, the final user experience testing should be done by those for whom the device is designed; those who will actually use the device in their daily lives.

Testing assistive technology is not only challenging and fascinating, but also, it is rewarding on many levels.  As my friend Max told when he introduced me to his assistive technologies, “Assistive technology makes life a little bit easier.”

The Brave New World Of Big Data Testing

As testers, we often have a love-hate relationship with data.  Processing data is our applications’ main reason for being and without data we cannot test.  Yet, data is often the root cause of testing issues; we don’t always have the data we need, which causes blocked test cases and defects get returned as “data issues”.

Data has grown exponentially over the last few years and continues to grow.  We began testing with megabytes and gigabytes and now terabytes and petabytes have joined the data landscape.  Data is now the elephant in the room, and where is it leading us?  Testers, welcome to the brave new world of Big Data!

What is Big Data?

Big Data has lots of definitions; it is a term often used to define both volume and process.  Sometimes, the term Big Data is used to refer to the approaches and tools used for processing large amounts of data.  Wikipedia defines it as “an all-encompassing term for any collection of data sets so large and complex that it becomes difficult to process using on-hand data management tools or traditional data processing applications.”  Gartner defines big data as “high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation.”  Big data usually refers to at least five petabytes (5,000,000,000 megabytes).  Sometimes the term Big Data is used to refer to the approaches and tools used for processing large amounts of data.

However, Big Data is more than just size. It’s most significant aspects are the four “v’s”.  Big data obviously has huge volume, the sheer amount of data, however, it has velocity, the speed at which new data is generated and transported, variety which refers to the many types of data, and veracity, its accuracy and quality.

Testers, can you see some, make that many, test scenarios here?  Yes, big data means big testing. In addition to ensuring data quality, we need to make sure that our applications can effectively process this much data. However, before we can plan our big testing, we need to learn more about the brave new world of big data.

Big Data is usually unstructured which means that it is does not have a defined data model. It does not fit neatly into organized columns and rows.  Although much of the unstructured big data comes from the social media such as Facebook posts, tweets, it can also take audio and visual forms.  These include phone calls, instant messages voice mails, pictures, videos, pdf’s, geospatial data and slide shares.  So it seems our big testing SUT (system under test) is actually a giant jelly fish!

Challenges of Big Data Testing

Testing Big Data is like testing a Jelly Fish; because of the sheer amount and its unstructured nature, the test process is difficult to define.  Automation is required and although there are many tools, they are complex and require technical skills for troubleshooting.   Performance testing is also exceedingly complex giving the velocity at which the data is processed.

Testing the Jelly Fish

At the highest level, the big data test approach involves both functional and non-functional  components.  Functional testing includes validating both the quality of the data itself and the processing of it.  Test scenarios in data quality include completeness, correctness, lack of duplication, etc.  Data processing can be done in three ways; interactive, real-time and batch; however, they all involve movement of data.   Therefore, all big data testing strategies are based on the extract, transform and load (ETL) process.  It begins by validating data quality coming from the source databases, then validating the transformation or process through which the data is structured and then validating the load into the data warehouse.

ETL testing has three phases.  The first phase is the data staging.  Data staging is validated by comparing the data coming from the source systems to the data in the staged location.  The next phase is the MapReduce validation or validation of the transformation of the data.  [I think you’re going to have to explain what MapReduce is here.  It’s basically the programming model for unstructured data; probably the best-known implementation is in Hadoop.]This testing ensures that the business rules used to aggregate and segregate the data are working properly.  The final ETL phase is the output validation phase where the output files from the MapReduce and are ready to be moved to the data warehouse.  In this stage, the data integrity and the transformation is complete and correct.  ETL testing, especially of the speed required for big data, require automation and luckily there are tools for each phase of the ETL process, the most well-known are Mongo, Cassandra, Hadoop and Hive.

Do You Want To Be A Big Data Tester?

Testers, if you have a technical background, especially in Java, big data testing may be for you.  You already have strong analytical skills and you will need to become proficient in Hadoop and other Big Data tools.  Big Data is a fast-growing technology and testers with this skill set are in demand.  Why not take the challenge, be brave and embrace the brave new world of big data testing!

Mobile Testing – How Much is Enough?

We all know that mobile apps can never be fully tested.  There are too many devices, too many OS versions, and too many different types of apps.  Testers continue to struggle with developing test plans and test cases the same way they did with traditional applications.  How do testers determine the correct scope of testing for an app to minimize quality and security risks for users and the organization?

First, look at organizational expectations.  Does your organization expect its software to work almost perfectly?  And does it fund and support development and testing to make that a realistic expectation?

If the answer to the first question is yes, but the second one is no, then the best you can do is set realistic expectations that may not be in line with what the organization wants.  If quality isn’t a high priority, then testing can be focused on high-priority areas only.

Second, is the app business-critical?  Does the organization depend on it to make money, service customers, or be more agile?  Or is it a marketing or public relations tool?  Does it provide valuable information to users, or is it simply nice to have?  If it is business-critical, then flaws could hurt the bottom line and quality become a higher priority.  And that’s true in all parts of the app, not just it’s operational aspects.  If users find poor quality in any part, they are unlikely to trust it to do business.

Third, consider the implications of a flaw to both use users and the organization.  A game or other entertainment app often has a high threshold of failures before most users think it’s not worth the effort.  Likewise a free app with limited extrinsic value.

But if the app performs an important function, or one that is counted on my users, a major flaw can have serious or even disastrous effects.  One example is the infamous iPhone time change bug in 2010, which failed to move from Daylight Savings Time on the appointed day.  Thousands of people were reportedly late for work or appointments as their alarm failed to go off on time.  It was an iOS flaw fixed by Apple a few weeks later.

Fourth, does the app use external services?  Many apps make use of other services within the enterprise, or commercial services for information such as weather or sports scores.  While testers don’t have to test the services, testers should read and understand what the service level agreements (SLAs) say about performance and capacity, and periodically test to make sure those are being fulfilled.

 

Now About Security

It goes without saying that security flaws are also quality flaws.  But many security flaws may have significant consequences to both user and organization.  If there is no data, or trivial data, to protect, security testing may not be important.  If it includes names, email addresses, or financial data, or any other identification information, that data needs to be protected at all cost.

In particular, testers have to know what data is being collected, and where data is stored.  On mobile devices, that can sometimes be a challenge, because it can be in internal storage or on a SIM, and it may not be readily apparent that data is being stored, and in any case often can’t easily be accessed.

In the era of hundreds of different devices and OS versions, as well as BYOD, it’s unrealistic to limit the kinds of devices that an app can be used on, even for internal users.  Testers have no control over the device or OS for testing or deployment purposes, and testers simply can’t test all combinations.

But teams likely have control over how and where the device stores data locally.  And they have control over how data is transferred to and from the device, and how it’s stored on the back end.  That’s what testers have to focus on.  Is any personal or identification data encrypted on the device, and is it encrypted during transmission?

Testing on mobile devices requires a combination of techniques, from traditional test cases to risk-based testing to device farm testing and if appropriate, crowdsourcing.  The test plan should be designed to use these techniques to test different aspects of the application.  Test cases, for example, can test traditional quality measures as well as security.  Device farms can be used for in-depth testing of popular devices.

Overall, test results should provide a clear picture of the quality of important aspects of the app given its purpose, and an overview of quality in general.

To what extent should testers and QA engineers be involved in software design?

Traditionally testers and QA engineers have had minimal involvement with software design. Design has been the role of the software architect, or team lead, for many years. Depending on the team, input from testers at this stage of the software development lifecycle isn’t always valued.

But in some circumstances that is changing. In particular, testers have a real contribution to make when one of the product goals is “design to test”. Architects who recognize that contributions can come from a variety of sources are soliciting testing feedback when creating an overall design.

And testers have even more design contributions in Agile projects, especially when using Test-Driven Development (TDD). Testers typically have a more complete picture of user needs, based on their in-depth understand of user stories and interactions with the Product Owner.

Because design is something that grows with the application in Agile, testers can always look at what the developers are doing. If the team starts letting the design get complex, or difficult to test, it’s time to have a talk with the developers about making the design more straightforward. It may require a hardening sprint or two, but it will keep the debt down.

For testers, here are some of the things you might consider as you share your expertise with architects and developers.

Do:
• Provide feedback on design for testability. You don’t want to accumulate testing debt.
• Get deeply involved in TDD projects. This is your area of expertise.
• Provide feedback on design decisions during an Agile project.

Don’t:
• Attempt to give advice outside of your area of expertise.
• Reject feedback on your design ideas. Everyone has something to contribute.

Join Me at QA&TEST 2013

I’m going to SPAIN!!!! I was just accepted as a speaker at QA&TEST 2013 in Bilbao, Spain on October 29, 30 and 31. I’m really excited to be part of this conference! The purpose of the conference is to showcase the latest quality assurance technological innovations and best practices; its aim is to give the attendees a lead in global competition. With the wide variety of tracks and impressive line-up of speakers, I’m sure the conference will meet and even exceed its goal.

This conference is for everyone including directors, project managers and all types of test professionals. There’s a great mix of technical and management tracks which means all the attendees will learn lots of valuable information. With technical tracks such as Test Automation, Testing Mobile Devices and Applications and Verification and Validation, and management tracks including QA Management and Test Team Organization and Testing, there is really something for everyone.

The speaker line-up is an exceptional mix of practitioners and thought leaders from many industries including banking, insurance, aeronautics and medical systems and academia. I’m really looking forward to seeing Carol Oliver’s presentation on using oracles in high volume testing. Carol is a Computer Science PhD student of Dr. Cem Kaner at the Florida Institute of Technology.

Bilbao is such an exciting location with the Guggenheim Museum and lots of scenic coastal areas and beaches nearby. It’s no wonder that the beginning of The World Is Not Enough was filmed here. Although I don’t think I’ll get to meet James Bond, I can’t wait to go. I hope to see you there.

The Android Diaries, Part 3

1/1/2013   It’s a new year and a brave new world among the androids.   As I start to play, otherwise known as exploratory test my new phone, I discover that it’s like dancing.  I must learn a whole new series of movements.  When I turn the phone on, it says “Swipe to Unlock”.   So what’s a “swipe”?  So I go on the web and locate the definition of swipe.  Here’s Merriam Webester’s definition:   Definition of SWIPE. 1: a strong sweeping blow <a swipe of a paw> 2: a sharp often critical remark <took a parting swipe at management>.   Ok, so now what do I do…attempt to hit it with my “paw” or make a nasty remark at it?    After a few attempts, I find that an android “swipe” is rather like a doing a waltz with one’s finger.  I would define it as a glide.   However it is defined, so be it.  Now that I’ve mastered the “swipe”,  it’s time to learn to tap.   Hmmm, I wonder if my old tap shoes are still in the attic?

The Android Diaries, Part 2

12/31/2012   Having thoroughly researched both the iPhone and Galaxy S iii, and learned that the only real difference is size; screen of the Galaxy S ii is larger.  Since my arms won’t grow any longer and I don’t always wear my glasses, I even more confidently return to the store to purchase my Galaxy S iii.  Now the most important choice…color.  Of course, I want a white one that must be brought in from a store 40 minutes away.  When it finally arrives, the geek must transfer most of my contacts from old phone manually since the old phone is so outdated.  Then he shows me how to download music and I’m off to join the android world.

The Android Diaries, Part 1

12/30/2012  Having made  my new year’s resolution to become more technical,  even though it is not yet the new year, I decide to start early with the purchase of my new smart phone.  As I confidently head out to the AT&T store, I’m contemplating Apple vs Android.  At the store, I listen in as the phone geek behind the counter shows an older couple how to use a smart phone and shows them differences between the iPhone 5 and the Galaxy S iii.   After hearing some of the pros and cons of both, the couple decides on an iPhone.  I ask if I can download tunes on the Galaxy and the answer is not from my iPod but there are 6000 free songs available online.  Can I download the books from my Nook?  The answer is yes.  And seeing the line of customers behind me become longer and more impatient, I leave the store and return home to research both smartphones on the internet.