Testing the Internet of Things: The Human Experience of a Watch, a Chip and the Boston Marathon

Mobile and embedded devices, more than any other technology, are an integral part of our lives and have the potential to become a part of us. This generation of mobile and embedded devices interacts with us, not just awaits our keystrokes. They operate in response to our voice, our touch, and the motion of our bodies.

Since all of these devices actually function with us, testing how the human experiences these devices becomes imperative. If we do not test the human interaction, our assessments and judgments of quality will be lacking some of the most important information needed to determine whether or not the device is ready to ship.

“Human experience” testing, or lack thereof, can lead to redesign of software, and sometimes, of the device itself. So what is testing the “human experience”? Although initially, usability comes to mind, human experience testing goes much deeper. Usability testing focuses the ways in which users accomplish tasks through the application under test.

Then the question becomes just how does “human experience” testing differ from usability testing? The answer lies in the scope, depth and approach.

“Human experience” testing focuses on the actual interaction. It involves not only the look and feel and ease of use, but also our emotional, physical and sensory reactions, our biases and our mindsets. It involves testing in the “real world” of the user; when, where and how the user and the device will function together.

Why is “human experience” testing so important to mobile and embedded devices?
Because when a mobile device is physically attached to us and works with us and through us, the more important the results of the interaction or collaboration becomes to us emotionally and physically. .

In conclusion, I’ll share a very personal example. It is a tale of two mobile devices attached to one woman, a marathon runner.

Join me on the starting line of the 115th running of the Boston Marathon, April 18th 2011. I’m standing in my corral, excitedly anticipating the sound of the starting gun. Last year, I surprised myself by qualifying for Boston, only 10% of runners do, and I’m hoping for another qualifying run.
I have pinned on my bib carefully keeping it flat as it contains the chip that will record my race for the Boston Athletic Association. The chip will record my time as I run over mats at various miles in the race. My current time, location on the course and my anticipated finish time will appear on the BAA website and will be texted to my friend’s and family’s smartphones so they can track my progress during the run.

I click on my Garmin watch and anxiously await it’s catching the satellite to start the GPS. It’s ready and the gun goes off. I’m careful to click the start button at the exact moment I step over the starting line to ensure a correct timing. As I run along during the early miles, I check my watch for the pace, to validate that I’m running the speed I’ll need to qualify. As I push myself up Heartbreak Hill at mile 20, I check my heart rate monitor for feedback confirming that I can continue to run my current pace or that I can continue at all. It reassures me that as exhausted as I feel, I’m doing fine.

As I look at the elapsed time on my watch, I confirm that I’m on pace to reach my goal of another qualifying run. As I turn left on Boylston and the finish line is in sight, look at my watch to see that, not only a qualifying run, but also personal record, is in reach! I dig in and give it everything I have left. As I cross the finish line, physically totally spent but emotionally charged, I click my watch off and I see it… My qualifying time and my personal record! The feeling of accomplishment and elation is beyond description!

Now I’m in the car, riding home, just basking in my own glory. My cell phone rings and a friend asks my gently what happened. I hear concern in his voice and wonder why as I tell him about the best run of my entire life. And then he tells me, “Your run isn’t on the BAA website”. My elation immediately turns to grief. The chip, the timing device embedded in my bib, had failed to track my run. The only record of my qualifying run and my personal record is held within my watch. At that moment my watch becomes a part of me. As one runner once said, “the pain is temporary, but the time lasts forever”. And now my Garmin holds the only record of my accomplishment. What if it didn’t save?

Immediately upon arriving home, I go directly to my laptop and download my watch. My heart is literally in my mouth as I wait for the time to come up on the screen, documenting my time forever. And there it is, 3:51:58! My qualifying run and personal record are mine forever. And I will be on the starting line in Hopkinton for the 116th running of the Boston Marathon next year due to the collaboration among my body, my mind my emotions and my watch.

The lesson is that devices that interact intimately with the user require a different type of testing than other types of embedded systems. The closer a device is to the human user, the more it requires human testing; it requires testing the interaction between the device and users’ actions, senses and emotions.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s