Twitter
Categories

Entries in research (13)

Friday
Feb082013

Preparedness is Essential. 

 

As I sat working at my desk with a snow storm fast approaching, a thought popped into my head. What does it mean to be prepared? Many people are having the same thought right now with the snow starting to fall heavier by the hour.

As an engineer, I am tasked with the challenge of being prepared to bring any idea that crosses my desk closer to reality. I love this challenge, but there is a large difference between the “what do we do now” conversation vs a series of preemptive “what ifs” throughout a project. This is the essence of preparedness.

At Essential there is a strong connection between the work of our designers, engineers and researchers, enabling the “what if” conversation to occur frequently throughout a project’s lifespan, versus an over the wall handoff. In order to be able to create well designed products and be able to preserve design intent the whole way to market, every member of the team needs to be prepared for each “what if” that could occur to prevent a “what do we do now” moment.

In 1948, Mayor James Curly wrote a letter to MIT president Karl Compton, in regards to finding a way to clean up the record amount of snowfall that occurred that year (proposed in it was the use of a large number of flamethrowers). A true “what do we do now” moment. Though there was a good correspondence between the Mayor and the institution, there was no immediate solution besides the current application of salt to the roads (BostInno did a nice write up of this correspondence). It seems that perhaps this would have been a good conversation to have prior to the snowfall.

Don’t get me wrong, avoiding these moments in our lives is near impossible. Often times it takes a few “what do we do now” moments to frame up a “what if” conversation.  Some of these moments can be the catalysts of our biggest innovations. Conversely, characterizing as many “what ifs” as possible is a good way to be prepared for any “what do we do now” moment!

We can’t avoid the upcoming storm, and similarly sometimes in product development we arrive at a tough “what do we do now” moment. A good way to deal with these moments is to be as prepared as one can possibly be, which means cross collaboration and openness throughout the development cycle. I find that this is one aspect of what makes the Essential team tick.

Happy snow day everyone. 

 

Tuesday
Dec042012

Finding the Right Participants

I think it's safe to say that the most mundane part of my job is putting together a rock solid participant screener. Anticipating all the exciting research to be had once we've found the right participants is what pushes me through this process. In the past few weeks I've been in Screener Land--a dry place where the inhabitants dress in black and line up obediently against a white landscape-- and thought I would share some techniques on screener development.

1. Be explicit with logistics and instructions.

In the screener I include a participant matrix with desired quotas, and an introduction to the study. I wrap up each screener with a detailed invitiation script that includes: further details about the study, incentive information, recording, directions, and homework assignment (if applicable). I expect the recruiter to communicate all research details to the participant because often we are going into people's homes with a video camera. I've been in several situations where participants were caught off guard: "this wasn't expected." This American Life aired a piece about the fine print in contracts, referencing a myth that popstars like Madonna are highly demanding because they include line items like "all blue M&Ms must be removed from the candy jar" in their contracts. The reality is that these line items are included to ensure that the folks who are setting up the concert actually follow all directions to a tee: if blue M&Ms are in the candy jar, what other line items were overlooked? 

2. Write down who you want to recruit, and how samples might diverge based on behaviors, attitudes, and demographics.

When I talk with recruiters, I usually describe the desired sample in high-level ways: "Mostly young women who eat oatmeal every morning because (Group 1) they like the taste (Group 2) they believe it's good for them." I often find that this initial description (devoid of specifics, but that which captures the essence of who we want to recruit) is what sticks with recruiters, so be sure you are being adequately descriptive. Beyond that, I jot down on a sheet of paper how I might describe my sample in three different categories:

A. Behavior

  • Do they buy instant oatmeal or steel cut?
  • How many times a year do they go to Whole Foods?
  • How likely would they be to try a new kind of oatmeal?

B. Attitude

  • They believe that instant oatmeal doesn't provide the same health benefits as steel cut
  • They rank the taste of oatmeal higher than health benefits
  • They agree that oatmeal has changed over the past 5 years

C. Demographics

  • Mix of ages between 18-45
  • 80% female
  • Income range must fall between...

3. Don't reveal too much about what you want to study (unless you need to). Avoid making respondents hyper-aware of what population they might represent.

 When it comes to the topic at hand (oatmeal), I like to broaden it to a larger theme (breakfast food) and begin with questions that will eventually lead to oatmeal eaters. I try to continue the theme throughout the line of questioning.

BAD: We are working in conjunction with a client who wants to talk with oatmeal eaters.

GOOD: We are working in conjunction with a client who wants to know more about what you think about breakfast foods.

Avoid questions that compel a Yes or No response.

BAD: Do you eat oatmeal?

GOOD: I'm going to read a list of breakfast foods, which ones do you eat at least twice a week..."

Introduce options that may not be relevant to your study (in particular to behavior and attitude questions) to avoid serial participants.

BAD: I'm going to read a list of statements about breakfast foods...which ones apply to you?

GOOD: I'm going to read a list of statements about what is important to as a part of your morning routine...which ones apply to you?

4. Make edits, and let others make edits for you. Pilot your screener.

My process involves the unforgivable hardcopy print out. And dare-I-say, I opt for one-sided in this step. It's important for me to spread out the entire screener on a large table and think-aloud as I proceed through the screener logic. With my trusty orange Sharpie, I make changes big and small that will reduce confusion and ambiguity for the recruiter. With orange marker all over the screener, I hand it off to someone else on the research team who may be marginaly invovled in the project: they ALWAYS have great suggestions that leave me slapping my forehead. ALWAYS. Oh the power of working in teams! I then pilot the screener with someone who isn't in research so they can tell me if a particular question makes no sense.

5. Stay involved in the recruit, this allows you to ask more precise questions.

When I work with third party recruiters, I like to include "HOLD" questions that compell the recruiter to be in touch with me. This method works for me because third party recruiters I've worked with tend to recruit during the day anyway when I am accessible at my desk. These hold questions compel the recruiter to do their due diligence and find the right participants for the research. This technique also allows me to be a bit more nuanced in my line of questioning (i.e., more open-ended questions as opposed to multiple choice).

6. Validate your participants during the research session.

Knowing that I have the right person doesn't end when I receive a schedule of participants. The first line of questioning I employ during the research aims to validate some of the assertions the participant made in the screener itself. Face-to-face questioning often yields more candid responses that will help you identify if someone is a true representative of your population, or someone who simply slipped through the cracks.

Monday
Mar192012

HFES Healthcare Symposium: Bridging the Gap

[image credit: HFES.org]

Healthcare is a notoriously convoluted system with many intangible issues dictated by deep-rooted cultures and significant power structures. Human factors is a discipline with the reputation of being vague, yet with tactical applications: validate this, make that safe, etc. At this year’s Human Factors and Ergonomics Society’s Health Care Symposium, the theme of “bridging the gap,” for me, meant bringing together our knowledge of complex healthcare problems in an attempt to find the most meaningful way to address those problems.

Our natural reaction to complex, large problems is to simplify them. At the symposium, I saw presentations that did just that: a presenter used multi-variate analysis to distill conversation weaknesses into clear cut themes, another used generalizing language without site-specific implications, and many others pulled frameworks of understanding from other disciplines to explain observed phenomena. But does this desire to simplify the healthcare landscape accurately reflect its reality? In this symposium, three big-picture (and very complex) issues came up frequently—compliance, obscurity, and fundamentalism—for which simple sweeping analyses could not be put in place.

COMPLIANCE

During a panel discussion on challenges in home health care, Dr. Eric Dejonge of Washington Hospital Center said that the “biggest human factors challenge is getting people to take their medication.” Mary Brady of the FDA added that part of the reason why in-home treatment is so difficult is because patients in need of home healthcare don’t necessarily consider themselves sick, rather they view themselves as “aging in the home.”

Chronic illnesses, like heart disease and diabetes, which require sustained long-term care (i.e. treatment therapies, home assistance) are a big financial strain on the healthcare system (and on family caregivers). Remedied through medication and lifestyle adjustments, it can be difficult to get patients to fully comply with treatment because it brings into question their autonomy, and without many acute symptoms, they’ve become accustomed to a different quality of life.

Human factors professionals are crucial in highlighting significant patient-initiated barriers that impact the design and development of medical devices and applications. The essential message behind patient compliance is that just because something is designed to be safe and effective, doesn’t ensure it will change a patient’s likelihood to use/leverage it. However, because compliance is a well-known problem, we can begin to understand it and create solutions to approach it (accountability features, gaming mechanics to encourage participation, etc.)

OBSCURITY

Being such a large entity, the minutiae of everyday events in healthcare become lost or condensed into generalizations of “how the system works.” How can human factors attempt to lift this veil of obscurity and identify the real problems that need to be solved? One way of doing it is expanding our methodology toolkit.

In sociology, there is a field of study called ethnomethodology, which looks at large systems of structure that maintain societal order. Ethnomethodological studies uncover the real problem that exists in society, not just the problem that makes it into record-keeping. For example, what may be recorded at a hospital is the incidence of a particular illness, to which our natural inclination is to find a solution for that illness. Ethnomethodology attempts to examine the motivations behind how that illness became a recorded event in the first place.

Ken Catchpole of Cedars-Sinai Medical Center presented an example of how adverse events (recorded mistakes) become documented. In this process of documenting an adverse event, what doesn’t get captured is what he deems “the more important close-call.” That is, the event that could have become adverse. In a subtle way he references a need for ethnomethodology to understand the healthcare landscape. In this case, he suggests that we need to solve the “close call” episodes using our human factors expertise, and not simply the recorded adverse events.

FUNDAMENTALISM

No complex system, like healthcare, is complete without structures in place that allow it to operate. The key quality of these structures is having a hard line perspective on “what works” and a resistance to changing the status quo. This resistance is most visceral when introducing new technology into a clinical setting; however, it is also seen in the human-to-human relationships and interactions.

Dr. Lucian Leape of the Harvard School of Public Health talked about the individualism stronghold among doctors, which enables a culture of disrespect in clinical settings. Disrespect, he argues, is rampant and widely the norm. At the same time, clinical settings have evolved into teamwork-reliant systems. However doctors are trained to make executive decisions, trumping all other input streams, should they feel the need. The legacy of individualism in a teamwork setting makes other team members feel bad about their work, discredits their efforts, and leads to a breakdown in communication. It has been demonstrated that these seemingly small interpersonal issues have a greater impact, which can compromise patient safety.

HUMAN FACTORS IN HEALTHCARE: UNNERVINGLY VAGUE OR REFRESHINGLY OPEN?

“Bridging the gap” for me, meant bridging the gap between the obtuse landscape in which healthcare human factors specialists must operate, and the desire to have simple and actionable solutions. Dr. Leape, in his highly inspirational and eye-opening speech, also mentioned that as treatment and clinical environments have gotten safer, they have introduced layers of involvedness that impact healthcare professionals and therefore patients. As human factors specialists, we need to leverage our notoriously vague disciplinary standing and attempt to identify complex issues and weaknesses in the system, and solve them in the most impactful way (which may not be a design or development solution, for example). This symposium was a great first step in getting folks to reexamine the utility of human factors and what it means in this setting: rather than being unnervingly vague, it can be refreshingly open.

Tuesday
Jun282011

Social Science Recap [June 15-29, 2011]

Social Science Roundup--June 15-June 29, 2011 Interesting internet finds linking anthropology, sociology, psychology (other ologies) to design, user experience, and technology. In this post: apps that show you that you're a blabber mouth. Claims can become fact pretty easily, a case of a scholar being called out. Channel your inner Zack Morris/Jessie Spano and check out old technology. Tall grass jewelry? And many more...

Click to read more ...

Wednesday
Feb162011

Paying more for a simpler experience

 

Take a look at an interesting research article Siegel+Gale released on Global Brand Simplicity.

Watch out simple food does not equal healthy food! :)

Consumers in countries around the world want to simplify their lives according to a new survey from global strategic branding firm Siegel+Gale, and they are willing to pay between 5 and 6 percent more for brands that offer simpler experiences and interactions. The trend is most pronounced with technology and electronics offerings, where 22.5 percent of consumers surveyed say they are willing to pay more than 6 percent more for simplified brand experiences. Retail fashion brands also have much to gain, with 15 percent of consumers surveyed willing to pay a Simplicity Premium™ of 6 percent more.

The first annual Global Brand Simplicity Index™, conducted by Siegel+Gale in late 2010, polled over 6,000 consumers across seven countries in Europe, the Middle East, Asia and North America to uncover the perceived points of complexity and simplicity in people’s lives. It also uncovered the Simplicity Premium™, an exploration of the added value people would place on having a more simplified experience with brands in various industries...

Friday
Dec032010

You are biased!

Brian Heidsiek of Sandbox Innovation Studio wrote an interesting article about bias in the design process that is worth reading. You may recognize some biases you never thought you had. One message in his article is that designers have a responsibility to work beyond their biases and that empathy-driven relationship building is the way. I had a little think about the anti-innovation bias he described and added a comment to his article. Take a look and add to his article (or this note) if you like.

 

Monday
Nov292010

Online Education Tools 

I have been looking into some new methods/approaches and tools that we can incorporate into our research toolkit.  Back in the dim time (when I first started working at Essential), we drew some inspiration from cognitive behavioral therapy to develop hands-on activities, such as "Three Wishes" and "Transformational Portraits."

A friend of mine, who's a teacher, suggested that I look into online teaching tools.  Here are some great tools I've found so far that allow participants to collaborate create.

stixy.com acts like an online bulletin board.  It looks like it can be used to help keep families and teachers organized and up-to-date with each other, but we can use it for online collaboration, or as a more fun and interesting way to organize an online diary.  What's great is that it has sharing capabilities (which means we can see a participant's progress in using it).  I tried this out with Kate today and she said it was easy to use (drag and drop), her only gripe was that the briefing message I had included in the invitation was buried deep in the message.

the stixy interface. I asked Kate to contribute (and I've included the instructions to the participant at the top).

protonotes.com would be great if we are showing prototypes to participants.  Put an image of the prototype onto a webpage, and in the HTML, put in the code that will allow participants to add stickynotes on the prototype.  This would also be a neat tool to use with clients.  Click here to try protonotes!

Intel has some interesting education tools (they're not the prettiest things, and usability is another issue altogether), one of which is a ranking tool.  What's really useful about it is that once all participants have completed the ranking, the researcher can compare rankings all on one screen.

the visual ranking tool showing the comparison between two participants' rankings.

There are many more out there I'm sure, but this was just a small example of some that have come about through the education field.