We get a lot of philosophical exhortations to be inclusive, transparent, participatory, and ethical, but we don’t see any tactical ways to bring that to our practice. “Quantifying Qualitative Decisions” was a talk that Lynn Boyden has been developing for the past 9 years and we invited her to present her talk on to help us confront bias in our work.

The venue was further east than our prior events, so we made sure to plan for attendees who were arriving early. Julie, aka Jesse’s mom, donated beer, sparkling water, and ice, and brought us coffee and pan dulce from Mi Cafecito Coffee. Thank you, Julie! We appreciate you! Sending heart-fingers your way…

Key takeaways from Lynn’s talk

Lynn told stories of how she has used this framework in both her personal and professional lives, including deciding which high school her daughter should attend and helping the developers at USC decide on a tech platform to build their app for the App store.

A good and ethical decision, Lynn says, has the following qualities:

• Fair
• Value-driven — brings out the qualities are important
• Transparent — shows where and how decisions are made
• Adaptable — used across industries
• Inclusive — all stakeholders are involved
• Scalable — can be used by 1 person as well as by 100 stakeholders
• Consensual
• Participatory
• Documented — has an artifact at the end of the discussion

Once we understood what a good decision looked like, Lynn went through the steps of her framework to make sure that we saw how qualities are identified, then weighed in terms of importance to the stakeholders — the people involved in the decision-making process. The stakeholders then voted on in terms of the option being presented. Each quality was scored a 0, 1, or 2 to rank its importance to the stakeholder instead of a 5- or 7- or even (heaven forbid) 10-point Likert scale. Keeping it down to 3 points helps reduce decision fatigue associated with a longer Likert scale.

Some tips from Lynn’s experience

• The more people involved, the broader the quality criteria; the fewer people, the more granular.
• Send out a survey to participants asking them to determine how important qualities prior to the workshop — builds transparency for decision-making process when scores are calculated
• Send out final results through a spreadsheet or Google Sheet once or while results are tallied

Then it was our turn

We broke up into 6 groups with the goal to choose the best classroom pet. At least 1 person in the group was a representative from the Parent Teacher Association (PTA). We started with identifying and defining the qualities of a classroom pet, such as longevity, budget, convenience, and portability. Then we weighed the importance of each quality. Lynn assigned us 3 random animals as possible choices for a classroom pet. Pet choices may include a bunny, a hamster, fish, a lizard, or even the Madagascar hissing cockroach.

Some of us voted out loud; others voted on paper and passed a piece of paper around. We became passionate stakeholders. Animal rights and ethics became part of the discussion. But it didn’t matter. Lynn’s system eventually narrowed it down and the weights that were assigned to each criterion calculated played down the politics that are usually found in group discussions. And it turns out the goldfish won as the PTA-approved classroom pet for a couple of tables, except for the one group that was assigned a cockroach. Yes, even the cockroach was a winning choice for the classroom pet. Who would have guessed?

Challenges were defining subjective phrases, creating new phrases if a quality came up that was important to include, and comparing different qualities for the same pet. For example, does a fish refer to a goldfish or a koi fish? We had to clarify which type of fish it was to get a better understanding of how big it was, the level of care that particular fish needs, etc. It was tough for those who are not subject matter experts, but stakeholders could easily choose 0 or 1 in that case. Did you know that the hissing cockroach can live up to 5 years?! 😖

Bringing it back to SGVUX

The room was buzzing at the end because we were now equipped with a new tactical skill. As event organizers, we’ll be using this framework to evaluate our events. We now know that a good event is not located too far away from its attendees in terms of time and mileage and that it provides its attendees some kind of tactical skill.

Meanwhile, “Quantifying Qualitative Decisions” has given us a framework to remove bias from design by evaluating the criteria we use to look at design objectively. It’s a matter of defining what that criteria is meaningful to the stakeholders as well as what ethics are involved with what makes a good decision, such as what do the stakeholders find valuable? Are all the stakeholders included in this decision-making process? Is this decision adaptable for any changes the future might bring? This task is easier to do by following the aforementioned qualities of what Lynn said makes a good and ethical decision.

Resources

Presentation Slides (Google Drive)

• YouTube video + transcript (coming soon)

Next month, join us in Temple City as we look at designing events. What does it mean to design an event? We’ll be using the time to plan BBQ UX — where we bring together our love of BBQ and design and plan a conference that exemplifies a good event that is inclusive, accessible, and transparent — from registering, attending, and participating.

Our next event is July 2, 2019 from 7 pm to 9:30 pm in Temple City, CA. You can register at https://ti.to/thesgvux/sgvux5

Thank you

• Jessica Despard for researching venues, figuring out the projector,
• Julie Despard for figuring out food, drinks, ice, and partnerships
• Jennifer Du for copyediting and keeping sanity
• Frank Lee, Demi Minjarez for the photos
• to all the people who attended and helped out

And much appreciation to Lynn Boyden for speaking, facilitating, and fact-checking this article.