Menu

A journey from graphic design to UX research

Moving from graphic design to user research was a major shift of practice. This article looks at what it’s been like to transition from a purely creative space to a role that explores and defines user’s needs.

Working in a graphic design role for over 20 + years, there was a growing desire within me to not be the last step in the design process. From personal experience, I was handed briefs and schedules to deliver concepts based on a style guide or what the client wants. I never had the privilege of knowing what users actually need.

I’d started to see and hear about the growing popularity of User Experience (UX) and how a few of my college friends were beginning to make the switch from graphic design roles. Stumbling upon a social media post from General Assembly – an international training platform – about UX, I signed up to a free lunchtime talk called ‘What is User Experience’, held at their Sydney location.

Coming away from this introduction talk, it was the cliché lightbulb moment of my career is at a crossroads, and this was the road I wanted to take – to become a UX Designer! Enrolling in a course seemed like the most logical next step, and I was soon sitting in a classroom every Saturday for 8 weeks, at General Assembly Sydney, completely immersed in the world of design thinking.

Looking back, learning everything about UX in 8 weeks was challenging – wishing for more time to focus in on each stage of the design thinking process. However, the course gave me the fundamentals to start building out a career in UX.

There was also a false sense of hope that when you finished the course, you would be able to ‘jump’ straight into a UX position. Luckily within my role at the time of completing the course, my team were about to begin a major digital consolidation project. A multi-disciplinary team was set up with a mix of internal and external designers; user research, user interface, content, as well as product owners and front-end developers. I saw this as an opportunity to move across teams to use what I had learned at General Assembly and apply it to a ‘real’ project.

After a couple of months observing what role user researchers play in design thinking, it became apparent that that was where I wanted to be, as this is what I loved the most when studying UX. I transitioned to assisting the user researchers and began working with them. I loved every minute! 12 months later, I moved into a newly appointed Digital Experience Role and began working on small projects as the research lead.

Moving to Melbourne in 2019, I spent time moving between design and research until I joined U1 in 2022, working as a full time User Researcher. I’m exactly where I want to be and working in a space that I am passionate about.

Differences between design and research

Coming from a graphic design background, it has been challenging to switch my brain from being solution focussed to starting each project with no assumptions. However, it’s refreshing to not know what the solution may be. Having the opportunity to actively listen to user’s needs first, before even thinking about what colour the designers should use for a call-to-action button or that the solution is not always interface related, rather more at a product or service level.

I sometimes get asked how I embraced the transition from being solution focussed to insight focussed. For me, if you had asked me in high school what I would like to do with my career, becoming a graphic designer was surprisingly not on the list. However, I’ve always liked the beginnings of something, and I always favoured the conceptual side of design, and that’s what ultimately led me to UX and the researcher’s role – being part of the beginning (when possible) and understanding how that research can drive decisions for how a product/service is delivered. I feel that I’ll always be a designer at heart and want to suggest moving things around on a homepage, however, now those recommendations are based on user’s insights and how it could improve a product or service.

Tips for transitioning to research

The best tip that I could give anyone transitioning into research is to essentially be a sponge. Soak in advice from other researchers because we all arrived at research from different backgrounds, for example, content design, psychology, product design, and those different skillsets and experience can offer you different perspectives and knowledge.

I have quickly learnt that everybody has their own style of engaging with users, synthesising, and reporting the insights and even stakeholder management, however the one thing that you need is the ability to communicate and engage with users to allow for them to feel ‘safe’ to be open and share their insights with you. I have always seen the role of user research as being the voice of the user. There is something empowering about having the privilege to interview people, listening to how they use products and services, in addition to their motivations and expectations to identify opportunities or gaps. We are then able to analyse, and present recommendations based on those insights.

Another tip is to stay inquisitive, and openly seek out feedback from your colleagues as an opportunity to grow. It might pinch your ego sometimes, like when a client tells you they don’t like the colour palette you’ve chosen for a design, however, it’s an amazing opportunity to refine your craft as a researcher.

And most of all, never assume. That’s the best part of this role, you get to squash those assumptions by speaking directly to users.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Where do industrial design and UX research practices overlap?

As a UX researcher with an industrial design background, I sometimes reflect on my journey to get here. Recently, I have been thinking about what the Venn diagram might look like between the fields of industrial design and UX research, and I have been giving lots of consideration to where these two practices align.

So, I did what anyone would do in early 2023… I asked Chat GPT, and this is what it told me:

“When it comes to designing products that are both visually appealing and functional, industrial design and UX research share many similarities. While operating in different domains the two fields overlap in their focus on the user experience, problem-solving and innovation.”

At its core, industrial design could be described as the practice of creating concepts that fulfil both an aesthetic and functional role, as defined by their brief. Industrial designers traditionally have applied principles of engineering, material science, business data and visual art to create products and systems that are thoughtful, usable, durable and beautiful. They focus on understanding user needs as well as market trends and technological advancements.

Similarly, to industrial design, UX research is concerned with understanding user needs, expectations and motivations in order to create products that inspire, delight and embrace their user whilst achieving business goals. Additionally, both practices share a commitment to innovation, striving for product enhancement through qualified feedback data and contributing to concept generation.

UX researchers use a variety of techniques to uncover insights that are familiar to industrial designers, and whilst nomenclature and terminology may differ, both disciplines focus on understanding user needs and designing products that meet those needs.

In my opinion, it could be argued that the most important element that industrial design and UX research share is a drive to identify pain points experienced by users in their everyday lives and then surface them.  Whether this is designing a more ergonomic computer keyboard, a more comfortable kitchen chair or piece of furniture or a more intuitive mobile app, the goal is to create products that solve real-world problems and improve the user experience.

 

By combining these overlapping skillsets, both UX researchers and industrial designers help create products that truly meet the needs of users

 

Some of these shared common attributes between both practices are outlined below. Whilst not a comprehensive list, these characteristics embody the approach, skills and general curiosity inherent to both fields of expertise;

  1. Empathy: Given that UX researchers and industrial designers have the unique opportunity to shape and influence the future technologies they are involved with, it is hopefully not a coincidence then that both practitioners are typically passionate about having a positive impact on people’s lives
  2. Human-centered design: Industrial designers are trained to take a human-centred approach to design, focusing on the needs and desires of the end user. This same approach is critical to how UX researchers conduct effective practice, which strives to empower the needs, motivations, and behaviours of users
  3. Data synthesis: Strong data analysis skills are required of both UX researchers and industrial designers to synthesise and make sense of both quantitative and qualitative data sets. Combining these skill sets with a general curiosity allows practitioners in both fields to gain deep and evolving insights into how users engage with products and systems
  4. Attention to detail: Experienced industrial designers should inherently have an eye for detail, from the shape and texture of a product to the placement of buttons and other controls. This same attention to detail is required of UX researchers and is crucial to their practice, where even seemingly insignificant details can have a big impact on the user experience
  5. Creative problem-solving: Industrial designers are expected to be skilled problem-solvers, with an ability to conceptualise innovative solutions to complex design challenges. This same creativity, curiosity and problem-solving ability are invaluable skills exhibited by UX researchers, where practitioners deep dive, identify and surface user pain points and other issues that may be affecting the overall user experience
  6. Collaborative mindset: Industrial designers are typically highly collaborative, working closely with engineers and sales/ marketing teams to bring products to market. This same collaborative mindset is a valuable asset in UX researchers, where they usually find themselves working closely with designers, product owners/managers, developers, and other stakeholders to design and test hypotheses, prototypes and other stimuli to correlate findings

As the Internet of Things (IoT) overlays more of our everyday products, the intersection of these practices becomes increasingly important, as do opportunities for businesses to leverage these similarities between industrial design and UX research. Industrial designers, with their expertise in creating products that are both functional and aesthetically pleasing, are intuitively positioned to align with the practice of UX research… in my very biased ‘foot in both camps’ opinion.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Reflections on remote design research: The consultant’s perspective (Part 2)

This article concludes our series on the experience of transitioning from in-person to remotely moderated design research at U1. Our first article reflected on the experience of the consultancy as a whole, in the next article one of our consultants, Luella Paine, reflects on her individual experience. In this article, another of our consultants, John Eggleston, reflects on his experiences.

What have you learned?

That there is no loss of fidelity in the insights through conducting remote research. There are more variables such as participant’s devices and internet connection, but these variables contribute to a more authentic user experience. They certainly do not negatively impact the findings or the depth of the insights.

What has surprised you?

The effort which participants are willing to put into ‘making it work’. Participants have repeatedly proven to be tenacious in making sure the research session works smoothly despite using unfamiliar programs. Often participants need to install an unfamiliar program or browser extension to enable the research session to flow smoothly. I’ve found it surprising how often participants are happy to invest effort into making it work.

What has challenged you?

The few occasions when participants require additional support in using remote conferencing software. For example, if a participant has not installed a required program correctly (such as using a web viewer instead of a dedicated client) or is unfamiliar with a program and struggles to share their screen or access the chat functions.

These challenges should be resolved ahead of the research session and falls to the recruiter to effectively prepare the participant.

Other challenges include the environmental considerations occasionally affected the sessions such as dogs barking, children interrupting, noisy neighbours renovating their houses. However, these interruptions were normally fleeting and caused negligible impact on the flow of the sessions.

Generative research is better face to face, particularly when attempting to remotely replicate workshop style activities. Body language and the ability to stand up and move around actually plays a large role in enabling strangers to build on each other’s ideas. Previously a workshop with 8 participants could provide an opportunity for a range of paired and individual activities. Remotely, workshops are still effective generative approaches, but I’ve found they are more successful when limited to 4 participants.

What are the positives?

The increased convenience for participants improves convenience for researchers. There is less impact on the participant’s day, (no transport, parking, directions, or new devices) which means fewer instances of late participants or ‘no-shows’.

Additionally, it is much easier (and cheaper) to ‘cast a wider net’ and recruit participants from all over Australia and internationally. The alternative to remote research is the in-person interviews limited to either the Melbourne labs or requiring a researcher to travel broadly to access participants in other locations.

What are you looking forward to?

When the recruiters effectively prepare participants for remote sessions.

I suspect (maybe hope) that post COVID-19 vaccine there will be a balanced approach to remote and in person research as I think the benefits of remote research could be carried forward into the time when social distancing is no longer a requirement.

What will you take forward from the experience of the last 6 months?

I’ve developed positive habits for a work/life balance in the home. At the beginning of lockdown, I was struggling to isolate work life from home life and my focus and efficiency suffered causing work to bleed late into the evening.

I’ve become more familiar and more effective at providing remote troubleshooting support for a range of devices, programs, and operating systems.

Anything else you’d like to add?

Ugg boots and board shorts should become acceptable office attire once we return to the office.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Reflections on remote design research: The consultant’s perspective (Part 1)

In a previous article we reflected on U1’s experience of transitioning from in-person to remote research. That article reflected the experience of the consultancy as a whole, in this article one of our consultants Luella Paine reflects on her individual experience.

What have you learned?

Over the past six months of only conducting remote research, I’ve learnt to use many different remote testing platforms. I’ve also had to come up with creative ways to conduct sessions to cater for different kinds of research projects, and for participants with different levels of technical literacy.

With all our projects we don’t want the extra layer of a remote technology (such as Zoom, Teams or Lookback) interfering with the quality of our sessions or with a participant’s perception of the product or service we are researching. As a team, I think we’ve done a great job working out processes to seamlessly onboard participants and get them comfortable with interacting in a remote environment.

What has surprised you?

I think discovering that you can still get deep insights and rich data via a remote setting was quite surprising. With usability studies, there’s virtually no difference in the process whether it’s face-to-face or remote. But I’ve been pleasantly surprised that depth interviews and contextual studies in a remote setting also yield excellent results. In a recent remote study, my participants walked me around their homes to demonstrate where and how they used their virtual assistants. It was just like I was there at home with them.

What has challenged you?

Becoming familiar with the different platforms that can be used for remote research was initially a challenge. Each solution definitely has its pros and cons. I don’t think we’ve found the ‘perfect’ platform but we now know each one intimately and understand how to optimise its use for the requirements of every project.

Having everyone on the team dispersed remotely really helped to road test each of the platforms and nut out workarounds when a particular piece of software was less than optimal, but the client needed us to use it for a specific project.

And just when you think you’ve nailed it, then there’s a whole other set of constraints to deal with if the participant is using a mobile phone or tablet as opposed to a desktop or laptop computer.

But after six months of using multiple platforms and devices for all kinds of projects I feel that we have nailed how to get the best out of any remote situation.

What are the positives?

The positives of remote testing definitely outweigh the negatives. For starters it means that we have a much wider pool of participants to access from both within Australia and worldwide. We are no longer limited by how far we can physically travel and it opens up opportunities to reach people in really remote areas.

I find that participants also like the convenience and comfort of attending sessions from their home or workplace rather than having to travel to our labs. Anecdotally, I’ve noticed fewer no-shows and cancellations than when we were doing the majority of our sessions face-to-face in our labs.

I also think participants are less inhibited by being in familiar surroundings and using their own equipment. I’ve run sessions with participants on their couch or bed and even in their car (no, they weren’t driving)! Likewise, participants are very comfortable with their own computers and devices, so you get far more natural interactions.

What are you looking forward to?

I’m looking forward to finding a remote testing platform that does everything I need it to in a simple, user-friendly way. While our clients and participants would have no idea, sometimes we need to jump through a lot of technical hoops, or use multiple tools, to make our sessions roll out seamlessly for the required situation.

I would also really like to see my colleagues in the flesh! I haven’t been in the same room as any of them for over six months. While we have an informal remote catchup (a sit-down) every morning and there are numerous ways to collaborate remotely, I’d really like to pull up a chair and talk about things the old-fashioned way.

What will you take forward from the experience of the last 6 months?

One thing that working remotely has reinforced is the need to be prepared. While that was the same with face-to-face research, it’s been even more of a requirement with remote sessions.

Having that level of preparation, where participants have clear onboarding instructions and receive information about what to expect in a remote session, means that you can open up the session to expect the unexpected. Without the remote technology getting in the way, you are ready for the messiness of research – and that’s where the magic happens.

Anything else you’d like to add?

I would encourage anyone thinking about generative or evaluative research to consider the possibilities and benefits of reaching users remotely.

Even when COVID normal opens up the opportunity for face-to-face interviews, I think there is a real advantage in doing at least some of the interviews in a qualitative research project remotely.

Sorry. Gotta go. My participant has just arrived for a remote session!

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Employee Experience Research

What is your employee experience in a COVID-Normal or Post-COVID workplace?

We can help you understand what your employees need to continue adapting to new ways of working and ensure smooth transitions as your business transforms.

Talk to us about how we can apply human centred design and research methods to;

  • Evaluate the impact of the changes you may already have in place.
  • Work with you to innovate and explore new norms creating the right environment and conditions for employee satisfaction, retention, inclusion, and engagement.

We can answer the questions you have at team, department and organisation levels and help you understand employee behaviours across locally and remotely distributed workforces. We can then bring the insights and recommendations to life through the creation of employee personas and experience maps, and detailed reporting to take the guess work out of ‘what next?’

Get in touch with us to start the conversation.

 

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Reflections on remote design research: The consultancy perspective

2020 has been a year like no other (and we still have a couple of months to go!). The way we live, and work, has changed with a rapidity that at times has had our heads spinning. Back in March, U1 readily adapted to operating as a distributed team working remotely from our homes, a situation that is likely to continue at least through to 2021. We recognise that U1 is hardly unique in that regard, but over six months in we felt it was worth reflecting on what we have learnt and sharing some observations about conducting design research in this changed environment without compromising standards.

As a specialist design research consultancy, U1’s way of working over almost 20 years has predominantly constituted an in-person experience. Whether it is planning and conducting research, synthesis and analysis of research data, or engaging with clients to share research insights, in-person engagement with colleagues, clients and research participants has typically been at the heart of what we do. We routinely invited participants and clients into our purpose-built research facilities or were invited by participants into their homes or workplaces. All of these previously in-person interactions have now shifted online.

Remotely moderated research was already part of the U1 toolkit, but generally as supplement to in-person research allowing us to increase geographical scope of qualitative research where budget or time constraints worked against in-person research. What has been most interesting about our transition to running all research remotely research is how it has influenced the tools we use, our participant recruitment and the participant experience.

Tools we’ve used

Like many, we have used a variety of web-based applications for facilitating research, covering a pretty familiar rollcall of Zoom, Lookback, Google Meet (nee Hangouts), Skype and Teams. Whilst we are yet to come across the perfect tool that ticks off all of our research needs, Zoom has been our preferred tool for remote research. Zoom has a lot of handy features but perhaps most importantly, we have found it to have the lowest barrier to entry for participants, possibly one of the main drivers of its adoption by the general public during lockdown.

Our recruitment partners have been instrumental in assisting participants to download and  install any required applications or browser plug-ins ahead of any research sessions. This has saved us valuable time in interviews and workshops, allowing us to focus on the research questions rather than providing tech support. This period, along with our pre-lockdown  experience of remote research highlights the importance of recognising that what seems simple to us can be a challenging and stress inducing experience for others. Despite the best effort of our recruitment partners to prepare participants, we have had our share of participants on the phone in tears as they battle and fail to download and install a browser plugin. This is a participant interaction that never occurs in our research labs where the technology is all setup and ready to go as soon as the participant arrives. So, it is important that we remember that while these tools are great enablers for us, they can represent a significant barrier for others.

Zoom and Slack have also been invaluable in maintaining our culture of collaboration and connection. Friday pub lunches and daily coffee catchups on Zoom have been a feature of our lockdown experience, helping us to maintain some of the rhythms and non-project related interactions we were used to when sitting together in an office.

Recruitment

All researchers know that good recruitment is key to successful research and we feel that remote research has made life easier for our recruitment partners. Scheduling is much simpler as travel time and locations are no longer a factor, nor is finding someone to look after the kids. The available pool of participants is broadened as well, improving the strike rate of finding people who meet the recruitment specification. We have observed that remote participant “no-shows” are way down when compared to in-person research which helps keep projects on time and budget.

Participant experience

Prior to entering lockdown, we took all steps possible to ensure our research participants were at ease and comfortable with the environment and were over any stresses they may have faced on their way to attend our research. Our ability to manage the research environment was obviously diminished with the transition to remote research, so early on in lockdown we devoted some time in each interview or workshop to ask participants about their experience of remote research. Overwhelmingly we found participants were responding positively to the experience. Below is an example of the types of feedback we received.

“I find I’m using it more and more, especially in these times. Even my daughter’s dance class is in Zoom! It’s very easy to use.”

“Remote is really good. When the tech works and works well like this I feel much more comfortable doing it online. That’s fantastic! In fact, I’m in quarantine at the moment so this was the only way I’d be able to participate. Thank you for the opportunity.”

“Considering this is only the second time I’ve ever done it [remotely], it’s very easy. I don’t know if it would be much different even if you were here with me. Well, I guess you are here with me, aren’t you!”

“This is good. I don’t feel that it is any different. I’d do it in the future.”

What hasn’t changed?

Since entering lockdown, we have delivered projects for clients based throughout  Australia and collaborated with international partners on a global study. During this time our research has included straightforward usability testing for clients in the health, energy, finance and government sectors but has also involved exploratory research into the set-up and use of virtual digital assistants, attitudes and behaviours associated with choosing energy providers, and the information needs of franchisees and franchisors to name a few. The upshot being that whilst the way we engage with our participants and clients may have changed, the research and insights we are able to deliver have not.

What have we lost?

We do seem to have lost the engagement of a wider range of stakeholders in observing research. Whilst that is not the worst outcome, it does mean core stakeholders have to work a bit harder to spread the messages coming out of the research. In our experience, some of the most impactful conversations have occurred in our observation rooms as stakeholders from different parts of a business, who may be meeting for the first time,  come together to observe research. It is somewhat paradoxical that whilst it has never been easier to view research via live streaming, we feel there are fewer people beyond the core team watching the research as it happens.

The inability to get out in the field, or invite people into our research labs, has made it much more challenging to include people who have limited access to the technology that many of us are fortunate enough to consider ubiquitous. We should not assume that everyone has a smart phone or laptop or can afford a data or broadband plan. This is a divide we should always be mindful of, but one that is accentuated during an extended lockdown that restricts movement and interactions with others.

Where to from here?

We envisage that even when in-person research becomes possible and safe, remote research will continue to play a part in most, if not all of our projects. It makes sense to do so given the positives highlighted above. However, as an industry we need to remember that it can also represent a barrier to participation for some members of the community and introduce unintended blind spots to product and service design.

So, that wraps up our discussion of the collective consultancy perspective. In upcoming articles, members of our team will write about their individual perspectives and experience of transitioning from in-person to remote research.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Sample size in design research

In a previous article that you can read here, I wrote about the rationale, and guidelines, for choosing an appropriate sample size in usability testing. However, usability testing, or evaluative research, is not the only time you might be asked to defend, or provide a rationale for, sample size in design research.

As design researchers, we are routinely conducting qualitative research that is generative, or exploratory, in nature. When using methods such as mobile ethnography, workshops or contextual enquiry and making decisions about sample size, the rationale(s) that underpins recommendations for number of participants for usability testing does not automatically apply.

As with usability testing, in generative research we are not aiming for a representative sample. Our goal is typically to use this research to inform and describe an experience, which in turn provides the insight needed to make good design decisions. Importantly, the goal is not to predict uptake of a product or service, but rather what it needs to deliver in order to meet expectations. Consequently, our approach to recruitment is based on behaviours, attitudes and goals, rather than simple demographics.

However, as this type of research informs the early stages of product or service design and development, stakeholders can become even more nervous about sample size. So, what can we say to reduce their concerns and make them feel more comfortable about the sample size?

Some theories to consider

A good starting place is a consideration of grounded theory and theoretical sampling, concepts associated with the social sciences that represent research methodology and data collection approaches that inform theory. I won’t go into detail on these concepts, but their relevance to this discussion is that they provide theoretical underpinnings to the way in which we approach generative design research.

Theoretical sampling describes a process of iterative data collection, where a theory emerges and is refined, with each round of data collection and analysis providing the researcher with direction on what questions to explore next and who they should recruit to deliver the required insight. This process continues until “data saturation” is achieved, meaning no new information can be obtained.

So how does this relate to design research and sample size? With generative research we are typically starting with a question that needs answering but an open mind as to the answer(s). We recruit the participants that we think will provide the best insights, rather than those who are representative of an entire audience. We then iterate, further refining our questions and recruiting accordingly, until we reach a conclusion and feel confident that the right thing is being built.

So, how many participants?

But what about sample size? Well in grounded theory, and theoretical sampling, no maximum or minimum sample size is recommended or prescribed. In reality, there is no definitive formula or method we can employ to determine sample size for qualitative research. So, in the absence of a hard rule or equation, we instead consider factors such as time available, budget, research knowledge and expertise, nature and availability of participants and the scope of the question(s) we are asking.

Although I mentioned above that there is no definitive formula for determining sample size, that does not mean attempts have not been made to create one. Victor Yocco has provided a thoughtful argument and rationale for a potential formula that can be read here. Victor’s article provides an excellent starting point for someone who is starting out as a design researcher and does not work with, or have access to, more experienced researchers. However, in general, I prefer to avoid the temptation to quantify the qualitative and would pay more attention to the variables that are used as inputs for the equation rather than the equation itself.

So, where does all of that leave us in our goal to manage stakeholder concerns regarding sample sizes in generative design research? We can point to the well-established theoretical foundations that underpin the way that we approach design research. We can also argue with confidence that this type of research is about representation, rather than representativeness. Our goal is to describe and inform rather than predict, so thoughts about representative samples do not apply.

At U1 we always suggest a minimum of twelve participants across two workshops, or alternatively eight depth interviews as a starting point for any generative research project. Ultimately, in the absence of infinite budget and no timetable for delivery, we’ll draw on our knowledge of good research design, our experience as researchers, the nature of the questions being asked, the characteristics of participants we need to recruit and of course the available time and budget. Using these parameters we’ll plot the best possible course to deliver research and insights that deliver the best bang for buck in informing design decisions.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

How many users do I need?

Have you ever wondered about, or been asked to justify, the sample size used in a usability test? Well, you’re not alone! In this article we’ll explain why we are confident that six is enough, and how limiting sample size allows for iterative usability testing in place of spending your entire research budget in a big bang at a single point in the product development lifecycle.

How confident are you?

A common, and understandable, question when commissioning research with the goal of using the insights to inform business and design decisions is “how many participants are required for us to be confident in the results?” Many organisations feel more comfortable with making decisions based on quantitative data sets, leading them to query the reliability and validity of the smaller sample sizes often used in qualitative research. So, as design researchers, we need to be prepared with an answer other than “it is the industry standard”. We need to provide an answer that helps people asking this question to overcome any concerns and become advocates for the work we are doing.

It is important to recognise a significant distinction between quantitative and qualitative data, particularly in regard to design research. Broadly speaking, quantitative data is best used for measuring something, whilst qualitative data is best used for describing something. The former delivers the “what” and the latter the “why”. Both types of data have important, albeit differing and complementary, roles to play in design research. No design researcher will ever say that qualitative data reliably describes how an entire population will behave or respond, but when designed properly and interpreted appropriately, this data can be used with confidence to inform design decisions.

I heard that 5 was enough…

When it comes to usability testing, there has been a long-established reference to the work of Jakob Nielsen whose research, along with Tom Landauer (read the article here), identified five participants as the optimal number. Many people working as design researchers are familiar with the graph below, which illustrates that as the number of participants increases beyond five, the number (or proportion) of usability issues identified with an interface, or product, decreases. I am not going to get into the equation they employed to generate this graph, but the argument is that whilst you could involve more participants and most likely identify more issues, once the study exceeds five participants there is a diminishing return on the investment per participant. It is also likely that the major, or “show-stopper”, issues will have been identified and additional participants will reveal lower priority issues. This forms the basis of the economic argument for iterative testing, spreading available budget over multiple usability tests during the product development life cycle, rather than in one single, large-scale usability test. It is important to keep in mind that this graph, and the research it is based upon, is focussed upon discovering usability issues with a product or interface.

 

Graph describing number of users vs percentage of usability issues likely to be identified

What are your goals?

As always, there are some caveats to the rule of five participants, including the complexity of the tasks that will be attempted within the usability test, the number of audience types that need to be included, the novelty of the interface and the eventual application of the product (e.g. supermarket website vs clinical decision-making tool). These caveats, and how they relate to sample size for usability testing, are explored in depth by Janet Six and Ritch Macefield (read the article here). This article suggests that rather than adopting a one-size-fits-all approach to determining the optimal number of participants, that we should focus on a range which takes into account the complexity of tasks and novelty of an interface/product, amongst other things. They recommend a baseline of 5-10 participants for a standard usability test, but that number could go up to 20 depending on a number of factors (see graph below).

Graph displaying suggested number of participants for comparative and issue discovery usability tests

Nielsen also suggests that if there is a desire to incorporate different audience types in a usability test, who may be exposed to different tasks within the same interface or product, then allowing 3-4 participants per audience type is sufficient. Nielsen’s contention is that there is likely to be considerable overlap in observations of different audiences, given that usability issues are more likely to be related to a misalignment between expectation and actual experience, rather than differences amongst audiences. For example, the interaction design that underpins the process of submitting a form to a medical database will be the same for a General Practitioner and Specialist, it is just the nature of the questions they respond to in those forms that will vary, which is where the nuance lies.

Budget is always a factor

Ultimately, resources such as budget and timelines will have a significant impact on decisions regarding sample size for usability testing. Our collective, and individual, experience at U1 supports a minimum of six participants as a good rule of thumb for a relatively straightforward usability test on a standard website or app with a single audience type. Expanding sample size is important when the complexity of what is being tested increases and/or there are different audiences to consider. Our experience suggests that a minimum of four participants per distinct audience type allows for detection of any nuance related to that audience group, consistent with Nielsen’s recommendation, whilst also strengthening the contribution to the general insights and usability issues.

Where the goal of usability testing is to identify usability issues with an interface or product, the research of others and our experience suggests that 5-6 participants is typically sufficient. A major benefit of this “rule” relates to the economics of usability testing, allowing budget to be spread across multiple rounds of usability testing to deliver the best return on investment. In turn, allowing organisations to live the mantra of testing early and testing often.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Global Health and Wellbeing Study

U1 is a member of the UX Fellows, a global network of experience research companies. In May this year, the network pooled resources to conduct a global study into health and wellbeing. The subject of our study is of particular relevance during a global pandemic when the health and wellbeing of individuals and our communities are never far from our collective thoughts.

The study

Health, fitness and wellbeing are trending topics all around the world. In these times of self-optimisation, achieving a better health status has never been of more interest. The health, wellbeing and fitness markets are growing and with the current global COVID-19 situation during which people must self-isolate and often have more free time to spend than usual, self-guided health optimisation is soaring.

Attitudes and behaviors regarding the maintenance and optimisation of mental and physical health are very individual and will differ from person to person. The impetus for this research is the desire to explore whether the motivations for maintaining and optimising one’s mental and physical health and personal wellbeing differ between countries and cultures. We are equally interested to understand what impact, if any, COVID-19 and the associated restrictions has had on people’s attitudes towards this topic. We wanted to investigate whether as a result of the changes in the way we live during this time, people’s health behavior has changed and how it might differ between countries and cultures.

We conducted 128 remote interviews with participants across 15 different countries:

  1. Germany (DE)
  2. France (FR)
  3. Italy (IT)
  4. Spain (ES)
  5. Portugal (PT)
  6. United Kingdom (UK)
  7. Russia (RU)
  8. Poland (PL)
  9. Finland (FI)
  10. Turkey (TR)
  11. Mexico (MX)
  12. Colombia (CO)
  13. USA (US)
  14. Australia (AU)
  15. Singapore (SG)

As we were interested in discussing health, we recruited participants with already established health goals that they were working towards (e.g. fitness, mental health, stress reduction, better sleep, longevity or losing weight). The participants were aged 18 to 58 years and either use, or have considered the use of, apps or trackers that assist with achieving health goals.

The results

To see the results in detail you’ll need to download our report, but the following is a taster of what we learned by doing this study:

  • A similar holistic definition of health was identified amongst participants, regardless of which country they live in

“Health is a mix of everything. It’s not just a checkup, it deals with your way of life, the balance you find between professional and private life, your physical and your mental health” Bianca (36), IT

  • Coronavirus related restrictions are having positive and negative effects on health and wellbeing

“I am able to eat healthier and I am able to control when I eat a lot more. Like during school I don’t get a chance to eat lunch or breakfast because I am running around in the morning.” Lizzy (32), US

“I am spending a lot less time on things (i.e. exercise) than I did pre-COVID. I am a social person and enjoy that environment. Without that, the motivation is reduced at home. I wanted to buy some equipment when the lockdown occurred, but the stores ran out.” Suzannah (20), AU

  • Technology can play a significant role in optimising health

“I have some apps for guiding my meditations and following up yoga routines for beginners, I prefer those specialised apps instead of YouTube videos” Lesly (24), MX

“The Fitbit wasn’t very expensive for what I wanted, it helps setting goals and motivating, it’s a good device for monitoring and it connects with my smartphone.” Camila (25), PR

  • There appear to be some cultural differences regarding perceptions about COVID-19 contact tracing apps

“I hope that the app will be safe to use and no problems regarding data privacy will occur. I feel that the app should be taken into use if needed. ” Sanna (39), FI“If my friends started using this app – I’d have seriously reconsidered them being my friends” Olga (34), RU

If the above sounds interesting and you’d like to read more about how we conducted this study and what we found, download our report here.

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry

Keeping user research on track in response to COVID-19

Our first priority is the health and safety of our team, clients and research participants.

 

Otherwise, it’s business as usual.

In these uncertain times U1 would like to take this opportunity to let you all know that as far as our operations go, it’s business as usual within parameters permitted by the most current health advice.

 

Our first priority is the health and safety of our team, clients and research participants, but this does not necessarily translate to halting all moderated user/customer research activities. Across the industry, we have been pivoting towards use of web conferencing software tools like Zoom, Lookback, and WebEx which facilitate collaboration and interaction despite geographical separation. At U1, we have considerable experience conducting remotely moderated research and will make this our preferred approach for the time being.

Coincidentally, we spoke on this very topic at the Design Research 2020 conference recently and at the request of some clients we are developing some information around remote moderated research. If this is of interest please get in contact.

It is our experience that with the appropriate planning and forethought, remotely moderated research does not mean any compromise on the rigour or quality of approach or insights.

In fact there is some benefit to testing with users in their natural environment and using personal equipment they are familiar with to create the most authentic experience research possible.

Below outlines how we are managing current and upcoming research projects based on the best available latest advice from government and industry bodies such as the Australian Market and Social Research Society (AMSRS). Of course, given the fluidity of the current situation, we are constantly monitoring these sources and updating our approach and protocol as required.

Research planning

At this stage U1 recommends running remotely moderated sessions instead of face-to-face sessions. This requires no change to scheduling, lead times, or moderation guides and only minimal changes to setup. Each session will be video recorded with recordings available for download at the end of each day of sessions as per normal operations.

Recruitment

To facilitate an easy transition from face-to-face to remote sessions, our recruiters are actively managing the process by ensuring participants have required software installed ahead of any session.

As a business we are moving to 100% remotely moderated research with very little impact on our recruitment processes.

We have noticed a broadening of the pool of available participants, due to an increased number of people working from home and the ability to recruit Australia wide for user testing which may have previously been limited to participants based close to capital cities.

(Our international research and recruitment remains unchanged)

Contact us

If you would like some advice about planning and executing research in these challenging times, or how we can help you to meet your user research needs in the coming weeks and months, please don’t hesitate to reach out.

e. info@u1group.com

ph. 03 96843470

Get in touch

Find us at:

Level 24, 570 Bourke Street,
Melbourne VIC 3000
(03) 9684 3470

Email: info@u1group.com

Online enquiry