Driverless Delivery:

First Usability Study

TL:DR

We had no understanding yet of what our driverless delivery product would look like and how a customer would engage with it in a user friendly way to retrieve their shopping.

I ran a Wizard of Oz test in which users explained how they imagined the service to physically work and we simulated this using tables, grocery boxes and bags.

I learnt authorising with the vehicle, height and tilt of presented groceries and onboarding upfront were key aspects that concerned the users.

This informed our initial hardware designs, allowed me to build a customer journey map and guided our study design for future iterations.

How did we approach it?

I set up a Wizard of Oz type experiment. This started with users imagining how a driverless delivery would work, from the notifications they expected to receive on the day of delivery all the way to finalising collection of their shop.

We then took their expectations and used boxes, tables, storage frames and bags of groceries to simulate this collection method and then followed up with further variations we had considered in a product ideation session.

In each variation users had to collect the bags of shopping as if they were their groceries and move them to the door of the lab (to simulate the walk from vehicle to front door).

What were we trying to understand?

Ocado had partnered with two companies providing driverless delivery software and had identified a driverless hardware provider. Our department was tasked with designed the hardware and software for the grocery retrieval system.

We had some design constraints for the exterior and size of our driverless delivery vehicle, but no understanding yet of how we would store the groceries within and how a customer would engage with the system in a user friendly way to retrieve their shopping.

We aimed to understand:

  1. How users expected the flow of a driverless delivery to work.

  2. What the key points of friction were likely to be in this experience.

  3. How physically a user might engage with the product.

What did we learn?

  • You can test anything, even if you don’t have the hardware yet to put in front of users.
    This underlying learning lasted through this project and all others during my time in Ocado’s robotics space and is the subject of a couple of conference talks I’ve given.

  • All users gave very similar expectations on the stages of the user journey.
    We built a user journey map that persisted throughout the whole of this product’s development and we used to guide future research.

  • Users were concerned about not understanding what to do and wanted clear expectation setting up front.
    We developed and tested an onboarding video.

  • Even small variations in presentation orientation and tilt made a huge difference to the user experience.
    We were able to guide hardware design decisions to be in line with comfort needs of users.

  • Amount to carry, more-so than distance, was highlighted as an issue for users.
    This guided our expectations on the proposed proposition of the offering from big basket to more immediacy type shopping.

Impact

  • I developed confidence to test hardware usability and experience before we even have initial prototypes built.

  • I created a user journey map that helped serve as a guide for what research needed doing in future throughout the product’s development.

  • I was able to give the hardware designers a clear steer on aspects around grocery presentation.

  • I was able to give the software team a steer on the expected front-end user experience.

  • The approach convinced the PM of the benefit of little and often UX testing throughout the product’s development, leading to around 15 different UX studies across the course of a year and a half.