YOUTUBE LIVE SEARCH RESULTS

2020

VALUE PROPOSITION

  • Affinity mapping workshop showed need to improve website’s search results usability

METHOD

  • A/B testing

IMPACT

  • Redesign reduced time to complete task by 48%

PAPER PROTOTYPE & GUERRILLA TEST - ITERATION 1 FEEDBACK

  • Confusion about what results were linked to content creators or channels

  • Dissatisfaction towards scrolling through search results

  • Desire for a direct link to artist’s official page

PAPER PROTOTYPE & GUERRILLA TEST - ITERATION 2 FEEDBACK

Added improvements from iteration 1:

  • “Subscribe” button of official pages was added to search results

  • Section dedicated to content creators was separated from other search results

    Feedback from iteration 2:

  • Visual polish with round form on artist’s or content creator’s photo while mainting remaining search results with square form Include clickable uploader’s name for quicker navigation

HIFI PROTOTYPE

  • High-fidelity prototype built in Figma

A/B TESTING METHODOLOGY

  • Duration: 10 minutes

  • Participants: 30 (15 group A and 15 for group B)

    • Inclusive criteria: 13-34 y.o., tech-savvy, high frequency and permanency of Youtube use (times per week), high discoverability of Youtube (number of different videos viewed, number of different pages visited)

    • Exclusion criteria: technophobes, non-Youtube users.

  • Task Scenario:

    • Task (a) - search for “justin t” keywords
      Task (b) - choose the music called ‘Cry Me a River’

  • Performance criteria:

    • Task (a) - one click and writing the keywords “justin t”

    • Task (b) - find the correct option (music called ‘Cry Me a River’) and one click

  • Variables:

    • Time: time elapsed between beginning and end of each task (no longer than 10
      minutes)

    • Clicks​: number of clicks

    • Errors​: number and frequency of errors committed while performing the task

    • Success​: success (task completed) or failure (task not completed)

  • Test environment:

    • Laptop computer

    • Logged out of Youtube personal account

    • Audio/Video recording with Bandicam

    • Brief/Consent/Debrief/Compensation

DATA ANALYSIS

  • Metrics: numerical and non-discrete (no need for categorization or normalization of the data set)

  • Outliers: no outlier removal needed due to fulfilment of the inclusion and exclusion criteria

  • Task (a) - same in both designs, not expected to present any performance differences

  • Task (b): 0 errors and 100% success rate through time on task and clicks variables

  • Mann-Whitney U test - used due to lack of normal distribution of data

RESULTS

  • The Mann-Whitney U test presents statistical significance (p<0.05) for the time variable, meaning that the difference between the time means on each group was significant and indicates that the redesign reduced the time to complete the task (7 vs 13.67 seconds)

  • The clicks variable didn’t present a statistical difference (p>0.05) even though the mean value for the redesign is slightly lower than the original (4.2 vs 4.866667), meaning there isn’t proof that the redesign affected the number of clicks throughout the task, which could have been affected by the number of users in each group to which it is advised that this number is greater than 30 for each group (60 in total) in the next iterations.

VALIDATION OF VALUE PROPOSITION

  1. Competitive analysis

  2. Identification of customer segment through personas

  3. Problem interview with users matching personas

  4. User flow of touchpoints in the user journey

  5. Wireframing

  6. Paper prototyping proper for Wizard of Oz

  7. Guerrilla testing

Competitive analysis

Personas

User flow

Wireframe