Data Analytics, Web based application
Defining data
Refining & evolving a tedious task
OVERVIEW
Problem Space
Analytical data is served up from hundreds of different sources across which can produce thousands of attributes from which to define a report. For most users, even highly experienced users, finding the attributes needed to get insights is difficult and very time consuming and for novice users it is overwhelming and can even be a blocker.
Product Goals
Evolve the data selections capabilities to match functionality from the old platform so all customers can migrate to the new platform to focus costs on one platform. All while refining the experience to make it simple enough for newer (less frequent) users to feel comfortable learning how to define reports.
Outcome
12 months of refinement initiatives after delivering core functionality to complete customer migration, we reduced time spent to get insights by 8 mins by reducing data selection tasks. This was nearly a 20% reduction.
UNDERSTANDING
With data selection being a critical step in the journey for over 100K monthly active users, we used several routes to learn about the space and uncover more helpful insight including:
- Led and conducted multiple rounds of qualitative research sessions with our external and internal users
- Collaborated with our product teams to get quantitative research from our online feedback tool
- Worked with other team members to collect and analyzed available analytics from our usage tool
- Met with multiple stakeholders to learn about the intricacies of the product and get their perspectives on the biggest problems and their vision.
Deliverables
- Created and shared a document that summarized the users we talked to and the major issues that they were experiencing
- This directly impacted the product roadmap
- Provided suggested next steps and areas that we should focus on first
IDEATION
As part of navigating the ambiguity of this space, I organized and ran a cross functional ideation session with members from our tech teams, product teams and fellow design team members to explore “how might we” data selection task.
What did we achieve?
By including those who might not normally be involved in the ideation steps, we gained some interesting ideas and perspectives. We were able to get better alignment across the teams as each member gravitated towards shared concepts.
Not only were we able to merge some ideas into stronger concepts, we also learned about some of the constraints in the system by ideas that pushed the borders of what is possible.
ALIGNMENT
Where did we focus?
With such a generalized problem space and many directions to pursue, it was critical that we identified and aligned on the areas that had the most value with a level of effort that could fit into the next 6-12 months. Based on value and effort, below are the top opportunities that we targeted:
- Natural language data selection – Allow users to simply tell the system what insight they were looking for in a single search query.
- Control the amount of complexity – Enable users to manage how much is displayed and empower them prioritize options that are most used.
- A shorter list of contextual options – Based on the data type we could present a shorter more relevant list of options
- Inline searching – Provide easy ways for the users to find the “one” thing amongst the many
Step 1: Natural Language Data Selection
Providing users the ability to simply enter in natural language what insights they are looking for was part of the original vision for the product. But it was considered way too much effort until the teams could lean on AI based learning models.
The Design Compromise The Discover analytic product was designed around a component (called the data summary) that helps identify how a report is defined. The “data summary” is the feature that would introduce natural language search (NLS) as an enhancement in the first delivery.
Due to technical constraints we could not swap out the new NLS view. It had to live as a supplemental view. This was not desired bc it added more visual complexity and it was meant to be a feature that replaced the other. As a first release, the NLS was delivered as an additional view as the feature is enhanced and adoption increases.
Step 2: Shorter Contextual List of Attributes
With hundreds of data sources feeding together there are hundreds attributes to select from in order to define or refine your report. But not all of those attributes are needed or even relative. We learned from our research that users typically use the same 8-10 attributes so we wanted to offer a more contextual feature to display these options.
This could be extended to many different scenarios including in our natural language search feature where users can quickly switch attributes that the system identifies from the search string
The Tech Challenges This feature was not able to be offered to all dimension types and all report types. The first iteration of this was considered to be delivered as a beta feature that users could explore the feature and are not required to use it. The language model was developed to best handle our most popular data inquiries.
Step 3: Control the Amount of Complexity
We learned in our research that based on the user type users were either overwhelmed or comfortable with the amount of attributes that were presented in one view. We redesigned the initial view to provide more structure and opportunity for prioritizing the attributes that are used most commonly.
Image callouts: Favoriting · Expand & Collapse Sections · Custom view settings · Customizable content settings
The Design Compromise Since we didn’t have any “intelligence” in the system to lean on, we had to rely on user interaction to manage it. Our initial ideas had more to do with “recommended” attributes based on the data. We wanted to remove the dependency on the users to manage it themselves.
Step 4: Inline Searching
Since we are dealing with a large quantity of attributes that are available, searching is a basic need for users. This feature was something that was originally implemented in the product at its inception but the technical approach made it unusable. It was later removed bc it was so frustrating to use. It was important that we met some basic expectations before attempting to re-deliver it.
- It must be able to search across all instances in the respective attribute type.
- It must return results in less than 3 seconds.
- It must be able to differentiate attributes from folders of attributes
VALIDATION
The validation (concept testing) for each of these feature sets were conducted separately as they were delivered at different times.
Step 1: Natural language data selection
What did we learn?
- The system’s ability to provide prompts and guide users through selections was appreciated, making the overall experience more user-friendly.
- Users did not understand the significance of different colors used, e.g., blue for matched dimensions, AI gradient for something Arthur suggests.
- The processes for changing time periods, making edits, and handling multiple selections was not intuitive for all users.
How did we iterate? We included suggestions to help users create more prompts that could produce more relevant results
Step 2: A shorter list of contextual options
Advanced users had become so accustomed to interacting with the side panel with all of the options that the new list menu was received with some apprehension by that user group. Whereas newer users or “lower-needs” users behaved like it was expected functionality
Step 3: Control the amount of complexity
Users responded very positively to all of the features tied to this initiative and how the content was organized. Users expressed willingness to manage their views but user analytics showed low usage in that specific area.
Step 4: Inline searching
Searching was considered a basic need and users were expecting that functionality all along so feedback was very positive mixed with a little bit of skepticism by users who had experienced the search feature that had been originally delivered.