Product Excellence Summit: Save the Date! Oct 4th, 2022 – San Francisco

Register now

Designing Insights Trends: How we evolved understanding customer feedback in Productboard

Designing Insights Trends: How we evolved understanding customer feedback in Productboard

Productboard’s customer-centricity is best embodied in the product by the Insights board: this is where customer feedback is effortlessly collected, product makers learn about customer needs, and can then easily link these needs or feedback to budding product ideas.

However, the Insights board was designed for smaller teams trying to collate feedback from multiple sources. As our customers grew, we faced a fresh design challenge: how to help our enterprise customers quickly find relevant feedback in thousands of captured notes?

Introducing Insights Trends — a brand new surface area within the Insights board that allows our enterprise customers to analyze customer feedback, at a glance.

Curious? Let’s take a look behind the scenes, and show you how we designed Insights Trends. You’ll read about our discovery process, what we gained by moving beyond Figma prototypes, and finally, we’ll share selected design highlights from the solution

Going from “Inbox” to “Insights”

The Insights board was originally built for smaller teams who needed to understand what their customers were saying, in one place. Its main value was in centralizing feedback from many different sources into one repository.

But, as Productboard grew larger, and larger organizations began using it, the volume of feedback collected started to outgrow the solution.

Our largest customers capture tens of thousands of notes and it’s been hard for them to identify opportunities in such a large volume of customer feedback. We started hearing about how much manual work they had to do to understand what was going on and saw that reviewing individual notes had become unsustainable for them.

Simply put, the Insights board was primarily an inbox and we wanted to change that.

Our objective became to move toward a vision where working with feedback at scale was easy and less manual. We wanted the Insights board to be a place for identifying opportunities, a place that enabled our customers to continuously learn about what their users need.

Discovering the right direction

With this vision in mind, we started reviewing what we knew.

And that was primarily this: one of the key features in Productboard built to help scale feedback, is the User Impact Score on the Features board: a number that aggregates the volume of feedback and its importance, linked to individual features.

That said, we knew from existing feedback that customers had a hard time learning about the trend in time. This meant they were still unsure whether customer need was growing or if a feature was generating more or less feedback after a release. It was also hard for them to answer questions like “What is our target segment asking for?” and “What topics are coming up recently in my product area?” without a lot of manual work, including diligent linking of feedback to feature ideas.

We needed to learn more about the key jobs-to-be-done, but we also had some early ideas that we wanted to test. We talked to our customers, combining traditional research interviews with early concept testing.

An early concept we used to accelerate our learning.
An early concept we used to accelerate our learning.

From these interviews, we eventually discovered three main jobs-to-be-done:

  • Identify opportunities during planning in my product area or for a specific segment
  • Analyze trends in time for specific topics when prioritizing between multiple ideas or during post-launch analysis
  • Never miss an important insight in customer feedback I haven’t yet reviewed

And we tested the early concept as well. It gave us a lot of good insights about which solution ideas had potential. Based on our discovery work up until this point, we identified several different possible solution directions:

  • Optimize the Features board for trend analysis
  • Create a new surface area in the Insights board with a trend summary
  • Build a full-blown, customizable reporting capability
  • Focus on summarizing raw feedback using machine learning

All of these solutions would have made sense in their own way.

But to choose the right direction, we had to consider our “appetite” — or how big of an effort we wanted to invest into this, and the vision we had for the Insights board: evolving it from an inbox to a place of continuous learning at scale.

Optimizing the Features board for trend analysis wouldn’t help us move toward that vision. A full-blown reporting capability would, but it would also be a larger endeavor, and we wanted to start delivering value to our customers quickly. And finally, summarizing raw feedback with ML would be valuable indeed, but it only addressed one of the main jobs-to-be-done.

All of our thinking led us to choose to create a new surface area on the Insights board with a trend summary as the best bet. It would allow us to move toward our vision and deliver value to customers reasonably fast.

Deciding to build on top of the Insights board gave us a great foundation: powerful filtering of notes that we could use to summarize the data. It would also give users a lot of flexibility without building a full-blown reporting functionality.

The next question was: Which data points were the most useful to surface?

We collected a lot of data about this in our initial discovery. Examples of things that PMs were looking to answer included:

  • Does customer feedback support our strategic focus?
  • What are the most requested things in my product area or by target segment?
  • Is this a growing need?
  • How does the feedback look after a new release?

To progress, we ran workshops in our product trio (PM, designer, engineering lead) where we analyzed the jobs-to-be-done — this helped us generate ideas around the type of data points we’d need in order to best support them.

We generated a lot of content ideas and focused on those which would best answer the questions above. After some synthesis, we ended up with 4 clear-cut modules:

  • Feedback volume over time
  • Trending items in feature hierarchy
  • Trending segments
  • Trending tags

Each of these would show trends based on how many notes were created that fit the user’s filters.

Figma prototype with four trends modules.

Having clarity about the content, I initially designed a Figma prototype (shown above) and we went to test our ideas with customers again.

While we learned something about what worked and what didn’t, it was quite limiting as long as we were using fake data. Customers struggled to understand the value of what we were showing them. Instead, we needed to show how the solution worked on customers’ live data.

We prioritized learning fast over polishing things up-front, so we made a plan for how to get this to market fast and decided to iterate afterward.

We cut out and built a very minimal and raw coded prototype that uses live data and we enabled it for selected customers in the beta programme.

Coded prototype with live customer data.

As you can see, we deliberately focused on surfacing the important data points without much regard to visual polish or additional functionality. No responsivity, no actions on individual items, and no configurability just yet.

We were ruthless in our cutting down so that customers would get this in their hands as soon as possible and we could continue learning.

Once in beta, we started collecting the most valuable feedback so far. We interviewed customers while looking at what Trends were showing them with their data.

This gave us a lot of new ideas for improvements. Throughout the beta phase that lasted over 2 months, we iterated on the solution a lot — in terms of functionality, but also the design polish this deserved.

Let me share some of our learnings and solution highlights.

From early feedback, we saw that customers were unsure about the numbers we were showing them in relation to time. They were asking things like “What time frame am I looking at right now?” or “What periods of time are the trend % based on?”

To address this, we surfaced the time filter and added explanatory tooltips that spell out the date ranges behind each percentage. This helped with understanding. Surfacing the filter was also beneficial because adjusting it was a common need.

Surfacing the filter and adding tooltips helped customers answer time frame-based questions.

Because Trends was an additional way of viewing data on the Insights board, we now had two kinds of views and needed a way to switch between them.

Initially, we designed a simple dropdown switch. The same pattern exists on the Features board and we reasoned users would be familiar with it and have no issues navigating to Trends. We couldn’t have been more wrong.

It soon became clear that users struggled with the discoverability of switching to the Trends view from Notes view. And even when they learned about it, it was inconvenient to always have to click twice to get there.

We decided to pivot to a one-click solution and used a segmented control instead.

Switching between Trends view and Notes view was cumbersome until we implemented a segmented control.

The result was dramatic: 80% lift in engagement. That was a big win for us since discoverability was more important than the consistency. The lesson? Obvious always wins.

During the beta phase, we spoke to product managers of different levels: from individual contributors (ICs) to product leaders (directors, VPs). It became clear that they needed to look at different levels of granularity when it came to the “Trending features” module. For context, the features hierarchy in Productboard has four levels: products, components, features, and sub-features.

We initially included only features and components in the module, omitting the less-granular products and more-granular sub-features. This was enough for IC PMs but directors and VPs needed to see the product level.

As a quick fix, we simply started displaying products in the module, but we knew this was far from ideal. The module needed to be configurable and that’s what we ultimately delivered: a set of options to customize which levels of the hierarchy PMs wanted to see.

Configuration settings for the Trending features module.

In addition, we also enabled customers to display breadcrumbs to understand where different features sat in their hierarchy. As we learned, this is especially useful when seeing features of the same name, but from different products.

As early as our initial discovery, we started hearing from customers that they wanted to “drill down” into the data. And this theme continued in the beta phase. But when we probed about what “drilling down” actually meant, we were getting different answers.

Some customers meant adding the given item as a filter so they could see the same view, just filtered on that one thing. They needed to go one level deeper.

But others meant reading notes linked to a trending feature or seeing customers from a trending segment.

Feature sidebar with Insights tab as a way to drill down into a trending item.

Addressing the two implied very different levels of effort. Opening items’ sidebars (which would address the latter group of use-cases) was a significantly cheaper solution than implementing drill-down filters which would require fundamental changes to the filtering logic.

We made this pragmatic decision, fully knowing that the drill-down filters are still valuable and likely something we’ll still build in the future.

Last but not least, we finally paid back the design debt of the live prototype by making the whole view responsive. We also worked closely with our design system team on improving the visuals of the underlying components.

The result is a view that’s not only visually pleasing but more importantly, easier to consume and work with.

We made sure that Insights Trends would be usable even on smaller screens.

It was a great journey through the beta phase. We learned so much and improved the solution step by step by listening closely to, and working with, our customers.

We ultimately released Insights Trends to all of our Enterprise customers just before Christmas. It’s an important first step in our new direction for the Insights board.

It’s now easier for larger customers to understand customer feedback at scale but we know there’s still a lot to do. We’re working hard on reducing the amount of manual work product makers need to do to really understand what their customers need and making Insights in Productboard a place for continuous learning.

We’re very excited about delivering further on our vision later this year! 🚀

Interested in joining our growing team? Well, we’re hiring across the board! Check out our careers page for the latest vacancies.

Credits to the cross-functional team behind this feature and the design team for the collaboration on this. Also to Zdenek, Kami and Allie for their help with this article 🙌

 

You might also like

7 ways to improve your designer–PM collaboration
Life at Productboard

7 ways to improve your designer–PM collaboration

Adam Amran
Adam Amran
Scaling our permission model
Life at Productboard

Scaling our permission model

Karel Skopek
Karel Skopek
How we measure adoption of a design system at Productboard
Life at Productboard

How we measure adoption of a design system at Productboard

Filip Danisko
Filip Danisko