This is phase 0 of an end-to-end product development process at Nuvocargo. 

Check out the blank template for this phase here.

This project originally came up as an onboarding project when I joined Nuvocargo, with the expectation that it would be a pretty simple Google Sheet kind of thing. Carrier scoring had been buzzing around as an issue at the company, specifically that we didn't have any. And with a two-sided marketplace, you expect scoring on both sides. So it had initially surfaced as like, "Hey, can you create a very scrappy V1 of this because there's a need?" 

But precisely because of the Phase 0 process, as I was actually digging in and exploring the problem and why we should be solving it, it started to become clear that this wasn't just a little onboarding project but actually something that required much more thought and resourcing. It rose to the top as a critical issue we needed to solve to succeed moving forward.

As a part of my problem exploration, I did a bunch of market research on our competitors. Do they have a scoring system? And if so, what does it look like? Just looking at the companies we were aspiring to be peers with, it was clear they all had a scorecard. For me, if every major competitor you're going up against has something, there's something there. It also means we don't need to be reinventing the wheel. Therefore, what was essential to get the project prioritized was being really specific about why this is a problem, how it affects different teams, and why solving it would be what we need to do next.

The hardest part of the P0 was the metrics. In any project where what you are doing is going from this doesn't exist to this exists, there appears to be inherent value in existence. But in reality, you have to determine precisely how existence is better than the status quo. 

There were specific north star metrics that the carrier team was driving towards. I knew that if I could map the scorecard to the metrics that the carrier team cared about, that would help me in two ways: it would demonstrate the impact of this project in their language, and it would also align with the other initiatives that the team was taking on to drive those metrics.

In part, success is creating a foundation for measuring our carriers' quality. However, my hypothesis is that that foundation will be driven through these dynamic inputs, or the things that the carrier can control. We don’t want to score carriers on things that are outside of their control. If we focus on the dynamic inputs, then that's something that we can actually influence because we can see how they're performing and give them feedback. In turn, carrier performance will improve, and we hypothesize that improving the performance of our carriers will drive revenue retention for our shippers and platform engagement.

To determine what was in vs. out of scope, I kept coming back to the problem success criteria and thinking, "What is the smallest scope that allows us to achieve this?" To define that, I worked with the head of the carrier team, the head of our marketplace team, the data team, and our backend engineers to understand the level of data fidelity we needed to feel confident in giving feedback and making decisions. 

Had it not been such a collaborative process, it would have been easy on the one hand to over-scope it because the head of the carrier team thought it should be this really huge thing. Or, on the other hand, I might've gone too small. We really tried to think through the most foundational thing we could build right now and then iterate upon moving forward.

Weekly revenue retention was a North Star metric for the success of the business at the time, and platform engagement laddered up to that. So it took a lot of thought, but I felt good about ultimately being able to connect the foundation-building piece to what the company as a whole cared about and my primary stakeholder team goals.

Anaid Chacon

Isa Oliveres

I set out to make a scrappy, lightweight scorecard for Nuvocargo's marketplace suppliers, but quickly discovered that it was a huge opportunity for the business.

Driver: @Isabel Oliveres 
Approver: 
Collaborators:
Informed: 

What are the problems or opportunities being addressed?

What are the user problems/opportunities?

Note: projects may address only one of these problem types. That's ok.

We currently have no programmatic way of assessing the quality of our carriers, which creates numerous problems for the business:

It hamstrings our ability to deliver best-in-class service:

 with only anecdotal knowledge of which carriers are “good” and which ones are “bad”, shippers can have bad experiences, and we cannot hold our carriers to a consistent standard. We don’t have a holistic picture of how our carriers are performing on a per-shipper basis, which impacts our leverage in rate reviews and ability to address operational issues.

It limits the effectiveness of our matching efforts:

 we don’t know what the correlation is between carrier size, price, and service quality, which prevents us from making data-driven decisions when connecting both sides of the marketplace. In turn, our matching efforts are grounded in a relationship-based lens that rewards known carriers, who may not objectively be the best carriers.

It prevents us from enacting incentives for carriers to improve:

 without scoring carriers and giving them visibility into how they’re measured, we can’t use carrots/sticks to influence their behavior and reduce our cost to serve

This causes a number of problems for internal teams, carriers, and shippers such as:

 can’t compare service quality across carriers, so they default to assigning loads to carriers they’ve worked with in the past, reducing their leverage/efficiency, and limiting their ability to give feedback to carriers

 cannot develop Standard Operating Procedures for carriers since we haven’t defined or measured what good service quality means.

 cannot account for service quality in their carrier matching suggestions

 have no clear incentive to improve their service or become better Nuvocargo partners (e.g. engaging more with the platform, which reduces our cost to serve)

 have inconsistent experiences with Nuvo - recently, some new customers have had very negative first shipment experiences, and we don’t have a way of knowing if these were known issues with the carrier that could have been prevented by assigning a different carrier.

Carrier scorecards are common in the industry

 and allow companies to improve carriers' service quality and drive adoption of their tech products.

Who are the internal customers? How concrete or speculative are these customers?

Carrier Reps - see scorecards for their carriers and use it give carriers feedback and make better load matching decisions

Managers - understand service quality of carriers and take action to improve matching, incentivize better service, and inform carrier acquisition strategy

Marketplace Team: use scorecard metrics as input to marketplace initiatives (e.g. understand service quality delivered on a per-shipper to inform rate-reviews)

ICs: Advocate for shipper/carrier matching based on customer needs, feedback

Note: Depending on the size of the carrier, this could be all one user or multiple people.

Why is this important? What is the expected impact?

Why should this be done now? Why are we proposing this at all What happens if we do nothing?

What does success look like? What metrics will be tracked? What key result/ KPI are we hypothesizing this will drive?

In order for Nuvo to deliver best-in-class service to customers, we need to define and measure the quality of our carriers. Doing so is expected to:

Increase weekly revenue retention for shippers (through improved experiences with Nuvo)

Establish a programmatic measurement of carrier service quality in order to hold carriers accountable to these metrics and take actions internally to find providers that deliver a high level of service for shippers. This will enable an MVP target process of: 

Carrier reps using the Scorecard to understand the relative service quality of their book and leverage it to inform most effective load booking (e.g. give the carrier with the best service at the target price the load, even if it’s not the usual carrier)

Carrier reps use the Scorecard to give carriers feedback on their service quality and suggest improvements, compliment achievements (e.g. tell carrier their on-time pickup rate is unsatisfactory and needs to go up in order to continue working with a big client)

Carrier reps work with pod COps counterpart to gather feedback on effectiveness of scorecard-informed matching

Nuvo is at a stage where developing and rolling out a Carrier Scorecard is both feasible and a critical foundational step to ensure the company can scale successfully.

The Carrier Matching project will benefit from being able to include Carrier Scorecard metrics to improve the relevance of loads carriers are matched with.

The Customer Ops team has been asking for a Carrier Scorecard to help improve which carriers get matched with which white-glove clients, and the Carrier team is eager to have a Scorecard to help manage the supply-side of the business.

What key result/ KPI are we hypothesizing this will drive?

 for measuring the quality of carriers via 

 (i.e. attributes/actions that Carriers can control)

This is expected to improve on-time pickup / delivery rates and POD upload speed

The Carrier Team is able to see how each Carrier performs across a range of attributes and holds carriers accountable to these metrics

Other teams (e.g. Marketplace, CX, Carrier EPD) can use scorecard metrics as inputs to their own initiatives

weekly revenue retention for shippers (by helping us improve the service quality they receive)

product adoption and engagement, since carriers will find a benefit to taking actions on our platform, which in turn reduce our cost to serve

all-in net revenue (through improved matching of carriers and reduced cost to serve)

What problems or opportunities are not being addressed? (What is out of scope?)

Can this be split into separate pieces to address risk and/or deliver value sooner? If so, how?

What cross-team dependencies is this project likely to have? Who are the other EPD or non-EPD teams (Compliance, Marketing, etc.) who need to be involved?

What problems or opportunities are not being addressed? (What is out of scope?)  

 A comprehensive scorecard that encompasses every possible attribute related to a Carrier

Yes, we can split this into a very scrappy v1 that only captures metrics we can already track, and is enhanced over time.

Problem Brief: Carrier Scorecard at Nuvocargo

Phase 0: Problem Brief - Carrier Scorecard

Problem Space

Sign up to see more