Ideas

Quality, Trust, and Safety: Building Tools People Can Count On

By Clarence Wardell & Aras Jizan

Published on

12 min read

Whether it's fixing a leaky faucet or sorting out the right set of workplace benefits, many of us are increasingly turning to AI-enabled tools for help with questions mundane and consequential alike. As we engage with these tools, each of us has faced a similar set of questions: how do I know this answer is right for me? And if I share more about myself to get a better one, can I trust that my information won’t be misused?

The higher the stakes of the challenge, the more those questions matter. For something like fixing a leaky faucet, if the answer isn’t right, it might cost a little more time and money to fix the problem, but the outcome isn’t catastrophic. For other decisions — the make-or-break challenges Navigation Technology (NavTech) aims to ease, like building savings, navigating a career path, or choosing between healthcare options — there’s little room for error.

Workforce leaders, state and local officials, community college counseling offices, legal aid teams, and community-based organizations are asking similar questions. They’re in the market for tools that can help them work more efficiently and deliver better outcomes for those they serve. But as these tools shift from deterministic, rule-based logic to probabilistic models, buyers will only be comfortable with the most sophisticated, safe, and effective solutions to support their workflows, systems, and ultimately, their end beneficiaries. As one state cabinet official told us, “We’re being inundated with pitches, and we don’t really know whether these things work, let alone whether they’re safe and we can trust them.”

A tool people don’t trust can’t change lives or sustain a business — so rigorous standards for quality, trust, and safety will soon be the price of admission to the NavTech market. Meanwhile, if we get this right, the upside is enormous (opens in a new tab): a market full of dependable, proven tools that bring expert-level guidance to families who’ve never had the means to access it before.

That’s why, at NextLadder Ventures, we prioritize partnering with founders and companies that see QTS as central to their success. And we’re helping to provide the tools, resources, and market infrastructure to better enable QTS to be at the center of product development in the long run.

Defining QTS

At NextLadder, we talk about this set of imperatives through the lens of “quality, trust, and safety,” or QTS for short. Here’s what we mean:

  • Quality: Does a product actually work as intended? Is it accurate, reliable, and actionable?
  • Trust: Is that quality visible and verifiable to the people who use, buy, and fund these tools?
  • Safety: Are there strong protections against harm? (e.g. data misuse and predatory business practices)

We teamed up with the Center for Democracy & Technology (opens in a new tab) to develop eight principles to ground our perspective on QTS. Together, they paint a picture of what we believe good looks like for a NavTech tool. One that gives people accurate, actionable guidance and works equally well for everyone it serves. One that protects user data and prioritizes their safety, and that is open about how it reaches its recommendations. One that earns trust by profiting only when its users do, keeps them in control of their choices and information, and makes sure a real person is accountable and reachable when something goes wrong.

These eight principles pull from established frameworks, such as the NIST AI Risk Management Framework (opens in a new tab) and OECD AI Principles (opens in a new tab). Those frameworks prioritize accuracy, fairness, and transparency, but NavTech demands more. For tools serving families under financial strain, how a company turns a profit — and the incentives that creates — is just as important as the product. The best NavTech companies are built so that the business grows when the people it serves get ahead. That foundational alignment is what allows a product to preserve, or grow, its impact as it scales.

This isn’t easy; we’ve seen plenty of well-meaning businesses end up leaving their target populations worse off, whether by drift or by design. But that’s why we shaped our principles for QTS in NavTech through a consumer protection lens — to guide builders toward business models that succeed when the intended beneficiaries succeed.

Questions to Guide Design

Principles on paper only matter if they change what gets built in the real world. So we’ve distilled them down into four ready-to-deploy questions we ask of every prospective investment and that every builder should be able to answer.

Efficacy and Impact: Does your product work?

The most basic question is often the hardest to answer: does a product do what it says it will do? Not just produce answers that sound right, but actually hold up over time and deliver outcomes. A tool that can’t be relied on to produce accurate and actionable responses not only becomes useless over time, but can be actively harmful.

One of our first partnerships is with Vals.AI — an organization that (opens in a new tab) has done groundbreaking research into LLMs to evaluate if their answers to questions about financial health and (opens in a new tab) legal aid (opens in a new tab) are both accurate and relevant to a given situation. Their findings spotlight a real need for purpose-built tools that go beyond the generic one-size-fits-all answers to provide the right kind of advice: like a financial navigation tool that addresses the real-world choices its user actually faces, or a career navigation tool that provides a realistic roadmap to a higher-wage career. And these solutions need to offer answers that prove out across the full range of people and contexts it faces, not just those who look like its test cases.

These are tough challenges, where “good” can be subjective, and where the actual outcome won’t be known for years. Organizations like Recidiviz (opens in a new tab) and CareerVillage (opens in a new tab) have spent significant time and resources building the processes and infrastructure to give themselves and users confidence that their tools work. None of the current answers are perfect, but the field needs more exemplars like these.

Net Benefits: Does your product help people in the long run?

Sometimes, help can cost more than it’s worth. A tool can be great at keeping people engaged and still do little to improve their financial lives, or even harm them in the long run.

That’s why our Dollars in Pockets (opens in a new tab) metric is intended to reflect not just gains made in a given month, but how a particular intervention supports individuals and families to make progress over the long run. It’s the difference between economic security in the near term and a longer-term goal of true upward mobility.

The gold-standard model is an organization that offers guidance and navigation in the near term while helping people stabilize and climb to the next rung of the ladder over the long term. SaverLife (opens in a new tab) is one example. They use financial navigation to help their members build savings over time, creating sustainable pathways to financial security and lasting participation in the financial system.

Revenue: Do you win only when your users win?

The strongest NavTech businesses do well because they help the individuals or families do well.

The simplest version is a company that charges a fee that users continue to pay because they value the service enough. However, there are other business models that can work for NavTech companies that can allow them to make money while not taking advantage of end users. These range from government-payer models to advertising-based models. But the latter must be designed with care. For instance, Propel founder Jimmy Chen understands how trust with his community is critical to long-term business success. His platform runs on advertising revenue, and they only allow “advertisements on our app (opens in a new tab) [that] help put dollars into our users’ pockets, and we screen out anything predatory.” Meanwhile, other models tie revenue directly to outcomes: Advocate (opens in a new tab), which helps people apply for Social Security disability benefits, gets paid only when a claim is approved, through a federally capped fee drawn from the applicant’s back pay.

Finding that balance can be tough, particularly when trying to build sustainable businesses that can scale their impact. That’s why part of our work at NextLadder is helping builders find new ways to achieve sustainability and growth, from pooling demand across buyers to identifying new payers, to keep their businesses aligned with their users’ interests.

User Agency: Does your product respect user choice and agency?

“Nothing about us without us” — a foundational principle of the disability rights movement — points toward the importance of including user communities in product design and development. An inclusive process not only leads to a better product, but it recognizes that solutions intended to increase dignity and empowerment for their users have to be informed by those users. It’s the quickest way to build trust and to understand what will break it. Through partnerships like our work with Cortico (opens in a new tab), we’re putting this principle into practice: listening directly to people navigating economic transitions to shape where we invest and sharing what we hear publicly.

But agency shouldn’t end with design. For many NavTech users, there’s no one-size-fits-all “right” decision. The best path depends on a person’s circumstances and what they want for themselves. Tools that respect agency help people understand their options, clarify their goals, and move toward them, rather than nudging everyone toward the same “optimal” outcome. That extends to data, too: users need to stay in control of what a tool collects about them and how that data gets used.

After years of organizing, the National Domestic Workers Alliance (NDWA) knew that any tool they built to support those workers would not only have to be built with them; it would have to center worker dignity. So they built Ask Aya, an AI support system that helps those workers navigate hard workplace moments and advocate for themselves when they are alone, unsure, or afraid to ask someone else. They knew that a one-size-fits-all LLM chatbot would not be sufficient for the needs and concerns that were unique to those workers. Their approach, which they call “dignity-driven AI (opens in a new tab),” begins with the questions, “Whose life and work can this improve, and who gets to decide?”That approach shows up in how Ask Aya works. It doesn’t tell every worker to go demand a raise. It helps a specific worker in a specific situation figure out what they want, prepare for that conversation, and it knows when to hand it off to a human. The result is a tool workers trust, and one that has helped workers who were once hesitant to raise the issue negotiate for better pay on their own terms.

Measuring What Matters

Of course, these questions are only as useful as our ability to answer them empirically. Builders are telling us that the infrastructure for evaluating their tools doesn’t exist yet. Meanwhile, buyers need to know if a solution works before investing in it.

We don’t ask someone looking for a new car to crash-test it first. We built systems to test on their behalf. The future of NavTech, too, will depend on a similar approach: establishing shared benchmarks and methods of measurement, and institutions capable of independent evaluations.

That’s why we’re putting our capital to work tackling this challenge. We’re pioneering new category-wide ways to start evaluating how NavTech tools perform relative to those four critical QTS questions above.

As mentioned earlier, we’re working with Vals.AI (opens in a new tab) to build a set of social mobility benchmarks around issues like financial health (opens in a new tab) and legal aid (opens in a new tab). The early results make the case for everything above: even the most capable AI models are still missing critical details, like pointing people to quality local resources or asking crucial follow-up questions, when they give financial and legal guidance. These are just a few areas with lots of innovation that are of high interest and importance to those seeking economic stability — and where our ability to answer the question “is this a good solution?” has been lacking.

Similarly, our partners at the MIT BikeShop and their fellows spent a summer building some of this infrastructure for leading frontline nonprofits (opens in a new tab), helping those organizations build the evaluation infrastructure that gives them confidence to put their products in front of users (opens in a new tab).

These efforts are a meaningful start, but they’re just first steps. Eventually, we’d like to see the field establish a single, independent QTS credential: an industry-standard signal that can tell a user or a buyer at a glance if a tool is worthy of their use.

As Autonomy Grows, So Too Must Trust

Looking ahead, the future of NavTech is tools that don’t just advise you, but act on your behalf — like filing benefits applications or enrolling in a training program. That’s where these solutions can provide the most value. But it’s also where the stakes are the highest. One move can determine if a family gets ahead or slips further behind.

That’s why we propose a basic principle to guide NavTech development: The more a tool can do, the higher the standard of quality, trust, and safety it must show. And those higher standards won’t just be a nice-to-have — they’re going to be a strategic imperative for building a durable NavTech business.

But pushing those standards higher and higher is something no single company or fund can do alone. So consider this an open invitation. To builders, who are hungry to build tools truly with and for the people they serve. To investors, who value both impact and returns. To researchers, providers, and public sector leaders, who see every day what good looks like on the ground. Working together, we can build the exemplars, benchmarks, and evals that this fledgling category needs to take root.

And if we get this right, we’ll raise the bar for every technology that asks for people’s trust in their most consequential decisions.

Build

with

Us

Whether you have a question, would like to learn more about NextLadder, or are interested in becoming an investor, we’re here to help.

Get Started