How to Evaluate a Software Vendor's Technical Competence Before Signing

Every vendor deck has the same slides: logos, a tech-stack word cloud, a process diagram with "agile" somewhere on it. By the third pitch they blur together — and none of it tells you whether their engineers can actually build the thing.

Technical competence is verifiable. You just have to run the evaluation like an engineering interview instead of a procurement exercise. Here's the process that works.

Ask to See Real Code

Not a portfolio screenshot — actual code. Options that respect their client confidentiality: an open-source project, a sample repository, or a screen-share walkthrough of a past build (anonymized). You're reading for:

• Structure — Clear module boundaries, consistent naming, no 2,000-line files. You should be able to guess where things live.
• Tests that mean something — Not coverage theater. Look for tests around business logic and edge cases, not just happy paths.
• Commit history — Small, described commits suggest discipline. A history of fix, fix2, final fix suggests chaos.
• Documentation — A README that gets a new developer running locally in under an hour is a strong signal; its absence is a stronger one.

If a vendor can't produce any reviewable code, that's your answer.

Interview the Team That Will Actually Build

The people in the sales call are rarely the people writing your code. Insist on a technical conversation with the actual lead engineer, and treat it like a senior hire interview:

• Walk through your architecture and watch how they ask questions. Good engineers probe constraints — data volumes, integrations, failure modes — before proposing anything.
• Ask about a production incident they've handled: what broke, how they found it, what changed afterward. Teams that run real systems have scars and speak specifically. Teams that don't speak in generalities.
• Ask how they'd phase your build. Competent teams propose something shippable in weeks with a path to grow. Incompetent ones propose six months of work with a big reveal at the end.

Verify the Engineering Operations

The difference between a demo and a product is operations. Ask, concretely:

Area	What good looks like
CI/CD	Every merge runs tests automatically; deploys are one command, not a ritual
Environments	Separate dev/staging/production; you get access to staging
Monitoring	Error tracking and alerts from day one — they know about failures before you email them
Security	Secrets in a vault (never in code), dependency scanning, least-privilege access to your data
Code review	Every change reviewed by a second engineer, no exceptions

None of this is exotic. A vendor that hand-waves any of it is telling you how your production incidents will go.

The Strongest Signal: Do They Run Their Own Software?

A vendor that builds and operates its own product — with real users, real uptime pressure, and a real support inbox — has proven the full loop: not just writing code, but keeping it alive. Agencies that only build-and-hand-off have never felt the consequences of their own architectural decisions. Ask what they run, how many users it serves, and what they learned scaling it. The specificity of the answer is the evaluation.

Red Flags That End the Conversation

• A fixed quote before any discovery — They're pricing your project without understanding it; the gap gets closed later, at your expense.
• "We use AI so it's faster and cheaper" with no mention of review, testing, or architecture — tooling isn't competence.
• No questions about your existing systems — They plan to build in a vacuum and integrate "later."
• Refusing staging access or code visibility during the build — You're buying a black box.
• The senior engineer disappears after the contract is signed — Ask directly who is on your project and get it in writing.

The Bottom Line

Evaluating a vendor is an engineering interview, not a beauty contest — read their code, interview their leads, and audit their operations like they're joining your team, because they are. Reality Rift is built to pass exactly this evaluation: we run our own products (HelloAria serves 30,000+ users across 80+ countries), we start with a working demo before any commitment, and we hand over full source and IP.

Running a vendor evaluation right now? Book a free 15-minute call at cal.com/realityrift and put us through it.