How to choose a software development company when every vendor passes the checklist
Portfolio, reviews, technology, communication: every company on your shortlist will pass the usual checklist, which is why it rarely decides anything. Research on choosing software providers points to three tests that do separate them.
The short version
- The usual checklist does not separate vendors, because every serious vendor passes it. Research on provider selection found CVs and reference clients failed to separate competent providers from incompetent ones.
- Decide what you are buying before you compare who sells it: a project with a fixed scope, a team that owns a workstream, or engineers who join yours.
- Work on your own problem is the strongest evidence. A short trial screens out weak providers, but one study found it did not predict performance among those who passed, so make the trial look like the real work.
- How a vendor prices tells you how it will behave. When price decides the winner, the winner is disproportionately the one who underestimated the work.
- Ask who will actually do the work, by name, and whether they stay. Who owns the work predicts quality better than anything in the proposal.
Summarise with AI
ChatGPT
Claude
PerplexityEvery vendor passes the usual checklist
Choose a software development company on three tests the usual checklist misses: how it works on your actual problem in a short paid first stage, what its quote assumes rather than what it totals, and who exactly will do the work and stay on it. Put the same written questions to every vendor and compare the answers side by side.
Most advice on choosing a software development company is a checklist: look at the portfolio, read the reviews, check the technology stack, judge the communication, compare the price. It is reasonable advice and it rarely decides anything, because every company you would seriously consider passes all of it. They all have a portfolio, good reviews, the right stack and a responsive sales team.
Research on how software clients choose providers reaches a blunter version of the same conclusion. Magne Jørgensen at the Simula Research Laboratory showed that providers differ widely in productivity and quality, and that the traditional ways of evaluating them, such as CVs and evaluations by reference clients, fail to separate the competent from the incompetent.
One disclosure before the rest. We are a software development company, so we are one of the vendors this article is about. The tests below are the ones we would want a buyer to apply to us, and the evidence for them comes from independent research rather than from our own projects.
Decide what you are buying before you compare who sells it
Software companies sell three different things under the same name, and comparing a vendor of one with a vendor of another is how shortlists go wrong. Decide which one you need first.
| What you buy | Fits when | What it asks of you |
|---|---|---|
| A project with a fixed scope | The system can be described before it is built, and you want certainty on price and date | Someone who can decide the scope, and the discipline to price changes rather than absorb them |
| A dedicated team | The work is continuous, and you want a unit that owns a workstream and reports outcomes | Someone close enough to steer it, and outcomes clear enough to report against |
| Engineers who join your team | Your leads have the direction and need more hands | Leads with the time to direct the work day to day |
Each fails in its own way when its conditions are missing. Staff augmentation vs dedicated teams sets out the failure mode of each, and hiring a dedicated team covers that model in detail. If the real question is whether to build at all, start with build vs buy.
Three tests that do separate vendors
Test one: work on your problem
The strongest evidence of how a vendor will work on your system is how it works on your system. Jørgensen’s 2016 paper argues for exactly this: ask several providers to produce a sample piece of work and judge the work, which he calls trialsourcing.
A later field experiment by Jørgensen and Jon Grov tested it and found a limit worth knowing. Fifty-seven freelance developers with good ratings were invited to a two-hour trial task. The trial may have kept out developers without the skills, but among those who passed, performance on the trial largely did not predict performance on the project. The study involved freelancers and small projects rather than companies and large systems, so read it as a direction rather than a rule.
The lesson is to make the trial look like the real work. A two-hour coding exercise tests a skill. A short paid stage on your actual problem, such as a discovery that maps your integrations and data and ends in a scoped plan, tests how the vendor thinks, asks questions, writes things down and handles what it does not know. That is what you will be living with.
Test two: how they price
A quote tells you how a vendor understood your system and how it intends to behave when that understanding turns out to be wrong. Jørgensen’s review of estimation research found underestimation concentrated in price-competitive bidding, and names the result the winner’s curse: when a buyer chooses mainly on price, the winner is disproportionately the bidder who was most optimistic about the work.
So read a quote for what it assumes, not only for its total. What is excluded, which integrations were priced and how, how much contingency there is, and what happens to the price when the scope changes. The questions that explain the gap between two quotes are written for exactly this.
Test three: who actually does the work
The people in the sales meeting are rarely the people who build the system. Ask who will, by name and seniority, and how long they stay. It matters more than it sounds. A Microsoft study of Windows Vista found that measures of who owned the work, such as how many separate groups edited a component and whether one owned most of it, predicted post-release failures better than any measure of the code itself.
A vendor that cannot tell you who will work on your system, or that rotates people between clients, is offering you fragmented ownership. Meet the lead engineer before you sign, and ask what happens if that person leaves.
Signals that predict trouble
None of these proves a vendor is wrong for you. Each is worth a direct question before you sign.
| Signal | What it often means | What to ask |
|---|---|---|
| A firm price with no questions about your data or integrations | The vendor priced a picture of your system that may not match it | What did you assume about our data and each integration? |
| The lowest quote by a wide margin | Optimism, a narrower scope, or a plan to recover the gap in change requests | What is excluded, and how are changes priced? |
| No named team | The people will be whoever is free when you sign | Who works on this, at what seniority, and for how long? |
| Agreement with every requirement | Nobody has challenged the scope yet, so the arguments come later | What would you change or leave out, and why? |
| Code, accounts or hosting held by the vendor | Leaving later will be slow and expensive | Whose accounts is it built on, and what do we receive at the end? |
| Case studies with no one you can call | The proof cannot be checked | Can we speak to a client whose project resembles ours? |
One caution on references, given the research above. A reference call tells you whether a client was satisfied, which is worth knowing, but reference clients are chosen by the vendor. Ask the reference what went wrong and how the vendor handled it. That answer separates vendors better than the satisfaction does.
Questions to ask a software development company before you sign
Put the same eight questions to every company on the shortlist, in writing, and compare the answers side by side. Vague answers are an answer too.
Who will work on our system, and for how long?
Names, seniority and time on the project, and what happens when someone leaves.
What would you need to know before you could quote?
A vendor that asks about data, integrations and users has understood the work. One that asks only about features has not.
What did you assume is out of scope?
In writing, as a list. It is the fastest way to compare two quotes fairly.
How are changes priced?
Whether change requests are an exception or the business model.
Can we start with a paid first stage on our problem?
A discovery or a first release that produces something you keep, whoever builds the rest.
Whose accounts is it built on?
Code repository, cloud hosting, app stores and domains. They should be yours from the first day.
Can we speak to a client with a similar project?
And ask that client what went wrong and how it was handled.
What happens after launch?
Support, warranty, the handover of code and documentation, and who keeps the system running.
Does this vendor pass?
Tick what is true of the vendor you are considering. Score each company on your shortlist separately.
Where the company is based
Location matters less than it used to and more than some vendors admit. What it changes in practice is the working day: how many hours overlap with yours, how quickly a question gets an answer, and whether people can meet when it matters. Ask how many hours each day the team is available in your time zone, and how decisions get made when you are asleep.
We are based in Ahmedabad, India, so we have an obvious view. The useful test is the same for any location: whether the people doing the work can be reached when you need them, and whether you can meet them before you sign.
When you should not hire one
Choosing well starts with being sure you need a vendor at all:
- A product already does the job. Configuring software you buy is usually faster and cheaper than building it.
- Nobody on your side can own the scope. No vendor can succeed without someone who is authorised to decide.
- The process is still changing. Building it now fixes the wrong shape in code.
Questions about choosing a software development company
How do I choose a software development company?
Decide first whether you need a project, a dedicated team or engineers who join yours. Then judge each vendor on work on your actual problem, on how it prices and what it assumes, and on who will actually do the work. Put the same written questions to every vendor and compare the answers side by side.
What questions should I ask a software development company?
Who will work on the system and for how long, what they need to know before quoting, what is out of scope, how changes are priced, whether you can start with a paid first stage, whose accounts it is built on, which client you can speak to, and what happens after launch. The full list explains what each answer tells you.
Should I choose the cheapest software development company?
Not on price alone. Research on software estimation found underestimation is concentrated in price-competitive bidding, so the lowest quote is the one most likely to have underestimated the work. Find out why it is lowest before you trust it.
Are reviews and references enough to choose a vendor?
They are worth checking but they rarely separate serious vendors, since all of them have good ones. Ask a reference what went wrong and how it was handled, and judge the vendor on work on your own problem.
Should I ask vendors for a trial project?
A paid first stage on your real problem is the strongest evidence you can get. Keep it close to the real work: a field experiment found that a short, generic test kept out weak developers but did not predict how well the rest performed.
Who should own the code when a vendor builds my software?
You should, along with the repository, the hosting and any store accounts, from the first day. Check that the contract says so and that the accounts are created in your name.
Method and sources
The research below we read at source or, where the full paper is behind a paywall, from its published abstract, and its limits are stated in the text. The keyword volumes behind the topic come from a Semrush pull for the US on 5 October 2026. The questions, signals and the advice on trials are our judgement from being on the vendor side of these decisions, and are written as judgement rather than as data.
- Better Selection of Software Providers Through TrialsourcingJørgensen, IEEE Software 33(5), 2016. Providers differ widely in productivity and quality; CVs and reference clients fail to separate the competent from the incompetent; argues for work-sample tests. Read from the published abstract
- A field experiment on trialsourcing and the effect of contract types on outsourced software developmentJørgensen and Grov, Information and Software Technology 134, 2021. 57 freelancers invited to a two-hour trial; the trial may have screened out insufficient skill, but trial performance largely did not predict project performance. Freelancers and small projects. Read from the published abstract
- What We Do and Don’t Know about Software Development Effort EstimationJørgensen, IEEE Software, 2014. Underestimation concentrated in price-competitive bidding, and the winner’s curse
- The Influence of Organizational Structure on Software Quality: An Empirical Case StudyNagappan, Murphy and Basili, ICSE 2008. Organisational measures of who owned the work predicted failure-prone Windows Vista components better than every code-based measure tested
