A competitor matrix becomes unreliable when it treats a market as a permanent table of vendors and checkmarks. Products change, plan boundaries move, buyers redefine the job, and apparently identical features acquire different limits. Replace the static matrix with a dated decision model: preserve claims as observations, compare commercial movements, and retire fields that no longer affect a real choice.

What this guide covers

  • why conventional matrices become misleading;
  • the difference between a feature, an entitlement, and a usable outcome;
  • a living comparison structure with explicit evidence states;
  • rules for adding, revising, and retiring comparison fields.

The matrix fails before the cells become factually wrong

The obvious failure is staleness: a plan changes and the sheet does not. The deeper failure is conceptual. A row called “AI agent” may be checked for two products even though one is an editor feature, another runs asynchronously, and each has different approval, security, and usage boundaries. The cells can all be technically defensible while the comparison is commercially useless.

Official pages show why the unit matters. GitHub Copilot plans describes plan and feature boundaries, while the Cursor changelog records product movement over time. Neither can be reduced responsibly to one timeless “coding agent” checkbox.

Treat every matrix as an answer to a specific buyer decision. If no decision depends on a row, the row is decoration.

Compare claims at four levels

Use four progressively stronger fields:

LevelQuestionValid entry
ClaimWhat does the vendor publicly say?source, excerpt, observed date
EntitlementWhich plan, limit, region, or role can access it?explicit boundary or unknown
WorkflowWhat job can a buyer attempt?defined inputs, steps, and dependencies
OutcomeWhat result has been independently established?customer evidence or not established

A vendor page is primary evidence for the vendor's public claim and offer. It is not independent proof of the result. This distinction keeps a useful comparison from turning marketing language into a fake benchmark.

Replace checkmarks with evidence states

A checkmark conceals too much. Use one of these states instead:

  • documented: current official source supports the scoped claim;
  • qualified: available only under a named plan, limit, or condition;
  • announced: public statement exists, but availability is unclear;
  • observed movement: a comparable public field changed between dated records;
  • unknown: the public record does not answer the question;
  • not applicable: the field does not fit the product's intended job.

“Unknown” is a legitimate result. Filling every cell rewards the analyst for overclaiming.

Organize the living matrix around decisions

Start with three to seven decisions that recur in customer or product work. A team evaluating coding agents might use repository context, autonomous execution, review controls, team governance, usage economics, and deployment boundary. A support platform would need a different model.

For each decision field, record:

  1. why the field changes the buyer's choice;
  2. the normalized definition;
  3. source types allowed as evidence;
  4. the current state and observed date;
  5. the owner who reviews movement;
  6. the retirement rule.

The EXVIV market map can help identify the category, the company directory can establish the candidate set, and Evidence Compare can support a narrower before-and-after review. Those are different jobs; do not force one surface to perform all three.

Add a movement layer, not just a current-state layer

The current offer answers “what is visible now?” Strategy often depends on “what moved?” A change in value metric, included usage, enterprise control, or entry motion may matter more than a new feature.

Stripe's pricing and packaging strategy illustrates the interaction among value metric, tiers, packaging, and price. Therefore a matrix that records only headline price discards much of the commercial system.

Use a movement ledger beside the current table:

| Observed | Company | Field | Before | After | Evidence | Interpretation | Owner |
|---|---|---|---|---|---|---|---|
| YYYY-MM-DD | | | | | primary URL | labeled hypothesis | |

Never overwrite the previous state. Create a new observation and retain the old one so later readers can reconstruct the conclusion.

Apply retirement rules

Remove a comparison field when it no longer distinguishes a decision, cannot be defined consistently, or has no maintainable evidence source. Merge overlapping rows. Split a field when plan or workflow boundaries make one label deceptive.

Review the matrix in two loops:

  • event loop: update a field after a verified material movement;
  • quarterly model loop: ask whether the fields, peer set, and buyer decision are still correct.

This avoids both extremes: a forgotten spreadsheet and a noisy feed that changes the model every time a page moves.

A matrix should produce a next question

The output is not “Vendor A wins 17–14.” It is a smaller set of questions worth resolving: Does the entitlement apply to our required deployment? Does the usage allowance fit our workload? Is an announced capability generally available? Would the change affect an active deal or our roadmap?

Sources and further reading

Method note

This framework compares dated public evidence. It does not independently test vendor performance or establish private roadmap intent. Recheck all volatile offer fields before using the matrix in a live commercial decision.