Market intelligence platforms are compared badly more often than they are compared well, and the reason is structural: the dimensions that are easy to put in a comparison spreadsheet are mostly the dimensions on which vendors have converged, while the dimensions that decide whether a dataset answers a buyer's question resist being reduced to a cell.
This entry is the dimension index. Each dimension below is defined, given a way to score it, and linked to a deeper treatment where one exists. For the procurement process that sits on top of it — the sequence of steps, the questions to ask in a meeting, the RFP structure — see How to Evaluate Market Intelligence Providers.
The dimensions that actually differ#
| # | Dimension | The question it answers | Scored from |
|---|---|---|---|
| 1 | Source composition | What kinds of raw input produce these numbers? | Vendor documentation, verified against a sample |
| 2 | Coverage and completeness | What is inside the boundary, and how deep is it? | Known-item tests on a sample |
| 3 | Refresh cadence | How old is the newest usable record? | The sample's newest record date |
| 4 | Provenance and auditability | Can one figure be followed back to its components? | A drill-down attempt on a real figure |
| 5 | Granularity | At what level can the data be sliced without losing meaning? | Attempted queries at the level you need |
| 6 | History length and consistency | How far back, and is the series unbroken? | Back-period tests, restatement history |
| 7 | Access and export | How does the data reach the buyer's own workflow? | A trial extraction |
| 8 | Category fit and classification | Does the classification match how the buyer defines their market? | A classification test on ambiguous products |
| 9 | Analyst support | What happens when the data raises a question the product cannot answer? | The support terms, and a test question |
| 10 | Commercial terms | Rights of use, redistribution, continuity, exit | The contract, read before signing |
1. Source composition. Different source families fail in different directions, so knowing the mix tells you where this dataset's errors will live. A buyer should be able to say, for each metric they intend to use, which family produced it. Full treatment: Market Intelligence Data Sources.
2. Coverage and completeness. Presence versus depth — two questions that get merged, with expensive consequences. Score structurally rather than on an aggregate percentage. Full treatment: Data Coverage and Completeness.
3. Refresh cadence. Four separate clocks, of which only effective freshness affects a decision. Frequently over-weighted because it demonstrates well in a meeting. Full treatment: Data Refresh Frequency.
4. Provenance and auditability. Whether a specific number can be decomposed on demand. This is the dimension that matters least during evaluation and most eighteen months later. Full treatment: Data Provenance and Due Diligence.
5. Granularity. Category, sub-category, brand, SKU, variant, attribute — and whether the data holds together when aggregated up and drilled back down. Granularity is where a dataset that demonstrates well at category level often stops being useful, because most real questions are asked below that level.
6. History length and consistency. Length is easy to state and easy to verify. Consistency is neither, and it matters more. A ten-year series with an unrestated methodology change in year six is not a ten-year series; it is two series in a single column, and any growth figure computed across the join measures the vendor's operations.
7. Access and export. Dashboard, export file, API, or an analyst emailing a spreadsheet. The right answer depends on whether the data feeds a human decision or an internal system. Test it during evaluation rather than reading the description — extraction limits, rate caps and format constraints tend to surface only on contact.
8. Category fit and classification. The most under-examined dimension. Most buyers' commercial definition of their market does not match any platform's native taxonomy, so the vendor has to assemble it. How well that assembly is done, and whether it is done consistently over time, determines whether the numbers describe the buyer's market or an adjacent one.
9. Analyst support. Every dataset eventually produces a figure someone needs explained. What is available at that point — documentation only, a support queue, a named analyst — is part of what is being bought, and it is the part that decides how quickly an unexpected number becomes an actionable one.
10. Commercial terms. Use rights, redistribution rights, continuity if a source relationship ends, and what happens to delivered history at exit. Covered as due diligence in Data Provenance and Due Diligence, and worth reading before the dataset is designed into anything client-facing.
Weighting: decide before the demos#
Weights should be set from the decisions the dataset will serve, and written down before any vendor is contacted. Weights chosen afterwards get fitted, unconsciously, to whichever vendor demonstrated best.
| Buyer's primary use | Weight highest | Can accept less |
|---|---|---|
| Category sizing and annual planning | Coverage breadth, history consistency | Refresh cadence, granularity below brand |
| Competitive response and pricing | Refresh cadence, granularity, category fit | History length, breadth outside the category |
| Published or client-facing research | Provenance, sourcing and redistribution rights | Cadence, dashboard polish |
| Product and innovation research | Granularity, review and social depth, classification | Breadth outside the category |
| Feeding an internal system | Access and export, schema stability, continuity | Dashboard capability entirely |
The pattern worth noticing: no row weights everything highly, and every row explicitly accepts less on something. A comparison in which the chosen vendor is best on every dimension is usually a comparison that was scored after the decision was made.
Scoring mechanics#
- One scale, applied to every vendor. Three points is enough — meets the need, partially meets it, does not. Finer scales create false precision and invite the total to be tuned until it agrees with the preferred answer.
- Score from evidence, not from claims. Each cell should cite what it was scored from: a sample test result, a documented statement, a trial extraction. A cell scored from a sales meeting is a record of the meeting.
- Same questions, same category, same period, every vendor. Letting each vendor choose the demonstration is how a comparison becomes a set of highlight reels.
- Record the disqualifiers separately. Some findings are not points on a scale — a missing redistribution right, a category not covered at all, no export in the required format. These end the conversation regardless of the total, and burying them inside a weighted score is how a vendor with a fatal gap wins on aggregate.
Five ways a comparison goes wrong#
- Comparing features instead of data. The failure mode described at the top: easy to tabulate, largely converged, decides nothing.
- Weighting by what is easy to observe. Interface quality and refresh cadence are visible in an hour; provenance and classification consistency take a day of sample work. Comparisons drift toward the visible.
- Accepting each vendor's own demonstration. Every vendor is strongest somewhere and will choose that ground. Fix the category and the questions in advance.
- Scoring the pitch rather than the dataset. Responsiveness during a sales cycle is a real signal about the relationship, and no signal at all about the data.
- Not recording why. A comparison without a written rationale cannot be revisited when circumstances change, and the same evaluation gets run from scratch in two years.
The output: a comparison record#
The artefact worth producing is short. For each vendor: the score per dimension with its evidence, the disqualifiers found, and two or three sentences on where this vendor is the wrong choice. Then, for the selection: the weights used, the dimensions on which the chosen vendor is worse than a rival, and why that was accepted.
That last item is what makes the record useful later. When a limitation surfaces in the second year of the contract, the question in the room is always whether it was known at the time. A comparison record that names the accepted trade-offs answers it in one line — and a buyer who cannot name a single dimension where their chosen vendor is second-best has not finished comparing.
Where to look next#
Dimension deep-dives: Market Intelligence Data Sources, Data Coverage and Completeness, Data Refresh Frequency, Data Provenance and Due Diligence. For the procurement process: How to Evaluate Market Intelligence Providers. For testing a dataset before purchase: Sample Data Verification.
Common questions#
How is a comparison framework different from an evaluation checklist?#
An evaluation checklist establishes whether one vendor is credible, and it can be run on a single provider in isolation. A comparison framework establishes which of several vendors fits a specific buyer better, and it only works across a set. The difference shows up in what each produces. A checklist produces a pass or a fail. A comparison produces a ranked shortlist with the reasoning attached, including the dimensions where the winner is worse than a rival and why that was accepted. Buyers who run only the checklist end up choosing whichever credible vendor they saw last, because nothing in a pass or fail distinguishes two vendors that both pass.
Which dimensions should be weighted most heavily?#
It depends on the decision the dataset serves, and this is the choice that should be made before any vendor is contacted. A buyer sizing categories for annual planning weights coverage breadth and history consistency highest and can treat refresh cadence as near-irrelevant. A buyer running competitive response weights cadence and granularity highest and can accept a narrower boundary. A buyer whose output will be published or shown to clients weights provenance and sourcing rights above both, because the figures will be challenged by someone with an incentive to find fault. Setting weights after seeing demos means the weights get fitted to whichever vendor demonstrated best.
What is the most common mistake in a platform comparison?#
Comparing on features rather than on data. Feature lists are the easiest thing to put in a spreadsheet, so comparisons drift toward them, and they are also the dimension on which mature vendors have largely converged. The result is a matrix of near-identical ticks that discriminates nothing, while the dimensions that genuinely differ — what the data covers, how it was produced, how granular it is, whether an estimate is marked as one — go unscored because they are harder to reduce to a cell. A comparison whose deciding column is a feature checklist has usually not compared the datasets at all.
How many vendors should a shortlist contain?#
Three is the practical number for a full comparison, and two is often enough. Each vendor added multiplies the work, because a genuine comparison means running the same category-specific sample tests against each on the same questions, not reading each vendor's own material. Below two there is no comparison, only a decision about whether to proceed, and buyers with a single candidate consistently overrate it because nothing is available to contrast it against. Above three, the marginal candidate rarely changes the outcome and the added time usually comes out of the sample testing, which is the part that actually discriminates.