# How to Evaluate Market Intelligence Providers

> Five criteria for choosing a China or APAC e-commerce intelligence provider: platform coverage, SKU depth, panel vs census, history and cadence.

Source: https://moojing-global.com/wiki/how-to-evaluate-market-intelligence-providers

Last updated: 30 September 2026

Choosing a market intelligence provider for consumer e-commerce in China or APAC comes down to five things, namely which platforms the data covers and from when, whether it holds figures at SKU level, whether sales are counted from observed transactions or projected from a panel of shoppers, how deep and how consistent the history is, and how often each data type is refreshed and settled.

These criteria are written for the brands, retailers, agencies and investors who buy data on consumer goods sold through online marketplaces and social commerce. A buyer of commodity, industrial or B2B market data needs a different checklist, because that data comes from trade flows, exchanges and price reporting agencies rather than from marketplaces, and the questions that test it are different ones; [Market Intelligence Data Sources](https://moojing-global.com/wiki/market-intelligence-data-sources) sets out how the source families differ and what each can honestly support.

## Who is asking, and what they weight

The same five criteria matter to every buyer of e-commerce data, but they do not matter in the same order, and the order depends mostly on who in the organisation will have to live with the figures once the contract is signed.

| Role | Weights highest | Because |
|---|---|---|
| Marketing and brand managers | Platform coverage, SKU granularity | Their questions are about their own products against named rivals, on the channels where their campaigns run, including the content-led ones |
| Insights and strategy teams | Panel or census, historical depth | Their figures go into board papers and category plans that someone will challenge, and a multi-year series is only as good as its least consistent year |
| Procurement | All five, scored the same way across a shortlist | Their job is the comparison itself, which [Market Intelligence RFP Checklist](https://moojing-global.com/wiki/market-intelligence-rfp-checklist) is written to structure |

A marketing manager who wants the weighting worked through dimension by dimension, with a way to score each one across several providers at once, will find it in [Market Intelligence Platform Comparison Criteria](https://moojing-global.com/wiki/market-intelligence-platform-comparison-criteria). This page takes the procurement view, which is how to test a shortlist of providers against the five criteria before a contract is signed.

## Criterion 1: Platform coverage

In China and across APAC a category's online sales are split between search-led marketplaces, content-led platforms and social commerce, and the split differs by category and by market, so a dataset that covers the marketplaces but not the content-led channels will misread any category in which short video and creator posts drive discovery. The first thing to establish is therefore a list, per market, of what the provider actually covers. In China that means asking about Tmall and Taobao, JD, Douyin, Pinduoduo and Kuaishou; across Southeast Asia it means Shopee, Lazada and TikTok Shop, each of which runs a separate site for each country; for cross-border sellers it means Amazon's regional marketplaces; and on the social side it means Xiaohongshu, Weibo, Bilibili and the other platforms where consumers talk about products before and after they buy them.

**What to ask:**
- List the platforms covered, market by market, with the date on which each platform's series starts.
- Which platforms are covered in full and which only in part? The prestige tier of beauty, which sells partly through department-store counters that no online dataset can see, is the standard example, and [Data Coverage and Completeness](https://moojing-global.com/wiki/data-coverage-and-completeness) works through it.
- Where a platform runs separate country sites, is it covered in each country you sell in or only in some of them, since a regional platform name on a coverage list does not say which?
- Has any platform ever left the dataset? A platform that drops out silently removes a market from any cross-market series that ran through it.

**Red flags:** coverage stated as a region ("APAC", "Southeast Asia") instead of a list of platforms and countries; no start date per platform; a content-led platform missing from coverage in a category where that platform shapes demand.

For how Tmall, JD and Douyin differ as channels, in their commercial models and in the audiences they reach, see [Tmall, JD and Douyin: How the Channels Differ](https://moojing-global.com/wiki/tmall-vs-jd-vs-douyin) and [Douyin E-Commerce](https://moojing-global.com/wiki/douyin-e-commerce).

## Criterion 2: SKU and SPU granularity

Most of the questions a brand team actually asks, such as which of its variants is being displaced by which of a rival's, which shade or pack size is carrying a product's growth, or what price a new format should launch at, are SKU questions, and a dataset held at brand or category level can only answer them by estimate. Granularity also decides whether figures can be rolled up from variant to product to brand without being counted twice along the way, which is the problem [SPU vs SKU](https://moojing-global.com/wiki/spu-vs-sku) describes, and a provider should be able to say at which level each of its figures is held before a buyer relies on any of them.

**What to ask:**
- At what level is each figure held: category, brand, product (SPU) or variant (SKU)?
- Can a category total be decomposed into the products that make it up, and do the parts add back to the whole?
- Is the level of each figure stated alongside it, so that a SKU figure and an SPU figure are never shown in the same column?
- Can attributes such as ingredient, format and pack size be analysed, and are they taken as sellers wrote them on the listing or cleaned into a consistent taxonomy? Attribute analysis is where [Price Band Analysis](https://moojing-global.com/wiki/price-band-analysis) and most product-development questions begin.

**Red flags:** a demo that never leaves category level; SKU detail offered only as "custom analysis" at extra cost; product and variant counts that change depending on who in the provider's team you ask.

For what a transaction-level record contains and how it is aggregated upwards, see [E-Commerce Transaction Data](https://moojing-global.com/wiki/e-commerce-transaction-data).

## Criterion 3: Panel or census

A consumer panel records the purchases of a recruited sample of shoppers and weights that sample up to the population, so its error is sampling error, and in exchange it can see across channels, offline ones included. Census-style e-commerce data observes activity within the platforms it covers and projects nothing beyond them, so its error lies at the coverage boundary, and within that boundary it can go down to SKU level without a sample running thin. Neither method is more accurate in general, and the buyer's job is to know which of the two they are buying and, just as importantly, where the provider has estimated a figure (sales volume inferred from rankings or review counts, for instance) instead of observing it. [Panel vs Census Data](https://moojing-global.com/wiki/panel-vs-census-data) gives the full treatment.

**What to ask:**
- Is each sales figure observed, estimated or projected from a panel?
- If a figure is estimated, what is the method, and is that stated next to the figure itself?
- If it comes from a panel, how large is the panel, how are its members recruited and weighted, and how thin does it become at SKU level in a mid-sized category?

**Red flags:** the word "estimate" appearing nowhere in the methodology; observed and estimated figures shown in the same column with nothing to distinguish them.

Whether a specific figure can be followed back to the records that compose it is a separate question from how the dataset was built, and [Data Provenance and Due Diligence](https://moojing-global.com/wiki/data-provenance-and-due-diligence) covers it; [Sample Data Verification](https://moojing-global.com/wiki/sample-data-verification) sets out how to test both on a sample before signing.

## Criterion 4: Historical depth

Year-on-year analysis needs a multi-year series collected under a consistent method, and the failure this criterion guards against is a series with an unrestated methodology change in the middle of it, which is really two series sharing one column, so that any growth figure computed across the join measures a change in the provider's operations rather than a change in the market. In e-commerce the commonest version of this is outside the provider's control: platforms restructure their category trees from time to time, and when they do, year-on-year comparison inside the affected categories breaks until a full year has passed on the new tree, unless the provider has mapped the old tree onto the new one and says so. Content-led platforms also entered e-commerce later than the established marketplaces, so any long series is shorter on some channels than on others, and a provider should state the start date for each platform separately, since a single history quoted for the whole dataset hides exactly that difference.

**What to ask:**
- When does each platform's series start?
- When the method changed, was the older data restated to match, and is the change dated and documented?
- How does the provider handle a platform's category restructuring, and does it tell clients when one falls inside a comparison window they are using?

**Red flags:** a single history claim covering all of the provider's platforms at once; no answer to the question of what changed, and when.

Promotional calendars raise the same issue on a smaller scale: the pre-sale cycle around [Singles' Day (Double 11)](https://moojing-global.com/wiki/singles-day-double-11) has been lengthened and redefined over the years, so a comparison of the festival period across years is only meaningful if the provider says how it defined the period in each of them. The time axis in [Data Coverage and Completeness](https://moojing-global.com/wiki/data-coverage-and-completeness) covers how to test a series for breaks of this kind.

## Criterion 5: Update cadence

The right cadence is a property of the decision the data has to serve, and buying a faster refresh than the decision needs costs money and adds revision noise without making the decision any better.

| Decision | Cadence that supports it |
|---|---|
| Pricing response | Daily to weekly, and only for the platforms and categories where prices actually move that fast |
| Competitive defence | Weekly to monthly |
| Share, size and growth | Monthly, on settled figures |
| Annual planning | Monthly to quarterly |
| Category research | Quarterly, with a consistent history |

The distinction e-commerce buyers most often miss is the one between when data is collected and when a month is settled and final, because returns, cancellations and late-settling orders take days or weeks to resolve, so a figure published before they have is a first estimate that will move. For share, size and growth, which is what most brand and category decisions rest on, a settled monthly figure is the correct basis, and a faster one is simply a less finished version of the same number.

**What to ask:**
- How often is each data type refreshed, stated per platform?
- When are a month's e-commerce figures settled, and are they revised afterwards, and if so within what window?
- Does social data run on a different cadence from sales data, and which platforms are on which?

**Red flags:** a provider that describes its data as refreshed faster than it can say how often, per platform and per data type; no distinction between when data is collected and when it is settled; processing lags that nobody in the provider's team has measured.

[Data Refresh Frequency](https://moojing-global.com/wiki/data-refresh-frequency) separates the four clocks that sit behind any claim of freshness, and explains why the fastest available cadence is usually the wrong thing to buy.

## How to test all five before you sign

Procurement is one of the few moments at which the question of whether a dataset is any good can be tested empirically before committing, and a structured comparison makes the most of it:

1. **Ask for a sample on your own brand**, in your own category, on the platforms you sell through, for a period in which you already know your own sell-out figures, because a benchmark against numbers you already hold is the only accuracy test that does not depend on the provider's word. [Data Accuracy Benchmarking](https://moojing-global.com/wiki/data-accuracy-benchmarking) explains how to read the result.
2. **Run a side-by-side query** by picking a question you genuinely need answered and asking each provider to answer it from its own data, with the method disclosed.
3. **Score each provider against the five criteria** on the same scale, so that the scores can be set side by side once the demos have been forgotten.
4. **Trial the platform on a real workflow** through a paid pilot, which most reputable providers offer, because the dashboard that is still in use after a month of real questions is the one worth buying.

[Sample Data Verification](https://moojing-global.com/wiki/sample-data-verification) sets out what to check in a sample once it arrives.

## Methodology transparency, which runs through all five

Each of the "what to ask" lists above is a methodology question, which is why methodology transparency does not appear here as a criterion of its own. A provider whose methodology page answers those questions per platform and per data type, says what is excluded, and explains how any estimated figure was produced has already done much of the evaluation for you, and a provider whose page answers them only in aggregate, or in adjectives, has told you something as well.

## Where to go from here

For the whole four-pillar approach to consumer e-commerce data, see the [market intelligence platform](https://moojing-global.com/market-intelligence) overview. Teams whose question is what their rivals are selling, and at what price, will find the relevant use cases under [competitive intelligence](https://moojing-global.com/competitive-intelligence), and teams whose question is why consumers choose what they choose will find them under [consumer insights](https://moojing-global.com/consumer-insights).

Within this wiki, the other entries in the buyer's guide are [Market Intelligence Platform Comparison Criteria](https://moojing-global.com/wiki/market-intelligence-platform-comparison-criteria), [Market Intelligence RFP Checklist](https://moojing-global.com/wiki/market-intelligence-rfp-checklist), [Market Intelligence Pricing Models](https://moojing-global.com/wiki/market-intelligence-pricing-models), [Build vs Buy Market Intelligence](https://moojing-global.com/wiki/build-vs-buy-market-intelligence) and [Data Refresh Frequency](https://moojing-global.com/wiki/data-refresh-frequency).

## How Moojing approaches this

Moojing's e-commerce data is held at SKU level from marketplace observation across Chinese and international platforms, census-style within the platforms it covers, with history on Taobao, Tmall and JD going back to 2016 and on Douyin from 2022. E-commerce data refreshes monthly; social data is mixed, monthly on Weibo, WeChat and Xiaohongshu and daily on Douyin, Bilibili and Zhihu. Coverage and history are stated per platform in the [research methodology](https://moojing-global.com/about/research-methodology), and scoped sample data is available during evaluation. For how this fits the four-pillar approach, see [market intelligence](https://moojing-global.com/market-intelligence).

## Common questions

### What evaluation criteria should marketing managers use for a competitive intelligence platform?

A marketing manager should weigh three things ahead of the rest, and the first is whether the platform covers the channels where the brand's campaigns actually run, including the content-led and social commerce platforms where short video and creator posts drive discovery. The second is whether it holds sales and price figures at SKU level for the brand's own products and for the rivals it is measured against, so that a question about one shade or one pack size can be answered from data rather than by estimate, and the third is whether its refresh cadence matches the campaign cycle the figures have to inform, which for most brand teams means a settled monthly read, seldom the fastest number on offer. Historical depth and methodology still matter, but they tend to matter more to the insights team that inherits the figures later, and the comparison-criteria entry sets out how to weight all of them across a shortlist.

### How should I weigh methodology transparency vs use-case fit when they conflict?

Use-case fit is best treated as a quick check for a single decision, and in practice it comes down to the first two of the five criteria applied to the question in front of you, namely whether the provider covers your category on the platforms you sell through and whether it holds figures at the level of granularity the question needs. When time is short those two are the ones to test first, because a vendor with a perfectly documented methodology that fails them is of no use for that question. Methodology transparency, which runs through all five criteria, carries more weight for anything that will be repeated, compared across years or put in front of a board, since a vendor that will not document how its figures are produced is useful once and dangerous after that, and on the day its conclusions are challenged in a board meeting nobody in the room will be able to defend them.

### Why is buyer inertia such a real factor in this category?

Procurement teams routinely underestimate the cost of switching providers, because the visible cost is the contract while the hidden cost is retraining the analysts, rebuilding the dashboards and re-establishing the historical baseline against which every past report was written. Industry buyers in beauty and FMCG describe the same pattern, with real appetite for new data sitting alongside real reluctance to change the provider that supplies it. The practical implication is that a marginal improvement in data quality rarely justifies a switch, and that it usually takes a clear gap in scope, such as a platform the incumbent does not cover, or a specific and repeated problem with the incumbent's figures, which is precisely what trial periods exist to surface.

### What does a credible methodology page actually look like?

Credible methodology documentation can be told apart from marketing copy in a few ways. It names what is *excluded*, since every honest dataset has gaps and a page that mentions none of them is not being honest about the rest. It specifies the method *per platform* and per data type rather than in aggregate, because coverage, history and cadence differ from one platform to the next and the differences are what a buyer needs to know. And it explains how estimates are produced wherever a figure cannot be observed directly, for instance how marketplace rankings or review counts are converted into sales-volume estimates, so that a reader can tell an observed number from an inferred one. A methodology page that does all of this is rarer than it should be.

### What does an honest scoping conversation with a provider sound like?

An honest provider will tell you which categories and platforms it is strongest in and which it is not, and will say so plainly when its fit for your question is poor, even if that means recommending you look elsewhere. It will be specific about coverage limits in your markets, for example how much of prestige beauty it can see when part of that tier sells offline, instead of offering a blanket claim of "Asia coverage", and it will show you sample data for *your* category, on the platforms you sell through, before any deep meeting. The shortcut signal is to ask where the provider would not recommend itself, because silence or deflection is an answer in its own right, and the buyers who run procurement well treat that question as the most informative one in the cycle.