Data dictionary v1 · schema 2026-09
ConventionsCoverageCompany metricsConsumer trendsSearch interestTrial vs subscription
Conventions
- Key.
entity_id(formatTICKER#slug) joins each division;company_idis the company key across datasets — STABLE across ticker renames and M&A, so key backtests by it (resolve a ticker or permID via/resolve). Company-level values are the sum of a company's entities; brand-level values (subscription) roll up to the entity. - Date.
dateis an America/New_York calendar day (ISOYYYY-MM-DD). - Shape. Tidy-long: one row per key × date × metric.
- Values. Counts are whole numbers of posts.
- Encoding. UTF-8 CSV with header, or JSON from the API.
format=parquetandformat=arrowreturn a time-limited download URL for bulk pulls — available on trial too, through the rolling six-month lag. - Provenance. Only derived counts are shipped: no post text, ids, user objects, or third-party raw data.
Coverage
One row per entity. An entity is one company's consumer business in one category (Apple's devices, Match Group's dating apps). 659 entities across 497 tickers, built from 4,579 brands and products.
| Column | Type | Meaning |
|---|---|---|
entity_id | string | Per-division / entity key. Example: AAPL#consumer-electronics-dev. |
company_id | string | LikeFolio company id (tied to Refinitiv/LSEG permID) — the stable primary key, unchanged across ticker renames, re-listings and M&A. Key everything by this. Resolve a ticker or permID to it via /resolve. |
perm_id | string | This company's Refinitiv/LSEG PermID (a permanent cross-vendor id). Blank if permid.org hasn't issued one for this entity. |
ticker | string | Current listed symbol — a mutable, point-in-time alias (join to your own price and fundamentals as-of a date), not a key. The universe carries dated ticker history; delisted symbols keep their last symbol. |
company | string | Company name. |
division | string | Consumer category the entity rolls up to. |
brand_label | string | Set when the entity is a single acquired brand carved into its own division. |
valid_from / valid_to | date | Ownership window. Blank valid_from means it was owned since the start of the data; blank valid_to means it is still currently owned. Set on 130 entities (acquired, divested, renamed). |
core | bool | False for non-core entities (excluded from ticker roll-ups). |
Company metric series
Daily counts of English public posts on X naming the entity's brands, that also demonstrate that metric with natural language. Retweets, promotional, and spam posts are excluded. History from 2021-01-01. Refreshed daily. Served by GET /company/{companyId}/metrics — every signal is a metric you pull with metric=; there is no separate "demand" endpoint. Key rows by company_id, not by the mutable ticker (resolve a ticker via /resolve).
| Column | Type | Meaning |
|---|---|---|
entity_id | string | The division this row belongs to. Example: AAPL#consumer-electronics-dev. |
company_id | string | LikeFolio company id (tied to Refinitiv/LSEG permID) — the stable key, unchanged across ticker renames, re-listings and M&A. Key everything by this. Resolve a ticker or permID to it via /resolve. |
perm_id | string | This company's Refinitiv/LSEG PermID (a permanent cross-vendor id). Blank if permid.org hasn't issued one for this entity. |
ticker | string | Ticker as of this row's knowledge date. A mutable, point-in-time alias — not a key. |
division | string | Human division label; present on rollup=entity rows. |
date | date | Eastern calendar day (the event date; bound with start/end). |
metric | enum | One of the 16 metrics below. |
raw | int | Posts counted that day, as pulled. |
corrected | int | Raw with the chosen correct= categories replaced by a cleaned count, as whole numbers. Equal to raw where nothing was corrected — a blank flag, a demand_event day, or a category you excluded. Use this column by default. |
flag | enum | Blank, or the editorial verdict on a reviewed spike: one of meme, incident, marketing, investor, demand_event. By default every category except demand_event is corrected (replaced in the corrected column); demand_event is kept as genuine demand. You choose the set with the correct= parameter; the flag tells you which category each day belongs to. Bots and spam are removed upstream and never appear as a flag. |
kdate_from / kdate_to | date | Bitemporal window (present when kdate=all): the knowledge dates over which this value was current. kdate_to null = still current. |
change_reason | enum | Why this row supersedes the prior value (present when kdate=all): as-pulled, spike-reclass, repull, negation, methodology, mna. |
date axis is the event day. The separate knowledge-date axis is set with the kdate query param: it returns values as LikeFolio knew them on that Eastern day (no look-ahead through restatements or M&A), and kdate=all exposes the full bitemporal history via the three columns above.Metrics
| metric | What it counts | Tickers with meaningful data* |
|---|---|---|
mentions | Every post naming the brand or its products. The headline demand series. | 482 |
pi | Purchase intent: bought, buying, ordered, want, need, signed up. | 335 |
ps / ns | Positive / negative sentiment. Share positive = ps / (ps + ns). | 401 / 294 |
npi | Negative purchase intent: cancelling, returning, switching away, never again. | 137 |
switch_in / switch_out | Switching to / away from the brand. | 13 / 17 |
promo | Deals, coupons, sales, discount language. The "is demand bought or organic" flag. | 246 |
political | Boycott and political framing. Read as a rate over mentions. | 266 |
price_value / price_pain | "Worth it" vs "too expensive". | 161 / 141 |
defect | Product problems, broke, doesn't work. | 155 |
outage | Service down, can't log in. | 156 |
safety | Recalls, injuries, contamination. | 116 |
labor | Strikes, layoffs, unions. | 72 |
availability | Out of stock, sold out, back in stock. | 61 |
*Covered = averages ≥25 posts/week over the trailing year, or spikes to ≥20% of that day's mentions on at least one day (≥10 posts that day) — so a metric that's quiet most days but carries real signal in bursts still counts. Switching and availability are genuinely thin — meaningful on only a few dozen names.
Consumer trends
141 consumer behaviors ("cutting back on spending", "taking a road trip", "cancelling a streaming service"), each a hand-built phrase rule counting people describing doing the behavior. Daily since 2016-09-01, refreshed daily, spike-checked and corrected like the company series.
| Table | Columns | Notes |
|---|---|---|
trend_catalog | trend_id, name, group, mapped_tickers, mapped_datasets | Names and the trend→company and trend→dataset maps are available at every tier. group is one of 15 themes (Health & Fitness, Eating & Drinking, Investing/Crypto/Gambling, …); mapped_tickers / mapped_datasets count the companies and public-data series each trend is mapped to. |
trend_series | trend_id, date, raw, corrected, flag (plus kdate_from, kdate_to, change_reason when kdate=all) | Same semantics as the company metric series, including the kdate knowledge-date axis. |
trend_ticker_map | trend_id, ticker, side, weight, added_at, removed_at | Which companies a trend should move, in which direction (side = long / short), and a suggested weight (0–1) for how strongly the trend bears on that company. Dated membership so backtests use the map as it stood. |
Search interest final review
The search-interest dataset is in final quality review. Schema and coverage are stable, but published data is going through final quality control.
Brand-level Google search interest, daily, US, from 2020-09-01. Google scores each request 0-100 in isolation; tie all scores together so all brands sit on one global scale where the smallest covered brand averages roughly 50 and large brands read in the thousands. Any two brands on any two dates are always comparable. Refreshed daily. We map 9,508 brands to the covered companies. Of these, 6,034 return a usable 2026 Google signal (2,855 daily-served, 3,179 at weekly resolution); the remaining 3,474 are dark — too small or defunct to register at any resolution — so they are not published as flat-zero noise.
| Column | Type | Meaning |
|---|---|---|
brand_id | string | Trial: TICKER#b1… ordered by volume. Subscription: the brand name plus entity_id. |
ticker | string | Owner on that date (point-in-time). |
date | date | Calendar day. |
interest | number | Unbounded global index, comparable across brands and dates. |
What differs between trial and subscription
| Field | Trial | Subscription |
|---|---|---|
brand_label | withheld | included |
brand_id (search) | TICKER#bN | brand name |
| Latest date | 6-month lag (rolling) | latest refresh |
| Values | identical | |
Trial ids are stable across refreshes, so a model built on the trial transfers to the subscription by a one-time id join we supply at signing.