Tagging stock-market impact in the news feed — a working study

Status: in progress. Window studied: Jul 10–16, 2026 (one week). Author: automated markets-desk analysis over the HuggingNews corpus.

The goal

Most news does not move stocks. A minority does — and some of the biggest movers are not the stories that look like "business news." We want to tag, at publish time, which stories have likely stock-market impact, how big, which tickers, and in what direction, so that:

1. a model/agent can flag impact when a story is published (like Kimi K3 today), before the reaction arrives;
2. likely-impact stories rank higher (a routine earnings miss is not the same as IBM read as "AI eating its business");
3. the impact can inform the writing (a speculative market read, kept separate from the facts).

Two things must not be conflated:

• Hindsight labeling — using the cluster's reaction tweets, later versions, and quoted price moves to establish what actually moved. This is how we build ground truth.
• Publish-time prediction — inferring impact from the headline + facts alone, before any reaction. This is the actual product, and the harder task. We measure the gap between the two.

We deliberately over-tag rather than miss: better to flag a possible mover and be wrong than to only notice impact after it is obvious. very_unlikely is a valid, common label.

Method

Corpus. 1,908 distinct story-clusters over the week (deduped to the strongest snapshot per cluster). 1,206 are business-adjacent by tag.

Candidate pre-filter (tools/market_impact/build_candidates.py). An inclusive ranker: business-adjacent tags ∪ a market_anchor-style keyword net ∪ crowd signal (author count). It ranks, never drops — because the highest-impact story of the week (Kimi K3) contains no ticker or earnings verb and would be filtered out by any keyword gate. Emits per-candidate JSON (title, tags, counts, summary, cluster-tweet sample) for the judge.

Judge (tools/market_impact/market_impact_v2.txt). An LLM markets-desk analyst returns a structured object per story: market_impact (bool), magnitude (major/moderate/small/very_unlikely), score (0–100), tickers (with up/down/mixed), mechanism (one line), publish_time_signal, confidence. Run as a fan-out (one agent per story).

Magnitude calibration.

• major — moves an index/whole sector, a mega-cap a lot, or shifts a market-wide thesis. (Hormuz oil shock; IBM −25%; an open-weight frontier model / "DeepSeek moment"; a Fed policy-path shift.)
• moderate — a clear single-name or sub-sector move (earnings beat/miss, an M&A bid, a sector supply story).
• small — own stock only, little spillover.
• very_unlikely — no real move. Routine product/feature/model announcements from Google, Apple, OpenAI, Anthropic, Meta default here unless there's a specific tradable hook.

Proxies. OpenAI and Anthropic are private but pivotal → mapped to MSFT, GOOGL, AMZN, NVDA. Crypto counts (BTC, ETH, SOL, COIN, MSTR). Diffuse impact leans on ETFs (SPY, QQQ, SOXX/SMH, XLE, JETS, XLF, XLV, IGV, EWZ, KWEB).

Finding 1 — a keyword pre-filter misses the biggest story

The Kimi K3 launch — an open-weight, ~frontier, Chinese model — did not crack the top 55 candidates by keyword pre-score, because its market impact is entirely second-order ("open-weight frontier model" has no ticker, no earnings verb). Meanwhile the pre-filter over-ranks routine single-name earnings (GE Aerospace, Abbott, U.S. Bancorp) that move their own stock and nothing else. Conclusion: a heuristic is a fine inclusive ranker but cannot make the magnitude/ticker call; that needs a reasoning judge, and the pre-filter must never drop.

Finding 2 — the judge's first version imported the day's loudest theme (cluster-bleed)

Judge v1 produced good second-order reasoning (IBM→IGV/MSFT down + SMH up; ASML→SOXX/NVDA up; China clears H200→NVDA/BABA up; DeepSeek/Kimi→NVDA & closed-lab proxies down). But it had two systematic errors:

(a) Cluster-narrative bleed. 32 of 41 "major" tags were the oil/Hormuz complex, and several were flat wrong — a CPI print and a Bitcoin level were tagged as major oil-supply shocks purely because their cluster tweets mentioned Iran; a strike on a Kurdish opposition site got a full oil-shock major. The judge scored the day's loudest theme instead of the story's own event.

(b) The flagship under-rated. Kimi K3 scored moderate/42 while ~20 near-identical war stories scored major/88. The judge over-weighted the loud war cluster and under-weighted the structurally biggest AI story of the week. Direction was right; magnitude was wrong — and magnitude is the point.

Finding 3 — v2 fixes both, validated head-to-head

Judge v2 adds two rules: (1) score only this story's own event; cluster tweets are reaction-evidence for that event, not a license to import an unrelated theme. (2) magnitude is about size of market impact, which can be structural — an open-weight frontier model is major even if its cluster is quieter than a war cluster ("loud cluster ≠ big impact").

Head-to-head on a 25-story validation set:

Bonus distinction v2 surfaced: it now separates a capability release (moves markets → major) from financing/valuation news about the same lab (doesn't → small/none). DeepSeek's $7.4B raise dropped to very_unlikely, its IPO filing to small, the Korrun valuation mark to very_unlikely. The model ships the move, not the funding round.

Two borderline v2 calls to watch (both slightly conservative): TSMC's $100B Arizona + guidance raise slipped major→moderate (arguably still major on the AI-capex read); Meta's "Muse Spark beats GPT/Gemini" went to very_unlikely (defensible under the over-announce rule, maybe a touch low).

Full-run results (canonical, v3 over 300 candidates)

Distribution (v3, 300): major 38, moderate 40, small 50, very_unlikely 172 (57%) — tighter than v2 (very_unlikely 138) after the single-name-earnings→small and war-rhetoric/funding discounts. 68 distinct market events after theme-dedup, written to data/market_impact.jsonl. (v2's full-300 distribution was major 45 / moderate 76 / small 41 / very_unlikely 138 — the arc in Findings 1–7 uses v2 for the story; v3 is the canonical label set.)

Selected distinct movers (v2 full-300 hindsight, illustrative of the deduped output):

Major:

• Iran/Hormuz oil-supply shock (×19 near-dupes) — XLE↑ USO↑ JETS↓ SPY↓ XOM↑; the one genuinely index-moving geopolitical thread.
• IBM −25% — IBM↓ IGV↓ MSFT↓ SMH↑ QQQ↓ MU↑ (mega-cap crash + "AI eating services" drag, memory read-through).
• ASML beat + twice-raised guide (×14) — ASML↑ SOXX↑ NVDA↑ TSM↑ SMH↑ (AI-capex confirmation).
• Moonshot Kimi K3 (×4) — NVDA↓ MSFT↓ GOOGL↓ AMZN↓ SOXX↓ (the DeepSeek-moment thesis; correctly major in v2).
• Korea Exchange halt / KOSPI — EWY MU SOXX SMH (AI-memory unwind).
• Fed's Waller / 2Y yield (×8) — TLT↓ XLF↑ SPY↓ (rate-path shift, correctly not the oil theme).
• Fed Chair Warsh: AI funding may dry up — QQQ↓ SOXX↓ NVDA↓ MU↓ (an authority questioning AI-capex returns — a genuine second-order find).
• Russia diesel export ban — VLO↑ MPC↑ PSX↑ XLE↑ JETS↓ (refined-product tightness).

Notable inverse/second-order calls the judge got right: UAE OPEC +80% output → USO down (supply up = price down); China clears H200 for Alibaba/ByteDance/DeepSeek → NVDA/BABA up; DeepSeek $7.4B raise → NVDA down but treated as small (financing, not a release).

Reality validation

Where a cluster tweet quotes an actual %-move for the same instrument the judge tagged (reality_check.py): 95% direction agreement (18/19). The 2 misses are genuinely ambiguous stories — the KOSPI had both a 9% crash and an 8.2% rebound in-window, and United Airlines both raised guidance and warned of a $6B hit. A naive first version of the check reported false mismatches because it compared quoted moves to inverse/proxy tickers (yield↑ vs TLT, output↑ vs oil price, a lab's valuation vs NVDA) — those are the exact second-order relationships the judge is supposed to capture, and it got them right.

Finding 4 — coverage: the first pre-filter dropped real movers; fixed

At a 300-candidate cap, the first pre-score dropped genuine movers into the tail — most glaringly SK Hynix's $26.5B record U.S. listing (pre-score 3.9), plus Meta's first paid AI API (25% of rivals) and the Korea ETF-regulation order. The tail was otherwise dominated by duplicative war updates and OpenAI product-noise (correctly low-impact), so the miss rate on actual movers was modest but non-zero.

Fix (in build_candidates.py): boost the signals a keyword net under-weights — dollar magnitude ($Nbn/$Ntrillion), "record", "listing/IPO/debut", pricing hooks ("priced at", "at N% of", "paid API"), open-weight/frontier/"beats GPT/Claude", and market-structure terms ("circuit breaker", "plunge/crash") — and damp pure-conflict updates (strike/missile/casualty) that carry no business tag. After the fix:

Takeaway: the pre-filter is a ranker, and its ranking quality directly bounds recall at a fixed cap. Damping the war-update duplication frees cap slots for genuine movers.

Publish-time vs hindsight gap

[pending — the publish-time judge sees headline+summary only; we measure which movers are only catchable after the reaction]

Reality validation

[pending — where a cluster tweet quotes an actual % move, does the judge's direction agree?]

Finding 5 — calibration (full 300 hindsight)

Distribution: major 45, moderate 76, small 41, very_unlikely 138 (46%). Nearly half of business-adjacent candidates are correctly judged non-movers.

Over-coverage is correctly killed. Every story in the "Google/Apple/OpenAI/Anthropic/Meta over-announces" bucket landed very_unlikely: GPT-5.6/Grok/Muse Spark benchmark-leapfrogging, "Copilot picks GPT-5.6", Anthropic's $2.5B credit facility and $965B IPO-meeting scheduling. This was the predicted first-pass failure — the judge does not fall for it.

Not over-tagging earnings, but a v3 refinement exists. Routine single-name beats (GE Aero, Abbott, J&J, U.S. Bancorp, BASF, BlackRock) land at moderate but score 40–48 (the low edge). Refinement for v3: a pure single-name earnings move with no genuine sector spillover should be small, reserving moderate for real read-through (UNH→XLV, Delta→JETS, TSMC→SOXX are justified moderates).

Score bands separate cleanly (monotonic, minimal overlap), so the 0–100 score is a usable continuous ranking signal on its own:

Finding 6 — publish-time vs hindsight gap (the product-critical result)

Running the same judge with only the headline + summary (no reaction tweets) vs the hindsight judge:

• 95% of hindsight movers (major/moderate) are catchable at publish time (115/121). The impact is mostly inferable from the facts alone; the reaction tweets rarely change the magnitude. Exact magnitude agreement across all 300: 85%.
• 6 movers are missed at publish — the genuinely subtle cases that only look material after the reaction: Morgan Stanley's spot ETH/SOL ETF filing, Apple targeting AI-chip startups, Robinhood Chain DEX volume >$1B, SpaceX below its IPO price, Rio Tinto's copper-smelter outage, Crimea fuel prices. This is the training hard-list.
• The important asymmetry: publish-time OVER-tags (14 cases). Without the reaction, the judge over-rates two specific patterns that hindsight correctly kills:
  1. War rhetoric — "Trump Orders 1,000 Missiles at Iran" reads major/78 at publish but very_unlikely in hindsight (rhetoric; oil didn't actually move; already priced). Scary geopolitical headlines look major at publish and mostly aren't.
  2. Headline funding/IPO size — Anthropic's $965B IPO-meeting scheduling (P:major/72 → H:very_unlikely), DeepSeek's $7.4B raise and MiniMax's $2B round (P:moderate → H:small). The capability-release-vs-financing distinction is harder to make without seeing that the market shrugged.

Interpretation. The desired bias (over-tag rather than miss) holds at publish time, and the gap names exactly what a publish-time predictor must learn to discount: war rhetoric and raw funding/IPO numbers. Those two corrections would close most of the publish↔hindsight gap. Everything else the judge already reads correctly from the headline.

Finding 7 — v3 refinements validated

v3 adds two rules and was validated on the 17 stories that exposed them:

• RULE 3 — single-name earnings with no sector spillover is small, not moderate. All 7 pure single-name beats dropped moderate→small (GE Aero, Abbott, J&J, U.S. Bancorp, BASF, BlackRock, Morgan Stanley), while every genuine sector-spillover control held at moderate (UnitedHealth→XLV, TSMC→SOXX, Delta→JETS). One borderline: ASML slid major→moderate — defensible either way, as a bellwether beat-and-raise sits on the major/moderate line.
• RULE 4 — discount war rhetoric and headline funding/IPO size. "Trump Orders 1,000 Missiles at Iran" dropped to very_unlikely; DeepSeek's $7.4B raise dropped to very_unlikely; MiniMax's $2B round and Anthropic's $965B IPO meetings stayed very_unlikely. These are the exact publish-time over-tag patterns Finding 6 identified.

v3 is the default going forward. (market_impact_v3.txt.) The canonical labeled set is regenerated with v3.

Recommendations for integration

1. Ranking. Use the 0–100 score (cleanly separated by band) as a ranking boost, not the magnitude label — e.g. add a bounded bump so a major/major-scored story outranks a same-engagement non-mover. IBM (score 82) should clear a routine beat; a war-casualty update (very_unlikely) gets no bump. Cap the bump so it re-ranks within, not over, the core relevance signal.
2. Story text — a separate, hedged "market read." Because impact is speculation, keep it out of the fact body. Emit one optional line tagged as a read, e.g. "Market read: likely pressure on NVDA and the closed-lab proxies (MSFT/GOOGL) — a DeepSeek-style open-weight threat." Sourced from mechanism + tickers. Never state a move as fact; never put it in the RULE-ZERO fact body.
3. Publish-time tagging on ingest. Run market_impact_publishtime.txt on each new cluster at ingest, store {magnitude, score, tickers, mechanism} alongside the story. Apply the two learned discounts (war rhetoric, funding/IPO size) as guardrails so the live tagger inherits the hindsight correction without needing the reaction.
4. Feedback loop. Periodically re-judge older stories in hindsight and diff against their publish-time tags to keep measuring (and shrinking) the gap — the 6-missed / 14-over-tagged lists are the labeled examples to tune the next prompt or a distilled model.

Operational note

Running two ~300-agent judge workflows concurrently triggered server-side rate-limiting that failed ~96/300 and ~183/300. A single big workflow runs fine. Serialize large fan-outs. Recovery: keep the completed subset, compute missing ids, re-run only those as a single follow-up (done here — all 600 judgments completed across two serialized fill passes).

Reusable pipeline (tools/market_impact/)

• build_candidates.py — inclusive pre-filter/ranker over a date window → per-candidate JSON.
• market_impact_v2.txt — hindsight judge (sees reaction tweets).
• market_impact_publishtime.txt — publish-time judge (headline+summary only).
• aggregate.py — theme-dedup + data/market_impact.jsonl + ranked report.
• reality_check.py — direction-agreement vs quoted moves.