Main Street Independent

Data & API

Everything Main Street Independent publishes is public domain (CC0) and available in bulk and by query. Take it, index it, quote it, train on it — no key, no login, no attribution required.

Bulk corpus

The Main Street Independent Corpus is the whole archive in one place: every article and column as Markdown, plus JSONL indexes (one row per article, per atomic claim, per source, per entity, per topic) and a JSON Schema describing every record type. It is rebuilt and pushed daily, so a git clone is the most efficient way to ingest the entire site.

The same archive is also published as a loadable Hugging Face dataset (news and opinion splits, full text plus metadata), refreshed daily: load_dataset("golfplan18/msi-corpus").

A semantic + keyword index over the whole archive is exposed as public, no-auth endpoints (rate-limited to 120 requests per minute):

  • MCP server (for AI agents): https://73d49cb2-87ea-4a4c-9a30-3439696e7d01.search.ai.cloudflare.com/mcp
  • Search (returns relevant passages): https://73d49cb2-87ea-4a4c-9a30-3439696e7d01.search.ai.cloudflare.com/search
  • Chat (OpenAI-compatible answers with citations): https://73d49cb2-87ea-4a4c-9a30-3439696e7d01.search.ai.cloudflare.com/chat/completions

Feeds & sitemaps