Living CI Document: Competitive Intelligence That Updates Itself

A CI document goes stale the moment it’s created. A competitor changes their pricing, ships a feature, rewrites their positioning. The document sits there unchanged. The team makes decisions on month-old data.

The problem isn’t analysis quality. It’s the model: a manually maintained document is structurally static. Keeping it current requires an analyst’s time, motivation, and discipline. In practice, updates happen before strategic planning sessions — once a quarter, if you’re lucky.

This article describes the architecture of a CI document that updates itself automatically. Source monitoring, AI-powered change analysis, section updates, team notifications — all without manual work after the initial setup.

Architecture of a Self-Updating CI Document

The system has four layers, each solving one problem.

Data Sources
    │
    │  RSS, webhooks, scraping, APIs
    ▼
┌──────────────────┐
│  Orchestrator    │  n8n / Make
│  (automation)    │  Schedules + triggers
└──────────────────┘
    │
    │  Raw data
    ▼
┌──────────────────┐
│  AI Processor    │  LLM: analysis, comparison,
│  (analysis)      │  update generation
└──────────────────┘
    │
    │  Structured updates
    ▼
┌──────────────────┐
│  Document        │  Notion / Google Docs /
│  (storage)       │  Markdown in a repo
└──────────────────┘
    │
    │  Change digest
    ▼
Slack / Email / Telegram

The orchestrator manages the schedule and data flow. n8n (self-hosted) or Make (cloud) both work. The choice depends on infrastructure: n8n gives full control and doesn’t charge per operation; Make is faster to set up.

The AI processor receives raw data and the current state of a document section. It returns the updated section text and a changelog. It runs via the API of any LLM provider.

The document stores structured information about competitors. Each section carries metadata: last-updated date, source, confidence level.

CI Document Structure: Sections and Metadata

The document is split into sections. Each updates independently, on its own schedule, from its own sources.

Minimum section set:

Section	Update Frequency	Sources
Pricing & Plans	Weekly	Pricing pages, API
Product Features	Weekly	Changelog, blog, release notes
Positioning & Messaging	Twice a month	Homepage, landing pages
Team & Hiring	Monthly	LinkedIn, careers pages
Funding & Financials	Monthly	Crunchbase, press releases
Tech Stack	Monthly	Job postings, BuiltWith, Wappalyzer
Content & SEO	Weekly	Blog, social media, Ahrefs API
Customer Reviews	Twice a month	G2, Capterra, Product Hunt

Each section stores metadata:

{
  "section": "pricing",
  "competitor": "CompetitorX",
  "last_updated": "2026-03-27T10:00:00Z",
  "update_source": "https://competitorx.com/pricing",
  "confidence": "high",
  "change_detected": true,
  "previous_hash": "a3f2b1c...",
  "changelog": [
    {
      "date": "2026-03-27",
      "change": "Pro plan price increased from $49 to $59/mo",
      "significance": "high"
    }
  ]
}

The previous_hash field stores a hash of the previous source page version. Comparing hashes tells you if anything changed before you call the LLM — no change, no API call.

Setting Up Source Monitoring in n8n

Examples use n8n because it runs locally and has no operation limits. Adapting for Make means swapping nodes; the logic stays the same.

Workflow 1: Pricing Page Monitoring

[Cron: every Monday at 09:00]
    │
    ▼
[HTTP Request: GET competitor.com/pricing]
    │
    ▼
[Code: extract text, compute hash]
    │
    ▼
[IF: hash ≠ previous_hash]
    │         │
    YES       NO → [Stop]
    │
    ▼
[OpenAI / Anthropic: analyze changes]
    │
    ▼
[Notion API: update document section]
    │
    ▼
[Slack: change notification]

The key node — a Code node for text extraction and hashing:

const crypto = require('crypto');

// Input: page HTML
const html = $input.first().json.data;

// Strip tags, scripts, styles
const text = html
  .replace(/<script[^>]*>[\s\S]*?<\/script>/gi, '')
  .replace(/<style[^>]*>[\s\S]*?<\/style>/gi, '')
  .replace(/<[^>]+>/g, ' ')
  .replace(/\s+/g, ' ')
  .trim();

// Hash for comparison
const hash = crypto
  .createHash('md5')
  .update(text)
  .digest('hex');

return [{
  json: {
    text,
    hash,
    url: $input.first().json.url,
    timestamp: new Date().toISOString()
  }
}];

The hash is checked against the stored value from the last run. Match — workflow stops. Difference — data goes to the AI processor.

Workflow 2: Changelog and Blog Monitoring via RSS

RSS is simpler than scraping, and most SaaS companies publish their changelog and blog via RSS.

[RSS Feed Trigger: competitor.com/blog/rss]
    │
    ▼
[Filter: new entries from the past 7 days]
    │
    ▼
[HTTP Request: fetch full text of each entry]
    │
    ▼
[AI: classify — product update / marketing / hiring / other]
    │
    ▼
[Switch: by content type]
    │         │          │
  Product   Marketing   Other → [Skip]
    │         │
    ▼         ▼
[Update       [Update
 Features      Positioning
 section]      section]

Classifying content before updating the document is critical. Without it, marketing posts end up in product updates and the document loses its value.

Prompts for AI Section Updates

A bad prompt generates a summary. A good one extracts meaningful changes and updates the section while preserving the history of previous updates.

Prompt: Updating the Pricing Section

You are a competitive intelligence analyst. Task: update the Pricing section
in the CI document based on new data.

CURRENT DOCUMENT SECTION:
---
{current_section_content}
---

NEW DATA (pricing page text):
---
{new_pricing_page_text}
---

INSTRUCTIONS:
1. Compare the new data against the current section
2. Identify specific changes (prices, plans, limits, terms)
3. Update the section, preserving format and change history
4. Add a changelog entry with date and change description
5. Rate significance: high (price change >10%, new/removed plan),
   medium (limit or term changes), low (cosmetic edits)

RESPONSE FORMAT (strict JSON):
{
  "updated_section": "...",
  "changes": [
    {
      "change": "change description",
      "significance": "high|medium|low",
      "implication": "what this means for us"
    }
  ],
  "has_significant_changes": true/false
}

Prompt: Updating the Product Features Section

You are a competitive intelligence analyst. Task: update the Product Features section
based on new content (changelog, release notes, blog post).

CURRENT SECTION:
---
{current_section_content}
---

NEW CONTENT:
---
{new_content}
---

CONTENT TYPE: {content_type} (changelog / release_notes / blog_post)

INSTRUCTIONS:
1. Extract specific product changes (new features, improvements, deprecations)
2. Ignore marketing language — facts only
3. Categorize: new_feature / improvement / deprecation / integration / api_change
4. Update the feature matrix in the section
5. Flag features the competitor added that we don't have (gap analysis)

RESPONSE FORMAT (strict JSON):
{
  "updated_section": "...",
  "new_features": [
    {
      "feature": "name",
      "category": "new_feature|improvement|deprecation|integration|api_change",
      "description": "brief description",
      "we_have_equivalent": true/false,
      "competitive_impact": "high|medium|low"
    }
  ]
}

Prompt: Updating the Positioning & Messaging Section

You are a competitive intelligence analyst. Task: track changes
in a competitor's positioning.

PREVIOUS VERSION OF HOMEPAGE:
---
{previous_homepage_text}
---

CURRENT VERSION:
---
{current_homepage_text}
---

CURRENT DOCUMENT SECTION:
---
{current_section_content}
---

INSTRUCTIONS:
1. Compare the texts. Identify changes in:
   - Headline and value proposition
   - Target audience (who is named, who is addressed)
   - Key benefits and selling points
   - Social proof (customers, metrics, testimonials)
   - Call-to-action (copy, placement)
2. Assess: cosmetic change or a strategic positioning shift
3. Update the document section

RESPONSE FORMAT (strict JSON):
{
  "updated_section": "...",
  "positioning_changes": [
    {
      "element": "headline|audience|benefits|social_proof|cta",
      "previous": "what it was",
      "current": "what it is now",
      "interpretation": "what this means"
    }
  ],
  "strategic_shift": true/false,
  "shift_description": "description of shift, if any"
}

All three prompts share the same structure: current section state + new data + clear instructions + structured response format. That pattern is covered in detail in the context engineering guide.

Programmatic Document Updates via API

The document can live in Notion, Google Docs, or as a Markdown file in a git repo. Each has its own API for programmatic updates.

Notion: Updating a Block

// n8n Code node: update a section in Notion
const notionApiKey = $env.NOTION_API_KEY;
const pageId = 'YOUR_PAGE_ID';
const blockId = 'PRICING_SECTION_BLOCK_ID';

const aiResponse = JSON.parse($input.first().json.ai_response);

// Update block content
await $http.request({
  method: 'PATCH',
  url: `https://api.notion.com/v1/blocks/${blockId}/children`,
  headers: {
    'Authorization': `Bearer ${notionApiKey}`,
    'Notion-Version': '2025-09-03'
  },
  body: {
    children: [
      {
        type: 'paragraph',
        paragraph: {
          rich_text: [{
            type: 'text',
            text: { content: aiResponse.updated_section }
          }]
        }
      }
    ]
  }
});

// Update metadata (last_updated, source)
await $http.request({
  method: 'PATCH',
  url: `https://api.notion.com/v1/pages/${pageId}`,
  headers: {
    'Authorization': `Bearer ${notionApiKey}`,
    'Notion-Version': '2025-09-03'
  },
  body: {
    properties: {
      'Last Updated': {
        date: { start: new Date().toISOString() }
      }
    }
  }
});

return [{ json: { status: 'updated', changes: aiResponse.changes } }];

Git Repository: Committing via API

For teams that prefer Markdown and version control:

// n8n Code node: update Markdown in GitHub
const token = $env.GITHUB_TOKEN;
const owner = 'your-org';
const repo = 'competitive-intel';
const path = `competitors/${competitor}/pricing.md`;

const aiResponse = JSON.parse($input.first().json.ai_response);

// Get current file (sha is needed for updates)
const current = await $http.request({
  method: 'GET',
  url: `https://api.github.com/repos/${owner}/${repo}/contents/${path}`,
  headers: { 'Authorization': `Bearer ${token}` }
});

// Update the file
await $http.request({
  method: 'PUT',
  url: `https://api.github.com/repos/${owner}/${repo}/contents/${path}`,
  headers: { 'Authorization': `Bearer ${token}` },
  body: {
    message: `ci: update ${competitor} pricing — ${aiResponse.changes[0]?.change || 'routine update'}`,
    content: Buffer.from(aiResponse.updated_section).toString('base64'),
    sha: current.sha
  }
});

The git approach gives you free change history, diffs between versions, and the ability to review changes via pull requests.

Error Handling and Data Validation

An automated system without error handling produces garbage. Three layers of protection.

Layer 1: source validation. Before sending data to AI, check that the source returned expected content. An empty page, 404, CAPTCHA, or paywall — catch it before the LLM call.

// Validation before AI processing
const text = $input.first().json.text;
const url = $input.first().json.url;

// Check for empty or too-short content
if (!text || text.length < 100) {
  return [{ json: { skip: true, reason: 'empty_or_too_short' } }];
}

// Check for CAPTCHA / blocking
const blockedIndicators = ['captcha', 'access denied', 'please verify'];
const isBlocked = blockedIndicators.some(
  indicator => text.toLowerCase().includes(indicator)
);
if (isBlocked) {
  return [{ json: { skip: true, reason: 'blocked', url } }];
}

return $input.all();

Layer 2: AI response validation. The LLM may return invalid JSON, a hallucination, or an empty response. Parse the JSON, check required fields, run sanity checks — a 1000% price change is probably an error.

Layer 3: human review for high-significance changes. Strategic positioning shifts, major price changes, launch of a fundamentally new product — all of these go to review before being written to the document.

[AI: analyze changes]
    │
    ▼
[Switch: significance]
    │              │
  high            medium/low
    │              │
    ▼              ▼
[Slack:           [Automatic
 request           document
 confirmation]     update]
    │
    ▼
[Wait for
 approval]
    │
    ▼
[Update document]

Digests and Notifications

An unfiltered stream of notifications is noise. Two delivery formats fix this.

Instant alerts — only for high-significance changes. A competitor cut their price by 30%. They shipped a direct equivalent of a key feature. They closed a funding round.

Weekly digest — a summary of all changes over the week, grouped by competitor and section.

Prompt for generating the digest:

Generate a weekly competitive intelligence digest.

CHANGES FOR THE WEEK:
---
{all_changes_json}
---

INSTRUCTIONS:
1. Group by competitor
2. Within each competitor — sort by significance (high → low)
3. For high-significance changes, add a recommendation: what the team should do
4. Open with an executive summary (3–5 sentences, the most important things this week)
5. Close with "No changes" for competitors where nothing happened

FORMAT: Markdown suitable for a Slack message.
Maximum 500 words.

Scaling: From 3 to 30 Competitors

A system for 3 competitors and a system for 30 look different architecturally.

Up to 5 competitors. One workflow per section. Competitors listed in an array inside the workflow. LLM calls run sequentially. Make’s free tier or a single n8n instance is enough.

5–15 competitors. Configuration moves to a dedicated store — Notion database, Airtable, JSON in S3. The workflow reads the config and iterates over competitors. Parallel LLM calls, rate limiting at the orchestrator level.

15+ competitors. Prioritization. Not all competitors matter equally. Tier 1 (direct) monitored weekly. Tier 2 (indirect) every two weeks. Tier 3 (potential) monthly. The config stores each competitor’s tier; the workflow filters by schedule.

Competitor configuration:

{
  "name": "CompetitorX",
  "tier": 1,
  "sources": {
    "pricing_url": "https://competitorx.com/pricing",
    "blog_rss": "https://competitorx.com/blog/rss.xml",
    "changelog_url": "https://competitorx.com/changelog",
    "careers_url": "https://competitorx.com/careers",
    "linkedin": "https://linkedin.com/company/competitorx",
    "g2_url": "https://g2.com/products/competitorx/reviews"
  },
  "sections": ["pricing", "features", "positioning", "hiring", "reviews"],
  "notification_channel": "#competitive-intel"
}

Cost and ROI of Automation

A clear cost breakdown for a system covering 5 competitors, 8 sections, and weekly updates.

LLM calls per month:

• 5 competitors × 8 sections × 4 weeks = 160 calls
• Average volume: ~2,000 input tokens + ~500 output tokens
• Using Claude Sonnet 4.6: 160 × (2,000 × ~$3 + 500 × ~$15) / 1M ≈ ~$2.16/month
• Using GPT-5.5: 160 × (2,000 × ~$5 + 500 × ~$30) / 1M ≈ ~$4.00/month

Infrastructure:

• n8n self-hosted: $0 (Docker on an existing server)
• Make: ~$9/month (Core plan, 10,000 operations)

Total: ~$2–13/month depending on LLM provider and orchestrator.

Comparison to manual updates:

• An analyst spends ~2 hours on a full CI document update for 5 competitors
• Weekly cadence: 8 hours/month
• At $50/hour: ~$400/month of manual work

Automation ROI: $400 / $13 ≈ 31x. That doesn’t account for the fact that the automated system never misses an update and runs consistently regardless of team workload.

Limitations and What Can’t Be Automated

Automation handles most of the routine monitoring — in practice, somewhere around 70–80% of it. The rest needs a human.

Can’t be automated:

• Analysis of strategic intent (a competitor hired a VP of Enterprise — that’s a signal of an enterprise push, but an LLM won’t reliably connect those dots)
• Information from closed sources (conferences, networking, insider knowledge from customers)
• Qualitative product assessment (UX, performance, stability all require hands-on testing)
• Strategic recommendations (AI surfaces data; strategy is set by the team)

Technical limits:

• Sites with client-side rendering need a headless browser, not a simple HTTP request
• Anti-bot protection blocks automated scraping — resolved via proxies or API services like ScrapingBee
• HTML structure changes break parsing — resolved by hashing text, not the DOM

First Run in 2 Hours

A minimum viable CI document fits in a single session.

Hour 1: structure and content.

1. Pick 3 key competitors
2. Create a document with sections: Pricing, Features, Positioning
3. Fill sections manually — this is your initial baseline
4. Record source URLs for each competitor

Hour 2: automation.

1. Deploy n8n (Docker: docker run -d --name n8n -p 5678:5678 n8nio/n8n) or create a Make account
2. Build a workflow to monitor pricing — it’s the simplest section
3. Configure the AI node with the prompt from this article
4. Connect document writing (Notion API or GitHub API)
5. Set up a Slack notification

Once the first workflow is running, add more sections from the same template. One section takes 15–20 minutes to configure.

For more on building production-ready MCP servers for integrations like this, see the custom MCP servers article.

CI System Launch Checklist

• Competitors defined (minimum 3) with their tiers assigned
• Source URLs collected for each competitor, by section
• Document created and populated with initial data (baseline)
• At least one workflow set up with the full cycle: source → AI → document → notification
• Source validation in place: empty responses, CAPTCHAs, and 404s don’t proceed further
• AI response validation in place: JSON parses, required fields are present
• High-significance changes route to human review
• Weekly digest is configured and arrives in the team channel
• Page hashes are stored between runs (no unnecessary LLM calls)
• Error notifications are configured (if a workflow fails, the team is notified)

A living CI document takes the monitoring routine off your plate: a competitor changes their pricing on Monday, the team knows by Tuesday morning, not at the quarterly strategy session three months later. What to do with that signal is still the team’s call.

---

Need help building automated competitive intelligence systems? I help startups build AI products and automate processes — belov.works.

FAQ

How do you prevent the system from flooding the team with low-quality notifications and creating alert fatigue?

Three-layer filtering solves this. First, the hash comparison at the source level ensures no notification fires unless the page actually changed. Second, the significance rating in the AI processor prompt (high/medium/low) routes medium and low changes to the weekly digest rather than instant alerts. Third, the weekly digest prompt groups changes by competitor and sorts by significance, presenting the most important items first with a 500-word cap. The result is a single weekly message covering all competitors rather than a stream of individual notifications. Teams that skip the digest format and send every change as an alert typically abandon the system within a few weeks.

What is the right approach when a competitor’s website uses heavy JavaScript rendering that breaks simple HTTP scraping?

Deploy a headless browser at the orchestrator level using Puppeteer or Playwright in a dedicated Docker container. The workflow sends the URL to the headless browser service instead of a direct HTTP request; the service renders the page fully and returns the text content. This adds 3-5 seconds per page check but handles virtually any modern SaaS site. Alternatively, use a managed service like ScrapingBee or Browserless that handles browser instances in the cloud. Reserve this for Tier 1 competitors only — the added complexity and cost are not justified for quarterly monitoring of Tier 3 players.

How do you handle the system when a competitor undergoes a major rebrand or website restructure that changes all the URLs and page structures?

Treat it as a manual update event, not an automated one. When a major structural change is detected (the hash comparison flags the page as changed but the AI processor returns low-confidence output or an error), route it to a human analyst for a 20-minute manual review. Update the source URLs in the competitor configuration, refresh the baseline content for affected sections, and add a changelog entry noting the rebrand date and scope. Major rebrands are strategic events worth a human read regardless — they represent a positioning shift that deserves interpretation beyond what automated comparison provides.