Nightwatch — AI SEO Agent

Challenge

A lot of SEO work is research-heavy and repetitive: investigating keywords and markets, crawling and auditing sites, analysing the competition, and turning all of it into clear recommendations. Doing this well takes time and expertise, and it doesn't scale — every new project starts the research from scratch. Nightwatch wanted to give its users an assistant that could carry out this kind of multi-step SEO work automatically, drawing on the platform's own data and tools.

Solution

We built an AI SEO agent system: a team of specialised AI agents that work together to automate SEO research and analysis. Different agents take on different roles — research, crawling, SERP analysis, competitor analysis, content recommendations — and are orchestrated to tackle a task end to end, in parallel or in sequence as needed. The system draws on Nightwatch's data and external sources (such as Google Ads keyword data and live web crawling), and uses a knowledge base with vector search so the agents can ground their work in relevant context. Long-running tasks stream their progress back to the user in real time, and the whole service runs multi-tenant with per-user access and usage controls.

Project Results

Nightwatch users can hand off involved SEO research and analysis to the agent system and get structured, actionable output back, rather than doing every step by hand. Because the agents combine the platform's own ranking data with live research and a searchable knowledge base, their output is grounded and relevant. Real-time streaming keeps long tasks transparent, and the multi-tenant design lets it serve many users safely.

Features and Benefits

• Multi-Agent Automation: Specialised agents handle research, crawling, SERP and competitor analysis as a team.
• Grounded in Real Data: Combines Nightwatch's data, live web crawling and Google Ads keyword data.
• Knowledge Base with Vector Search:** Retrieval-augmented context keeps recommendations relevant.
• Real-Time Progress: Long-running tasks stream results back to the user as they work.
• Multi-Tenant & Controlled: Per-user access and usage limits make it safe to offer at scale.

Technologies Used

• Python & FastAPI: The agent service and its API.
• OpenAI: The language models powering the agents' reasoning and output.
• Agent framework: Orchestrates the team of specialised agents and their tools.
• PostgreSQL with pgvector: Knowledge base and vector search for retrieval-augmented generation.
• Celery & Redis: Background processing for long-running agent tasks.
• Google Ads API & web crawling: External research inputs for the agents.

Development Process

We designed the system around specialised roles rather than one monolithic prompt, so each agent could focus on what it does best and the orchestration layer could combine them for a given task. We connected the agents to the inputs that make their output useful — Nightwatch's own data, live crawling and keyword data — and added a vector knowledge base so they could retrieve relevant context instead of working blind. We built the service on FastAPI with background workers for the long-running jobs, streaming progress to users in real time, and wrapped it in multi-tenant access and usage controls so it could be offered as part of the platform.