URL Monitor System
| Key | Value |
|---|---|
| Status | Active |
| Owner | QA Automation |
| Updated | 2026-03-26 |
| Scope | Link crawling, broken-link reporting, and URL-quality monitoring across CNC sites |
The URL Monitor exists because broken links create a very different kind of failure from selector or journey regressions. The page can load. The UI can look fine. The user can still land on a dead page, a redirect loop, or a quietly expired campaign destination.
What The URL Monitor Does
| Capability | Why It Matters |
|---|---|
| crawl pages | finds the links the sites actually expose |
| extract links intelligently | keeps the signal closer to real user navigation |
| check URLs asynchronously | makes large checks practical |
| deduplicate results | avoids alert floods |
| track source pages | helps people find where the bad link came from |
| keep history | lets the team spot recurring URL issues |
Crawl Depth Options
| Crawl Mode | Best For |
|---|---|
| basic crawl | quick health check |
| nav-focused crawl | menu and structural link paths |
| full crawl | deeper audits and larger sweeps |
What Makes It Useful
The system is not just a raw link checker. It keeps enough context to help someone act on the result.
| Useful Context | Why It Helps |
|---|---|
| source tracking | you know where the bad URL came from |
| history | you know whether this is new or repeating |
| deduplication | one broken endpoint does not become dozens of noisy alerts |
| scheduled operation | issues surface without someone manually crawling the sites |
Typical Problems It Catches
| Problem Type | Example |
|---|---|
| broken navigation targets | menu item or footer link goes nowhere |
| expired campaign pages | old marketing or editorial landing page now returns 404 |
| backend 500s | link exists but destination is broken |
| decommissioned subdomains | historical links still live on the page |
Slack And Reporting
The URL Monitor should feel like an operator tool, not a crawler dump.
| Reporting Goal | How It Should Feel |
|---|---|
| clear severity | is this one broken page or a broad issue? |
| source context | where was the link found? |
| trend awareness | is it recurring or brand new? |
| low noise | one incident, not fifty near-identical messages |
Why It Belongs In This Repo
Links are part of product quality. For news sites, bad links affect:
- reader trust
- editorial campaigns
- SEO
- subscription journeys
- archive content
That makes URL quality a real operational concern, not just a maintenance nicety.
[EXPAND: Good operator workflow]
When the URL monitor reports problems:
- check whether the failures cluster on one destination or one source surface
- confirm whether the breakage is editorial, backend, or infrastructure
- fix the source link if it is a content issue
- escalate the destination if the source is correct but the page is broken
[END EXPAND]
Related Pages
| Need | Go To |
|---|---|
| general monitors | Monitoring |
| commands | CLI Reference |