This is the subsystem for discovering to new articles. Links are
discovered by checking section pages on all sites we monitor at a
frequency determined by the importance of the site and the average
number of articles published on the relevant section.
Link discovery in sync
This tells if sections are checked as frequently as desired, which
implies that about 2000 section pages are checked every second.
There are 2-300 link discovery processes running, and each link
discovery process is responsible for checking a set of sites. This
shows id all these proceses are functioning properly.
When links are discovered, another subsystem is downloading the article
pages the links points to.
Downloading servers operational
This tells if a sufficient number of downloading servers are up and running.
Downloading programs operational
This tells if a sufficient number of downloading processes is running
on the downloading servers.
DNS service on downloading servers
Monitors if the dedicated DNS servers are operational on the
Downloading queue size
Monitors if the queue of articles waiting to be downloaded is normal.