Introduction
As a website grows in size it becomes increasingly important to have two sitemaps on it. One will be an XML file to be read by search engines and the other will be an HTML file to help visitors to quickly find the sections and pages of the site.
I use Xenu's Link Sleuth to check my sites for broken links. It was last updated on September 4, 2010, with version 1.3.8, but still works well on Windows 11. As well as listing broken links, both by link and by page, the program also lists all the pages on the site, produces a site map, and produces statistics about the site. When run on the web server it can also list orphan files, that is files that are in the website directory tructure but not linked to. Here is an example of a Xenu HTML report.
Xenu's Link Sleuth is showing its age a little, for example, it does not count webp images. For smaller sites then Screaming Frog's free tools are a great modern replacement.
The Visualizations
The problem with many of the visualizations is how big and complex they can become. This site has 20 main sections containing 1,000 pages. This means that many of these visualizations cannot be used without panning and zooming. They look so cool though!
Org Chart
One method of visually representing the structure of a website is to use an org chart:
Directory Tree
By rotatimg an org chart a directory tree type of chart can be produced. This more closely resembles a text only sitemap such as mine.
Radial Directory Tree
By rearranging the elements of an org chart or directory tree a radial directory tree can be produced. This seems to be the default for many of the website mapping tools.
Simple charts like the ones above can be created by hand using programs such as Inkscape, but for larger sites it is probably best to use one of the pre-built tools.
Programs such as Cytoscape, Gephi and Graphviz are good at taking structured data and producing the graphs.
Screaming Frog's website visualizations are good and have segmentation, pan and zoom for exploring a site. It also has good help pages on how to use them to find specific information.
Kiran Tomlinson created Site-graph. It require Python and some other dependances to run but is very impressive.
Internet Maps
Over the years people have created some interesting maps or visual representations of the entire internet. Most look like star maps, the larger the number of visitors the larger the "star" and the more links they share the closer to each other they are.
Internet map from The Internet Map
Apart from The Internet Map there are others that are videos showing the changing internet such as The Opte Project and Webverse
One interesting visualization is one from Halcyonmaps that maps the internet as an old world map.
Internet map from Halcyonmaps