I started a new job recently with the goal of insourcing a network team. I was fortunate that I was able to convert a few of our contractors into full-time employees, which really sped up the process. One of the reasons that the organization decided to stop relying on managed services and build their own team was so there would be more ownership of the network and we wouldn't suffer from knowledge drain as different engineers came and went.

One of those reasons came to light with a vendor just last week. Their engineer asked if we had documentation around the part of the network we were talking about. Instead of going into a long explanation of why we didn't have the specific piece of documentation he was asking about, we just said, "No, but we can answer specific questions you have and then get back to you with anything we don't know."

It was a bit embarrassing for us, and it really highlighted something for me. We DID have some documentation around what he was asking about, but it was old, inaccurate, and wasn't really in a format that was conducive to sharing.

This changed how I think about documentation. Not just who we're writing it for, but which parts of it a person should be writing at all.

Who is this documentation for?

We talk about documentation like it's one artifact with one quality level, and the only variable is whether it's current. It's stale or it isn't. Somebody didn't update it. Usually the person saying that it's stale is also the person who didn't update it, and I've been him more times than I want to put in writing.

But most of the documentation I've watched die didn't die of staleness. It died because it was addressed to nobody in particular, so nobody in particular maintained it.

Three readers

At least three people open network documentation, and they all want different things.

The Architect - They want intent. Why this design? What was ruled out? What constraint was this built against that isn't visible in the result? What has to stay true for the design to keep working? What decisions were made that led to the final design?

I don't see this type of document very often, and it may be the most expensive omission in our field. Every refresh I've been part of opened with an archaeology dig: three people in a room reverse-engineering a decision from its artifacts, trying to work out whether that weird thing is load-bearing or whether somebody was just in a hurry in 2016. We are experiencing this now with our network. There are some pieces of the network that we know work, but we're not sure why some of the features were enabled, or what would happen if we removed them. We have spent several hours debating this. Unfortunately for us, we can't generate enough endpoint traffic in the lab to look like production, so we can't fully test the questions. That's a different problem.

The Engineer - They want to understand the build. Standards, naming, address plans, what "done" means for a new closet, and the specific reason the main campus's fiber path is the way it is. They also want build specs: what do I need to do to configure a device the same way, every time? Give me a checklist I can follow so that I don't miss any steps. We're least bad at this layer, because it's the one that hurts immediately when it's missing. What goes missing most often here are the gotchas we found along the way. If you don't profile the new model of AP in ISE, it will look like it works but won't actually carry any traffic. Why? And how do you actually do that?

The Operator - Wants to know what’s what at 2am. When something is broken, what do you check first? What can be touched without waking somebody? And who do you call when you can't fix it? That's a runbook, not a reference. If it makes me read three paragraphs of context before it tells me what to type, it failed at the only moment it was built for.

One document trying to be all three serves none of them. Too abstract for the operator, too shallow for the architect, and it rots because no single person's job gets harder when it does. Or it's a 15-page document that tries to be all things to all people and ends up being too long for anyone to read.

The fix isn't more documentation. It's fewer documents, each addressed to somebody specific. And it's fine if the same fact shows up in two of them with a different amount of context around it.

The hardest part of all of this isn't writing the documents; it's keeping them up to date.

Derived or decided

Here's the split that changed how I think about upkeep, which is the real problem. Anybody can write documentation once. Almost nobody keeps it true.

Derived - Topology. Inventory. Address plans. Interface descriptions. VLAN assignments. Diagrams. Most of it already exists in the network or in your source of truth, which means a human should never type it into a document again. It comes out of whatever tool you trust — Nautobot, SolarWinds, Catalyst Center, Apstra. Diagrams are rendered from the data. Configs are generated from templates, then validated back against what the device actually says.

If a person is maintaining derived documentation by hand, it isn't GOING to go stale. It already IS stale. It went stale the first Tuesday somebody made a change at 11 p.m. and updated the device but not the page. The only question is how long until it costs you.

Decided - Intent. Constraints. Tradeoffs. The judgment calls inside a runbook. Why we accepted this risk and not that one.

This cannot be generated, and it's the part people miss when they get excited about automated documentation. It isn't that the tooling isn't good enough yet. The information was never in the network. It only ever existed in somebody's head, in a meeting, in a decision that got made and never written down.

Ask yourself one question: can a change on a device make this wrong? If yes, it's derived, and it should be generated. If no, it's decided, and a person has to sit down and write it.

Automate the first bucket ruthlessly. Spend the hours you just freed on the second one, which is the only bucket that needed a human in the first place.

Where AI helps, and where it doesn't

I've been playing with AI on this for the last few months. My opinion is that it's useful in a narrower place than the pitch decks suggest, and more useful there than I expected.

Drafting from the derived layer - Give it a config and a change ticket and you get a serviceable first-pass runbook entry. Not a final one. A first one, which is the draft that historically never got written because nobody had forty minutes. It's hard to start with a blank page and build a document from scratch. If you know what your intent is, have some initial data, and have an AI setup that you trust with your company's data, you can use it to give you a first draft to get you started.

Telling you the doc no longer matches the config - This is the real win, and I think it's underrated. Drift detection is a better use of AI than drift correction. I don't need it to fix the page. I need it to tell me the page is lying, on a Tuesday, before somebody makes a decision on it at 2 a.m. In my home lab, I have AI look through my currently running configurations, compare them with my stored configurations, and call out the differences. Then I have it quiz me about why those changes were made and when, so I end up with not only a change log but a decision log.

Intent, where it fails in a specific and dangerous way - Ask AI why a design is the way it is and you get a fluent, plausible, confident answer reverse-engineered from the artifact. Sometimes it's right. When it's wrong, it's wrong in the exact register of somebody who knows, and that's worse than a blank page. A blank page tells you to go find the person who remembers. A confident wrong paragraph tells you to stop looking.

Finding the document when you need it - Putting a local AI on top of your document repository is the other place this pays off. If it has access to your documentation, you can ask it a question about your environment and get the answer without hunting through and reading five different documents. Better than that, it will tell you which documents it pulled the answer out of, so you can go read the rest if you want to.

Where we are today

Here's where my team actually is now. Our source of truth comes out of our management and monitoring systems. Those systems know what we have and how it is configured. We are actively working on anything that can't be discovered, like our standards documents.

The one thing that we haven't decided on is WHERE these documents should go. We're trying a few things, but like the types of documents themselves, there doesn't seem to be a good one-size-fits-all location. We are still in the theory phase of AI for document management, but I see that happening sooner rather than later, which may end up solving our question of where.

The question I'd ask about your own environment isn't when you last updated something. It's this: which of your documents could be wrong right now without anybody finding out, and which reader would it hurt?