Effective documention for software projects

Documentation of software is just as important as the functionality; without documentation it becomes nearly impossible to use, modify or contribute to it. But writing documentation is not easy, and if not maintained it creates more confusion than it solves.

I’m looking to collect guidelines, resources or advice on how to write documentation which

  • allows to quickly pick up and use the software
  • makes it easy to dive into the code, learn from it, modify and reuse it
  • clears up how to contribute and reduces as much friction as possible

One exhaustive resource is “The Documentation System” by DIVIO, which categorizes documentation into four types. It doesn’t go into more specifics regarding software though.

I think using commonly used software helps a lot for discoverability and onboarding, which puts FOSS projects in a tough spot: Many projects are on Github and use other proprietary services, while more niche platforms get overlooked, so creating an accessible project is a balancing act.

3 Likes

In terms of platforms, in the Hare project we recently moved from ad-hoc docs website put together with Hugo to a more structured approach using Sphinx, and the results are quite nice:

https://harelang.org/documentation/

This is the same documentation generator that the Linux kernel uses, though we elected to keep the much nicer RTD theme rather than the new default theme recent versions of Sphinx uses. I would highly recommend Sphinx for the “explanation” category of documentation per DIVIO’s model.

We also have generated reference documentation (being near to the code makes it easy to stay up to date), and a separate location for tutorials. I think one common mistake that some projects make is not realizing that each of these different kinds of documentation calls for a different approach, and that categorizing a given piece of content correctly is very important – can’t tell you how many times I’ve seen all of the docs for some project being crammed into one poorly suited tool, like a GitHub wiki, regardless of how adept that tool is for that kind of content.

Another underappreciated form of documentation that applies exclusively to free software is the source code. One thing I appreciate about the Go reference documentation, for example, is that you can click on whatever function you’re looking at at go straight to the implementation. I think we need to encourage reading the code as a form of learning, and make it easy to do so.

1 Like

I’ve started a few projects (including one that’s gotten fairly popular), and I always feel like no matter what, the documentation that the developer writes (that I write, rather) is never enough. In my case, I have an authorization/authentication plugin for Jellyfin. Because the plugin works over OpenID/SAML, and there are many different providers that can interact with the plugin, the documentation is always lacking surrounding compatibility with other providers.

Community contributions to documentation are useful, but also problematic—the documentation needs to be constantly updated. Just recently, an issue was filed about contributed documentation being out-of-date. As a single developer, it is near impossible to document these (very common) usecases because each setup is different.

I believe that effective documentation should follow these principles:

  1. Generalization: Documentation should avoid discussing highly detailed integrations, as these integrations are bound to change, and the documentation becomes out-of-date and difficult to maintain.
  2. Clarity (pretty self-explanatory)
  3. Staticy-ness-ity: Documentation should not change often! Software should be written such that the documentation should not need to be updated frequently (and the updates should be solely additions, not modifications). This reduces the burden on users of the software and makes it easier to work with the software.
3 Likes

Hi !
Writing here to bring my personal experience as a developper with ADHD issues, I thought it might be interesting to share some of my experience. It will be nothing too formal as I still have many things to learn. I just do it in a way that helps me. and might helps others with the same issues.

For me who quickly lose my focus, I kinda love projects where every “key” symbol (function, structs, …) has some documentation (even in private contexts). Not something too fancy or complete, but just to add a bit of context.

In such projects, being able to re-focus quickly is quite helpful (without having to navigate around each symbol to get back to the state before I lost my focus).
However it’s not easy to make my colleagues do it, as this can be rather time consuming.

3 Likes

Thank for your sharing all those experiences! My 2cents about documentation is that personally I find very it difficult to organize more than actually writing the documentation.

Writing is easy as soon as you find a good place for it. But there are many personas: newcomers, expert looking for a quick fix, non developer if your tools targets them.

And also many context: maybe the reader end up looking for a quick and easy response this time, but tomorrow it will may have more time and interest and it will look for a more descriptive and in dept article.

There is documentation that shows up in code editors or developer friendly shortcuts like macros or snippets.

Organize and serve everything we write in a way that makes sense is something I find challenging.

1 Like

Something else that occurs to me to mention about documentation is that writing docs is a skill independent of writing code. It gets better with practice! Writing high quality documentation is a skill, and it’s normal that it can be more difficult for people with less experience.

That said, only way to get experience is to try, and users appreciate any docs over no docs, even if they’re not as good as they would be from a more experienced technical writer.

A colleague of mine has developed the Diátaxis framework specifically with the purpose of authoring technical documentation.

In a nutshell, it splits the content of the documentation into four categories:

  • Tutorials
  • How-to guides
  • Technical reference
  • Explanation

I recommend you go check the website for a more detailed explanation.

I think the Django documentation uses this framework, and I think it’s fairly clear to the users.

1 Like

I always appreciate having examples in reference material. There are always unstated assumptions in speech, and there have been a number of times where I’ve been reading an explanation that didn’t make sense until I looked at the example, and realized that I was just on a different page than the author. One of the things I appreciate about the Rust toolchain in the ability to run “doc tests” to verify that examples are correct (and the supporting functionality to hide certain lines that should not appear in compiled documentation, but are necessary for doc tests to work).

1 Like

Much like @pieq I’m also a big fan if the Diátaxis documentation framework.

But that’s only half the battle. The other half is making every last bit off that documentation trivial to enumerate and then copy/download for offline study. Whether that’s Ramp Up or Reference documntation. Whether that documentation is aimed for the user or developer level.

This problem is so bad I literally started the Freedom Respecting Technology movement where addressing it is arguable the main tenet. See here for more: Freedom Respecting Technology: the Next Generation of Open Source, Free Software, Open Knowledge, Open Culture, and Technological Freedom

And yes this matters. Access to the full Open Knowledge Set associated with a given technology matters. Withholding docs from offline study is fundamentally no better than witholding part of the source. Or otherwise making it very inconvenient to get the source. Imagine for example if Lua didn’t offer taballs with their source for easy downloads. Imagine you had to scrape Lua 5.4 source code instead as your only option.

Go has probably one of the best documentation systems ever. You simply add a comment on top of the symbol and it just works with godoc and pkg.go.dev. You can add examples which also act as tests, so you can automatically detect when examples need to be fixed. Those examples are runnable and editable via the Go playground integration with pkg.go.dev. I wish more languages had such awesome tools built in like Go does, with an idiomatic and standard way of doing it.