The High Price of DevRel Debt
Why DevRel content creation needs to factor in maintenance from the outset.... and how to plan for it.
Introduction
“The goal for this quarter is to produce 2 new articles, 2 videos, and a new integration with our partner company.”
A phrase like this would not be out of place in a Developer Relations planning meeting but it hints at an issue just beneath the surface.
While content creation is one of the most effective ways for a tech startup to build organic awareness and highlight their product, such directives actively prioritize new content without factoring in the requirements of maintaining what already exists.
When the goal of DevRel content is to build developer love and support, a successful content strategy should balance between the creation of original content and time for effective maintenance of the existing work. Without such a balance, just like with technical debt, DevRel Debt can sneak up over time, creating lasting negative consequences for the company.
What is DevRel Content?
For the purposes of this article, I will be focusing on the materials that are generally expected from DevRel teams. These include:
Blog posts
Tutorials
Product Guides
GitHub Repos
Workshops
Demo apps
Integrations & Plugins
Videos
I am explicitly excluding product docs because, although it’s possible that a DevRel team may also work on docs, there are two reasons they may not be as relevant to this specific piece.
“It is a truth universally acknowledged…” that company docs should be maintained as diligently as possible and reflect the latest of each product version. A company that does not subscribe to this philosophy may have bigger challenges in their overall strategy.
There are a variety of wonderful tools and processes that are popular in the technical writing community that can automate a lot of maintenance. As someone with more experience in DevRel than technical writing, I’d encourage folks to investigate those separately.
What is DevRel Debt?
DevRel Debt is the backlog of potentially outdated materials that were created (with the best of intentions) by the DevRel team for various initiatives.
These can include written content that recommends the latest version of a certain framework without realizing that the latest version is no longer compatible with the existing steps, or a video highlighting a certain UI in which the entire interface has changed.
While not all older material is automatically outdated, many types of unchecked material could be unusable. Without a team review process, there is no way for Advocates to specifically understand what is still effective and what is actively detrimental for their users.
The debt can be internal or external.
External-facing DevRel Debt would be something that is encountered by a non-employee, a user, trying to complete a task using content created by the team, for example an undated tutorial that now breaks due to recent tooling updates that are not accounted for in the text.
What makes this so especially insidious is that many such issues are not obvious when a user undertakes the task. They may be pages into the tutorial or half an hour into a video when the error shows up. By this point they’ve invested a significant amount of time, and their frustration may be even higher than it would have been had they found the piece and known it to be not relevant to them from the outset.
Internal-facing DevRel Debt would be a material that would be expected to be a foundation or feed into another asset, which then again turns out to have out of date or unusable steps, for example a video for a specific site page that shows an out-of-date product version.
This requires the team-member who assumed they could use it in their own work to either discard their subsequent asset or go back and update the original, slowing down their productivity overall.
The way to counter DevRel Debt is by investing the team’s time and energy into consistent maintenance practices!
What is the cost of DevRel Debt?
In the same way that taking on technical debt allows a product team to move quickly and deliver certain features, while at the same time creating future drag on code creation and refactoring, so too does DevRel Debt deliver results from the outset (views, engagement, conversation), but later generate a variety of obstacles.
There are few main costs associated with unmaintained content:
Lost trust with users. If the goal of the DevRel team overall is to inspire trust with its users, outdated content has the exact opposite effect. Where in the best case scenario a user would be convinced by the seamlessness of a certain article or video, running into an issue halfway through reflects badly on the company overall. The halo effect of such an experience can go as far as to cast the product itself in a negative light, and dissuade users from trying to implement it or giving it a chance. This has implications across the entire brand perception — and can be especially egregious if other assets, for example those focused exclusively on marketing are in a good and updated state, giving the impression, whether true or not, that the company’s priorities lie more in advertising than in truly helping developers.
This lost trust may be challenging to quantify as well. It is not always easy to tell how many people would have signed up for your product, or would have been better engaged with your site had they had a better experience. In this case, furthermore, the general data can go as far as to be misleading. A broken tutorial may have a longer “time on page” than something that works, but not necessarily reflect that the user is engaged with the piece as much as that they are stuck and debugging as they work.
Heavier support load. Well written content and resources, as well as docs, are often an effective way to stem the flow of support tickets as users can resolve their own issues. It’s no surprise that the philosophy of RTFM has such a grip over the tech world. Nonetheless, assets that create the exact opposite effect, besides being a massive source of frustration for the users, can have financial implications. A company may need to hire additional support staff or slow down their support speed because of the number of questions caused by out of date content.
Decreased acquisition over time. Concurrently with the lost trust issue, a bad reputation from old assets overall can increase the barrier to user adoption as the company’s poor reputation can precede itself. With enough people investing the necessary time to try following an asset and failing, they may influence the overall decision to choose a competitor with a better overall experience.
5 Reasons why battling DevRel Debt is not a priority
Now that we’ve outlined the problem, it’s worth discussing in more detail why maintenance may play second fiddle to the development of new content.
Optimism. When a startup is new on the scene their product is groundbreaking and their content is fresh, maintenance doesn’t need to be factored into the equation overall because everything is current. In the first six months to a year of content production, the focus is exclusively on ramping up content, experimenting with effective channels and building up the brand. Not only would maintenance work here be unnecessary it would actively slow down one of the main acquisition drivers.
Because leaders and managers see the approach working seemingly without issue, it is completely natural that they would continue to push for the same technique moving forward. But in this case, time is not on their side, and the content naturally ages. It is here that the optimism starts to go from being natural to being misplaced. As time passes and the content grows older, it would behoove leadership and Developer Advocates to begin to push for maintenance in the overall strategy. In other words, past performance is not an indicator of future success.
Lack of knowledge or experience. Factoring in the common practice of creating new content, both among the creators as well as the leaders pushing for it, there may just be less institutional knowledge both around the value of maintenance and as well as how to actually do it. Among technical writers and product documentarians, there may be more visibility into effective techniques, but overall in the DevRel space there are simply more resources out there on how to create content than how to maintain it. This can furthermore be exacerbated by the average lengths of tech workers’ tenure. If people move between companies every couple years or so, they may not stay long enough to see their work age — thus avoiding the most detrimental effects of such changes and having less chance to practice maintenance.
This knowledge may be less well-known and more difficult to acquire through experience.
Overall in the DevRel space there are simply more resources out there on how to create content than how to maintain it.
Lack of visibility around the need. Organic content is a well known acquisition and awareness building channel that has proponents across a variety of industries. It is very easy to find thorough articles, videos, interviews with marketing leaders of all kinds explaining in detail how their specific approach to content marketing provided a cheap and effective source of new customers. Far rarer, however, is the discussion of “boring” maintenance in this conversation.
As teams and companies learn from other leaders in the space, there is less reason to factor in maintenance if the seemingly successful content marketing strategists don’t highlight it. In other words, “They don’t seem to find this to be an issue, why should we bother?” Nonetheless, it’s completely possible those original teams are doing maintenance behind the scenes but may not necessarily feel the need to share the nitty-gritty details in an inspiring marketing piece.
It’s challenging to automate. For many marketing heuristics, the use of metrics can quickly inform the success or failure of an approach — for example in the case of ads, email opens, etc. However, disparate DevRel content may not have any definitive visible metrics that can simplify this process. Even something as simple as, “Did you find this content useful? Thumbs up/down” while helpful, will not be enough to be actionable on its own. Nor is just feedback enough to rely on, as only a small percentage of users may reach out. There are lots of metrics that can inform whether content needs active review, but these alone are not enough, without a lot of manual lift to check the work itself.
It’s not as exciting. In the same way that Google has a reputation of releasing new products rather than effectively working on existing ones, it is far easier to pitch producing new assets across the board than the somewhat silent and less glamorous, time-consuming maintenance processes. While having up to date assets can pay off in terms of reputation, lowered support cost, and reduced friction in acquisition channels, it isn’t as impactful at a board meeting conversation nor at a company all hands.
Just like the difference of creating new vs refactoring existing code, for Developer Advocates there may be more recognition and overall ease in creating new content. Reviewing existing content, however, is both challenging because you have to navigate someone else’s thought process and approach, and there is often less space for acknowledgement, as the original author’s name would likely still be most prominent.
There is less to announce after a thorough recheck than when one can proudly announce that new content is live. Thus, the interest of individual contributors and managers can align to smooth the way exclusively to new content while adding significant friction when considering maintenance.
Nevertheless, while these reasons add up to a powerful resistance against maintenance, in writing this piece, I hope we can begin to open up the conversation in the opposite direction.
Leeway in DevRel Maintenance Strategy
As we begin thinking about maintenance, I want to offer a small digression on user behavior and introduce the idea of “forgiveability”. In other words, when a reader reaches an asset (a video, written guide, GitHub repo, etc), how understanding might they be when the asset turns out to be out of date. This is usually connected to how visible the date of publication or update would be.
Things that are tightly connected to something timely, therefore, are more obvious to assume to be out of date if their publication date has long passed.
A couple examples of these kinds of assets include:
Blog posts – even technical ones clearly have an associated date and an order within the blog.
GitHub repos – the metadata around the last commits highlights the last updates and activity.
Content published in external media outlets – like blog posts, these are tightly connected to a specific date of publication.
YouTube Videos – not only do these have an upload date, but their timeliness can be made even more visible if the versions of all of the software used is highlighted clearly in the video description.
Because the date is so actively visible, it is very easy for a reader to gauge whether or not the piece of content is appropriate for them. An older tutorial or video may still be very effective for their uses if the underlying technology it uses updates slowly.
Conversely, assets without clear dates or numbers can be especially brutal for someone trying to follow them. Some examples of these can be:
Guides and tutorials – these often are undated and have prerequisites that recommend installing the latest version.
Embedded videos — while YouTube videos can have clear descriptions, videos embedded into site pages have more obscure meta-information.
Tutorials and walkthroughs on your product on another company’s site — when these are guest contributed and also lack dates, these can both reflect poorly on your and your partner’s product if some part of it breaks
Therefore, it always makes sense to prioritize signals that highlight creation, update, or review date to establish a baseline understanding with your users and to set expectations accordingly.
Considerations in Maintenance Strategy
With all these details in mind, there are some very specific tips that can inform your maintenance strategy, although these are just suggestions, and nowhere close to the comprehensive plan necessary.
I’ve broken it down into four stages: Alignment, Outset, During Creation, After Publication.
Alignment
Before any of these steps can be taken, it’s key to get team and management buy-in. Committing to maintenance and aligning on the fact that is important for the company and team will make taking each individual step far easier.
Lacking this broader commitment can make specific efforts more disjointed and set up the review lifecycle to fail.
Outset
First, factor in what is “expensive” to maintain and plan for that. Certain assets and parts of your content are going to be especially difficult to maintain or to highlight as out of date. Some examples of this include:
Screenshots of UIs and especially (!!) screenshots of external UIs. These can change without notice in big or subtle ways. Unlike tools that have version numbers, interfaces can evolve without distinct updates. In general, one should really verify if UI screenshots are necessary for the written piece, doubly so if it is of an external UI. If you find that you must use screenshots, be deliberate with your team about how you organize images.
Solution: Create a plan from the outset, whether through names of the images themselves, the folders they’re stored in, or potentially how they're tagged to keep images orderly and easy to find and access when updates are necessary.
Content on specific languages, frameworks, or tooling that is not well-known throughout the team. This can occur when the company is looking to expand into a new vertical, like appealing to developers using a specific programming language, and hire a developer advocate specifically to create content of that type. This can work efficiently from the outset, but one should be especially mindful of how the work could be distributed if the person leaves the company.
Solution: Be aware of work that is harder for the team to update. Try to create content that is explicit about the necessary prerequisites and can be checked by anyone, even if they are not an expert in that field.
Second, use the planning process to facilitate easier maintenance later. Certain types of content can be especially effective as placeholders and triggers for more general updates:
Core resources. Keep a running tally of the types of materials that are widely relied upon. An example of this could be a feature walkthrough that is embedded within the company’s site. Having a small list of essential assets means you can be responsive to the most important changes while keeping the surface area manageable. You can also use updates to these key assets as a reminder to revisit similar content on other platforms. Furthermore, keeping these up to date can reduce the internal DevRel Debt, mentioned earlier, keeping teammates more productive overall.
External workshops. These are checked often because they are publicly presented to stakeholders outside the company, and the presenter will usually run through it a couple days before it’s live. This makes this a wonderful resource to tie other potential updates too — if the workshop needs to be updated, the presenter can alert the team that certain facts are no longer correct, and then trigger a cascade of updates throughout the team’s existing work.
Product Docs. These may have stricter requirements on being up to date than tutorials. Keeping in close contact with whoever is writing the docs may give developer advocates the opportunity to update other assets that are tightly connected to whatever feature has a new description in the docs.
In short, making an effort to reduce what could be challenging to the team as well as set up some checkpoints that can start a maintenance process can be a relatively small investment up front that can later simplify how content is updated later.
During creation
When working on the content itself, there are also a number of ways to later decrease the maintenance burden:
Include key cues to asset age. When possible, try to ensure that assets created have indicators that allow the reader to gauge its suitability for their purposes. Specifically, the date of creation or review as well as the specific version numbers of all the tools used within the piece.
Sidenote: Certain projects always import the latest version when being accessed. When a piece has a lot of dependencies and this is a practice across some percentage of them, it puts the asset at risk of breaking a lot sooner, as some changes may not play nicely with others. These also may be harder to debug as you won’t always know, which dependency caused the asset to no longer work as expected. Therefore, if you know this to be the case in something being created (a tutorial or video or similar), consider noting it and either scheduling it for review more often or expect the review to take longer when it does happen.
Pair written or video content with code. If you’re producing a video or a written publication, you may want to create an additional repo that stores the code from the project. This serves two purposes, first it associates the content with a date (in other words when the first or last commit was), and second code is far faster and easier to check than completing an entire tutorial or following a video. Having some external code allows the consumer to quickly review the feasibility of the project before working through the piece step by step. Additionally, it provides a way to pin the version numbers of the tools and create a project that works cohesively together.
Test automatically. When creating new repos and projects, consider adding testing to the finished product. Using some CI/CD (eg. GitHub actions), you can ensure that some automated checks are running in the background or that the sample app is regularly deployed and working. If a check fails, you can get notified and use it as a sign to manually update the code and associated assets that rely on it.
Containerize when possible. An alternative to pairing a piece of content with a repo, is to take it one step further and containerize the entire project. This step would again pin the version numbers of all the tools used as well as guarantee that the project would work on a local machine.
After publication
Once you have your assets in place and various strategies to keep them updated, there are post publication activities that can further support your maintenance approach.
Track especially successful content. This is something that may already be happening through the work of the marketing team, but having a running tally of which content has the greatest amount of views, engagement, usage, or conversions (whichever is most relevant to your company) can help you put certain pieces on the faster review schedule.
Include a way to collect (and track) feedback. Depending on what the asset is, a tutorial vs a video vs a premade script or cookbook, always include a way to collect and track feedback. For a tutorial, this may be a matter of something as simple as adding an email or form to collect comments at the bottom of the text, or asking on the page, “Was this useful?” and giving an opportunity to leave feedback. At the same time, this means actually taking the time to track feedback when it is collected automatically — for example reading through YouTube video comments regularly or reviewing issues opened in the relevant repo in a consistent cadence.
Schedule check-ins for external content. Because feedback on your content created for external partners will not come in automatically, one should be more deliberate in ensuring that it remains up to date. You can schedule a regular (quarterly? half-year?) review of the most impactful external content that is expected to be evergreen. In other words, this is completely unnecessary for, to underscore the point, external publications that have an associated date, but can be very useful for pages that actually bring traffic and conversions effectively to your site.
Another reason why it may be useful to have a regular check-in planned is because when working with external folks all changes have to factor in their schedule. Shepherding a PR through their system may take longer, of course, than through your own. A scheduled review can help you begin to make these changes without rushing once things are especially broken.
In this situation, additionally, one should do their best to get a generic address to reach someone at the partner company. For example, if you worked with Alice at the partner company, see if there’s a broader, devrel@company or marketing@company email that you can note for updates, in case Alice moves on to another opportunity.
Publicly recognize reviews. Checking an existing resource can be an intensive process. Publicly recognizing reviews can serve two valuable purposes. First, it can give recognition to the checker in the same way as the original author would, helping the act of review and debugging get greater public acknowledgement. In other words, once someone has reviewed the piece, explicitly add their name and date to the piece close to the byline. Second, time-stamping when a resource has been reviewed gives the person following the resource greater confidence that it is in fact something that is worth their time investment.
Sunset resources. Certain assets may have been useful at a certain point but there may come a time when they simply provide diminishing returns to the company. The worst case scenario could especially be a high traffic asset that explicitly doesn’t work and captures many eyeballs and then leaves users unhappy. It may be worth reassessing whether the piece is worth keeping at all. If the cost/benefit of a piece points to it no longer being useful, reflect on what’s the best way to gracefully sunset it. This can mean keeping it up with a deprecation notice (bonus points if it can link to an up-to-date replacement), moving it to some archived part of the site, or deleting it.
It can be sad to part with something that a lot of time and love was invested in, but there are points when this is the best approach.
Bringing it all together
There are many reasons why prioritizing maintenance may not always be top of mind for a team, but in the long run implementing some forward-looking strategies and regular reviews can complement the breakneck pace of production that DevRel teams sometimes set for themselves.
At the end of the day, the function of a DevRel team is to improve the experience of the developers using their product— ensuring that the resources that developers interact with daily are in the best condition possible makes sense not just for when they’re new and shiny, but for the entire lifespan of their existence online.
Acknowledgements
Thank you so much to:
Celeste Horgan, Developer Educator at Aiven
Justin Ellingwood, Technical Writer at Koyeb
Both shared their invaluable experience, offered detailed suggestions, and helped the piece take shape!
This article would not have been possible without them.
Get in Touch
Questions? Feedback? Suggestions? Feel free to reach out to me directly: etel [at] hey [dot] com