Chapter 16

Select a Content Management System

Summary

Selecting a CMS is a combination of research and vendor engagement. You need to identify prospective systems, investigate their capabilities, engage with the vendors for demonstrations or questions, and finally distill and synthesize all that information and come to a decision.

Does this Apply?

If you need to select a new CMS, then yes, absolutely. However, as we note below, the process we’re discussing here is full-scope, from soup to nuts. You might not need all of this, and we’ll talk about how to abbreviate the process at the end. If you’re not picking a new CMS, or have already picked one, then feel free to skip this chapter.

Narrative

Just about every website runs on a content management system (CMS). This entire book is predicated on the assumption you would be using a CMS for your website.

To be fair, you don’t have to use a CMS. Whether they’re generated by a CMS or hand-crafted, web pages are still just HTML, and there have been some fun advances in static site generation in the last few years. But for most organizations, usage of a CMS is a non-debatable point. Once your content gets over a certain volume, and especially if you have multiple editors working with it, then a CMS usually becomes a requirement.

At some point, you’re going to need to select which platform to use, and this is where things can get difficult. When you consider that the CMS as we know has been around for only about 25 years, you can imagine the level of volatility we’re still seeing in the industry. Some people on your team may have never evaluated a new CMS, or perhaps haven’t done it for a decade, which means the industry has probably turned over multiple times since then.

CMSs are complicated things, and so are projects. They both have a lot of moving parts.

You know those inflatable, spastic dancing men that car dealerships use to promote sales? The ones with the arms that whip around non-stop? Try getting two of them to dance in perfect synchronization. Now throw software vendors and their sales teams into the mix, and we’ve got a real party.

The best we can do is try to wrap some structure around the process so it doesn’t devolve into a confusing free-for-all.

(Disclaimer: you may read through this and say, “Wow, this is a lot.” And it is. But we thought it was better to show you the long-form method, and then let you dial it back if you like. We’ll address how much of this might apply to you in a section at the end. Hang on – we’ll hopefully put it all in context.)

The Selection Funnel

A funnel is a handy metaphor. A funnel can represent anything that starts out with a lot of something, then slowly whittles the number down over time.

The reality show Survivor is like a funnel. There are lots of people on the island, and we slowly vote them off as we move from the big end of the funnel to the small end. Finally, a single winner pops out.

With CMS selection, this funnel is just two or three steps. You’re going to start wide, asking vendors to provide general information about their systems, then you’ll eliminate some of them and narrow the group by asking the remaining vendors to relate their systems specifically to your requirements, then you’ll eliminate more, then you might go an extra step and ask very few of them to actually do some integration work for you.

Finally, you’ll end up with a selection.

The Long List

This is the widest end of the funnel. These is where you create a “Long List” of potential vendors and toss them into the mix.

How do you come up with the Long List? There’s no great answer to this. The longest list is every possible vendor on the planet. But clearly, that’s unmanageable and not really fair because you should only include vendors that have at least some chance of satisfying your requirements. In general, you should try to keep the “long” list as short as reasonably possible.

Two reasons for this:

  1. Selection processes take work. You need to manage requests and responses and do the evaluation, usually spread among multiple members of a team. Every extra vendor means more work, not to mention more confusion because they run the risk of distracting team members with crafty sales pitches and the inclusion of things they want to promote, but that don’t really help you or contribute to your project.

  2. Vendors are people too. When a vendor responds to your request, they’re investing time that they could be spending on other opportunities. Sometimes, they’ll work nights and weekends to get your request managed and out the door (especially towards the end of a quarter). You’re in a position to create work for someone, and don’t do that unless they have a reasonable chance of seeing a reward. Don’t waste people’s time.

When developing the Long List, ask yourself these questions as absolute top-line filters (if these don’t look at all familiar, then you might want to re-read the last chapter):

  1. What is the nature or “gestalt” of your project? Do you need to simply manage content, like technical documentation or other non-marketing content? Or do you really need a full-blown content marketing platform that happens to also manage content?

  2. What are your technical restrictions? If you must have a Java-based system, or you can only host in your own data center, then these are rigid requirements that will eliminate a chunk of vendors right off the top.

  3. What is your rough price point? Lots of people say “I don’t know” or "I want the vendors to tell us what this costs," which is not entirely wrong, but it’s disingenuous to say you don’t have some idea of the ballpark you’re playing in. Is your project budget $10,000? $100,000? $1 million? And how much of that can you spend on software?

  4. Who is going to integrate the software? Do you also need services, or are you going to do this internally? If the former, do you need services from the vendor, or will you get them from somewhere else? If the latter, what technology stacks can your developers work with?

Simply answering these four questions will eliminate a majority of vendors. You can compare your answers and requirements to the information vendors provide on their websites and from other online information – it’s not hard to understand the basic nature of a system and figure out if it even remotely overlays on what you want to do.

However, if you still have no idea how to form a list, you can employ outside help.

Analyst firms exist to help organizations with technology questions – Gartner and Forrester are big ones, but there are hundreds ranging from enterprise to boutique.

Companies pay a monthly subscription fee to these firms for a set number of consulting hours – or even full “advisory days” – and access to the firm’s published research. Your organization might have one of these subscriptions, and you might be able to get an analyst on the phone, describe your requirements, and ask them to suggest a Long List of firms for you.

Alternately, these analyst firms often publish some of their research reports. Generally, they publish their flashiest report, which is usually well-branded and well-known in the technology industry.

For example, Gartner publishes what it calls the “Magic Quadrant” (known as "the MQ" or just “the quadrant”) and Forrester publishes the "Forrester Wave." These reports provide a very top-level ranking of vendors in a particular space (there are MQ’s and Waves for dozens of categories of software).

If one of the analyst firms isn’t an option, consider a limited engagement with a consultant. Several exist in the CMS space (Deane used to be one of them, before he went to work for a vendor). For a set fee, they can advise you on your selection process. A small engagement might be reviewing your requirements and helping you form a long-list. A more complete engagement would be to help you write the Request for Information (RFI), the Request for Proposal (RFP), manage the demos and a manage a Proof of Concept (POC), and finally assist in analysis and decision-making.

If you work with analyst firms like Gartner or Forrester, they can help here. And there are smaller, content-focused firms like The Real Story Group and The Content Advisory.

How long should your Long List be? Opinions vary, but let’s say 5-8 at most.

The Request for Information (RFI)

To your Long List, you’re going to send a Request for Information (RFI). This is exactly what the title says – we’re asking the vendor to answer some questions so we can decide whether or not to move them along in the process.

There is no set format for this, so know that when someone says “RFI,” they could mean almost anything. Generally speaking, you’re sending a multi-section document that both provides some information and requests some information.

First, you’ll provide some information around the following:

  1. Your organization: What do you do? Why do you exist?

  2. This project’s background: Why did it it come about? What are the major problems you want to fix?

  3. The selection process: What are the steps in the process? What is the timeframe for selection? Who is the primary contact person? When is the response to the RFI due?

  4. The technical environment: What is your plan for implementation – internal, partner, or vendor? Where do you plan to host the software?

  5. The scope of your implementation: How many websites do you need to run? How much content, in terms of objects or pages? How many editors will work with content? How many page views do you think you’ll need to handle?

You’re explaining this to give the vendor some context. Remember that vendors get a lot of these requests, and many vendors need to decide if the project fits them (they will “qualify” the opportunity).

Vendors might respond to say they won’t participate, or they might just not respond at all. This is a good thing. Very rarely do I see a vendor withdraw that should have stayed in the process. If a vendor bows out, it’s very unlikely they were the right choice for your project anyway.

In addition to providing information, you’ll also request information. The heart of the RFI is a series of questions you want answered about the vendor’s product. You can ask about anything, but these are some common questions:

  1. Where is your company located? Where do you have offices? What is your employee headcount?

  2. What are the technical specifications of your product? What is its hosting disposition? What supporting software is required?

  3. What is a rough price range for your product, given the sizing information we have provided?

  4. How is pricing structured? What are the different variables that might affect the price?

  5. What is a common total budget for a project like the one we have described?

  6. How is your software normally implemented? How many partners do you have? How many are in our geographic region?

  7. Do you offer training or certification programs for customers using your product?

  8. Do you have customer extranet or community? Do you have an annual conference?

  9. How many customers do you have in our industry? (Or other defining factor – geography, organizational size, project size, etc.)

  10. Explain how your product does X? (Where “X” is any specific feature you would like to know more about.)

  11. What integrations with other software does your product offer around Functionality X? (It’s best to limit these to a specific function – A/B testing or email campaigns, for example – or else the answer could be so long as to be useless.)

  12. What would our relationship be with your company after purchase? Will we have a dedicated account representative?

The goal is to find out enough information about the company and the product to decide if this is a vendor that will move forward, down the funnel to the next round. You’re not getting too specific here about your plans or your needs. This is a general, high-level summary to get some questions answered and establish contact and rapport with a vendor.

Give vendors enough time to return this – at least two weeks, and perhaps more, depending on the number or depth of your questions.

Some vendors will ask for a phone call for “discovery.” They likely want a chance to ask you some further questions, though some of them might be looking for an opportunity to make an early sales pitch. Some customers refuse all contact requests, which can be counterproductive. Sometimes this is done with the goal of treating all vendors equally, but so long as all vendors have the opportunity for further discovery, then let them ask for it. More communication and transparency can do nothing but help you.

The evaluation of the RFI response is how your Short List is developed. The goal is to cut the vendor pool from the aforementioned 5-8 down to 2-3 for the next round.

The Short List and Request for Proposal (RFP)

Once you’ve notified a handful of vendors that you’ve decided not to move forward with them, you need to prepare an RFP for the remaining vendors.

This document has the goal of fact-finding, like the RFI, but it gets more in-depth, particularly in two respects:

  1. You will provide specific usage scenarios you want demonstrated

  2. You will ask for a binding price quote

The response to the RFP will usually come in two forms. First, vendors will provide a written response with a price quote. Second, vendors will perform a demo of their product, either remote or occasionally onsite.

Usage Scenarios

The bulk of the RFP will be a set of scenarios. These are examples of your expected usage that you’d like to see in action. Here’s a very simple example:

Mary wants to create a new press release. She logs into the system and selects the Press Release content type. She enters a Title and a Summary. She selects an image from her local machine and uploads it to be added to the content. She enters several paragraphs of text and formats it using a formatting toolbar. She previews the content as it will be displayed on the website, and schedules it to be published tomorrow at 8 a.m. When she submits the content, it appears as an approval task for Brenda, who reviews and approves the content for publication. At 8 a.m. the next morning, the content is published and automatically appears in the index of press releases.

This is a very simple, mainstream editorial workflow scenario. It’s helpful for something like this to be the first scenario demonstrated, because in showing the functionality to complete the tasks described above, the vendor gives you a summary tour of the CMS itself.

Further scenarios can get more complicated. Things you might want to see:

  • Marketing functionality, like A/B testing and personalization

  • Localization support ("Mary also creates a Spanish translation of the press release.")

  • Content quality tools, like spell checking, link checking, and reporting

  • Collaboration support

  • Advanced workflows, like content annotation and processing multiple changes as a unit (“changesets”)

  • User management and permissions

Keep your scenarios to seven or eight, at most. Remember that each one will take 15-20 minutes to work through when you account for questions and follow-ups from a group. You could be in for hours of watching a screen if you’re not careful.

Be realistic about how much time vendors have to respond. If your scenarios require customization work, 3-4 weeks is a reasonable amount of time for a vendor to respond to your RFP, and prepare and schedule a demo.

Pricing

For the pricing section, you will need to provide whatever information that vendor needs in order to price. What is this information? Well, it could be anything.

As we discussed last chapter, software is priced on an infinitely variable list of factors, and each vendor is going to need different information. Some common ways axes of pricing:

  • Number of servers the software will be deployed on

  • Number of editors who will be able to use the software (either the number of editor user accounts, or the number of editors who will use the software concurrently)

  • Number of content objects under management

  • Number of websites the CMS will manage and serve (note: exactly what constitutes a “website” is very much up for debate)

  • Number of aggregate page views the websites will deliver during a particular timespan (by month or year)

Just be prepared that a vendor might ask for anything in order to come up with a price, depending on what their business model is. Several vendors will likely ask for another discovery call, which is fine. We believe that more transparency is better, and it’s tough to predict what information a vendor might need to price your solution.

Software Demos

As part of the RFP, you’re asking for the vendor to demonstrate parts of their system for you. You can do this remotely over desktop-sharing system, or on-site and in-person. Vendors will gladly get on a plane for the right opportunity, which normally means that it’s (1) large enough in pure dollars, and (2) one that they have a reasonable chance of winning (described as “well-qualified”).

Either way, the vendor is likely going to need to build out functionality for your scenarios. Be understanding of their ability to do this for your demo. They will likely customize their system, but know that this isn’t production-quality code, and some of the functionality you want demonstrated is too in-depth to build in advance. In these cases, vendors might show you a combination of actual demo and then talk through some of the more in-depth points, explaining how it would work in a production deployment.

You need to interpret and discern here. While it’s unreasonable for a vendor to build everything out, you can get a general feel for the level of effort the vendor is putting into it. You want to gauge how well they’ve actually read your scenarios and how well they understand them. We’ve seen situations where it’s obvious the vendor first looked at the scenarios on the flight in.

For your part, you need to have an evaluation team prepared for any demo. If the vendor is putting forth the effort, you need to make sure your team is assembled without anything else to do. There’s little more disheartening for a vendor to spend thousands of dollars preparing and traveling, only to find that six of the eight people canceled, and one of the remaining two spends the day with their head buried in a laptop.

Plan on 3-4 hours for each demo. You need a couple hours to get through the scenarios, then another 30-60 minutes to discussing pricing and follow-up questions. Sometimes it’s also helpful to send the vendor out of the room for 30 minutes to discuss their presentation as a private group, then invite them back in for any questions that came up during that discussion.

The demo is your chance to see the system in action, and ask questions about what you see. Don’t make this a one-way event where the vendor just talks at you. If you have a question about something the vendor shows you, now is the time to ask. If something is confusing, ask. If there’s an interesting looking menu option on the interface, ask. If you have a usage scenario and you want to know how the system might handle it, ask.

This demo is for you, not them. Some vendors might try to gloss over issues, or redirect your questions and concerns to an area of functionality where they’re stronger. This is not a naturally adversarial engagement, but be prepared to be assertive in what you want to see.

Verifying the Decision with a Proof of Concept (POC)

For many organizations, an RFI and RFP is enough to come to a decision, but some organizations want to go one more step further into a Proof of Concept. In this step, the vendor works with the actual customer team to implement some solutions to project requirements. It’s a deeper dive, where the customer can get hands on with the system and interact with the vendor’s team.

Sometimes, two vendors do a competitive POC (usually in consecutive weeks). Other times, the customer has chosen a vendor after the RFP round, and the POC is seen as a verification step for that choice. If the chosen vendor fails completely at the POC, then the customer backs up to their second choice and has them do a POC.

Here’s the thing about POCs – you need to be very serious about the project, and you need to have a big enough budget to warrant this.

POCs should be paid for.

We’ll say it again: POCs should be paid for.

Vendors can only invest so much in an opportunity without any return, so you need to be prepared to invest in your POC. Often, POC costs are proposed as a “kill fee” arrangement, where the vendor is paid if the POC doesn’t result in a sale. (Other times, the customer asks that the POC fee is credited toward future services. Whatever, same thing.)

Another caveat with POCs is that you need to be sure your team has the time to invest in it. If you’re having the vendor come on-site for a week, then you need your team to be liberated from anything else they have to do that week. In fact, it helps to have POCs off-site if possible, so your team doesn’t get dragged back into other work.

Think of a POC as a “super demo.” Refer back to our scenario example above. If we were to expand this to a POC, then the vendor would actually work with your development, editorial, and marketing teams to implement the press releases functionality, including:

  • Modeling the content with all validation rules and editorial interface support

  • Setting up the users and permissions

  • Defining and creating the approval workflow

  • Creating the templates

  • Training the customer on how the system works

  • Some hands-on content operations work (e.g. each person on the customer team enters two complete press releases in a workshop/classroom environment)

The goal is to get end-to-end on some segment of functionality. Ideally, the hands-on work would even reveal some shortcomings which the vendor can then address, which would illustrate how the system changes and evolves post-launch, and the attentiveness and responsiveness of the vendor.

However, don’t confuse a POC for actual production code. You might solve some logical questions and come up with some technical design and implementation patterns and ideas, but the code generated during a POC should always be considered disposable.

Decision Time

We’re calling this section “decision time,” but that’s something of a misnomer. By this point in the process, you’ve likely made multiple decisions (remember: a funnel gets narrower with each step), and the team has probably come to some informal consensus. You shouldn’t still be wildly undecided by this point.

Nevertheless, at some point you have to call the process to a close with a final decision.

Some advice on how to make this decision:

  • Does the system do the basics well? Vendors might tempt you with high-end marketing functionality, but don’t overlook basic content management and editorial usability. Do you enjoy working with the system? Does it make sense to you? Will it actually reduce editorial friction and enable you to publish content more easily?

  • Paradoxically, you need to balance your considerations between right now, and the long term. While you should look down the road and ask yourself how the system will grow with you over time, don’t get hung up on functionality you’ll probably never use. Systems can do a lot of things, and you like to think you’re gonna do it all ... but, are you, really? Be realistic about your needs, plans, and capabilities.

  • Pay attention to how you’re going to work with the system after launch. In particular, how are you going to develop the system once it’s live in production Are you doing this internally? With a partner? With the vendor? The answer to that question will drive other questions about ongoing support and services, which need to be resolved.

  • How well did the vendor describe the system? How well did they teach? If you felt rushed, or that the vendor skipped over a lot of important points, push them on this, or go with your gut and look elsewhere. You need to understand the system, and if a vendor is openly contemptuous of this need, then that doesn’t bode well for the future.

  • Consider the vendor’s ecosystem. Does it make you feel isolated, like you won’t have many options if things get rough? If you’re talking with a partner, are there other partners available? How much documentation is available? How many people are talking about this system on tech blogs? Is there an active community?

  • If your internal technical team is involved – for development, hosting, or both – make sure you have their buy-in. It would be great if you could choose a system purely on features and functionality, but there’s often an intersection with technical stakeholders. We’ve seen systems sit on the shelf for years because an internal team had no say in its purchase and is either unable to unwilling to support it.

  • Be careful on costs. You don’t want to buy a very expensive house, then never be able to afford to take a vacation. Make sure you will have sufficient budget left for the implementation you need.

When you do finally reach a decision, you’ll need to enter a formal negotiation with the vendor. Remember, everything is negotiable. If you need a particular price or pricing structure to make your project work, ask for it. Vendors can be flexible. They want to make a sale, and they’ll deal if they need to.

Finally, be prepared for the paperwork to take some time, depending on the procurement process at your organization. It’s not uncommon for vendor vetting and due diligence at larger organizations to take 30 or 60 days. If the vendor has a binding expression of intent from you, they’ll likely let you get started pending final paperwork, but be sure you understand their policies and know when you can actually take possession of a license or account to get started on your project.

So ... Do I Really Need To Do All That?

Yes, yes, we know – this is a lot.

Rest assured, not every project needs all of this. Larger projects with a wide range of CMS options will need something resembling this, but other projects can cut this process way down.

Some options:

  • POCs are probably the exception, not the rule. There are reasons not to do one. As we note, POCs should be paid, they take a lot of effort from your own team, and they push out your decision timeframe considerably. Think very carefully before you embark on one.

  • You might skip the RFI. If you start with a relatively Short List of vendors, you might jump right to a combined RFI/RFP, which is an RFP with some additional questions. You might send these to a slightly larger group of vendors (call it a “Medium List”), and explicitly state that you will invite vendors to demo scenarios depending on their response. You might only have one vendor demo.

  • If you have a consultant you trust, let them guide you. If you find a consultant who you think has enough experience with your type of project, then perhaps let them make a recommendation? Have them tell you what system they would use for your project, then have that vendor demo. You might find yourself comfortable enough to make a decision solely from that.

Note too that, for some projects, you might not even be able to do all this even if you wanted to. The process detailed above assumes you have a willing set of vendors with sales teams ready to engage in a personal selling relationship. This isn’t always the case.

  • Some systems simply don’t do individualized sales. Smaller, unattended systems (e.g. - Squarespace, Wix, WordPress.com) don’t have sales teams that will engage one-on-one. Consider that some of these systems are less than $100/month. No one is going to do a personalized demo at that price point.

  • Open-source software often has no seller. For someone to sell you something, they have to be able to realize some revenue from it. An open-source system you’re going to host and develop yourself has no such champion. Unless some organization has a revenue model you might consider and evaluate, then there’s no one who is going to demo it for you or sell it to you.

There’s just no process that fits every project, every team, and every mix of potential vendors. Pick and choose, and come up with a process that works for you.

Above all, keep an open mind. Seek to learn as much as possible. Even a system that ends up not being the one for you can teach important lessons about what’s available on the market, what works, and what doesn’t. Sometimes, seeing something you don’t like helps you clarify what you’re looking for.

Finally, know that there’s no one system for everyone. The truth is that your project will likely be equally successfully with more than one system. Remember this when and if you get overwhelmed by the decision.

Do your best. Good luck.

Inputs and Outputs

You need the requirements that we discussed in the last chapter, and the integration stuff from the chapter before that. Also handy – a site map, a content model, and any designs and wireframes you have. The output of this phase is a binding CMS platform decision, and a purchase agreement for that system.

The Big Picture

Clearly, you need a decision on CMS before you can start building your site, or even before you start looking for an integration partner. However, you can’t rush into this right away – you need significant site planning in place before you have enough information to make a solid decision. You need to thread the needle between those two things.

Staffing

As we discussed above, a consultant is pretty helpful here. You can hire one for the entire selection process, or perhaps find someone in the web content management space who would give you just a few hours to review your requirements and make some recommendations for your Long List.

Besides that, the core project team should be enough, so long as everyone is involved. Everyone needs to see the CMS demonstrated and provide their feedback on it. Considering the typical expense here, you don’t want to surprise anyone.

Resources

Articles

Books