Post-implementation reviews are not the only way to measure success

This is a guest post by Elizabeth Harrin.

Most projects have a post-implementation review (PIR), also known as a project post-mortem or post-project review. This is often the only opportunity to assess success on a project, especially if your organization doesn't have a more robust method of benefits tracking over the long term.

Often the stakeholders are involved in a PIR. You may or may not choose to involve suppliers as well. Normally the whole project team will be asked to contribute, either in a big workshop-style meeting or in a series of smaller sessions.

PIRs mainly cover process things, like how we managed schedule changes as a team, or whether our monthly reporting schedule gave everyone the information they needed. Aside from the process topics during a PIR, it is also an opportunity to discuss statistics and metrics related to the project. These are normally backward-looking. What was the percentage of effort spent on testing? How many days did it take the quality team to audit the deliverables? These metrics and calculations can then be incorporated into future projects so that initiatives going forward have the benefit of experience and hindsight.

This is great, but this type of PIR doesn't help the project stakeholders. After all, for them, the project is over.

So why do we do post-implementation reviews?

Traditionally, a post-implementation review is the only way that project managers have to determine whether or not their project has been a success. You set success criteria at the beginning of the project and then at the end you pull them out of the drawer and have a meeting to decide if you hit them or not.

We do PIRs because it’s a way to assess performance and to gather data, and typically we look at retrospective measures.

We do them because we’re in the habit of doing them, and because our methodologies and standards say that we should.

What are the limitations of PIRs?

For me, there are two issues with PIRs: they only happen at the end of projects and they mainly focus on the project management principles and methods used. They don’t make the distinction between the success of the project and the success of the project management effort, and they mainly focus on the latter.

Sometimes customers will be asked to feed into the project evaluation process, but at that point it is too late to do anything practical about their comments. If they complain that they weren’t kept up to date, you cannot go back in time and provide more information on a regular basis. It is a case of, ‘How can I help you now it is too late?’ In fact, research from South Africa shows that project sponsors prefer a proactive approach to feedback over the post-implementation review process. They chose to work collaboratively with the project manager during the project to ensure that their expectations were met.

Of course, PIR discussions are immensely valuable for continuous process improvement, and I am not suggesting that you stop using this technique. Focusing on project management principles and methods used is essential to improve organizational project management processes. Could we have done better risk management? What scheduling lessons were learned? A good PIR meeting should discuss what went well and what did not go so well with this project, and this is great information for project managers and teams. If you don’t cover this stuff, you won’t learn how to do things better next time.

So, while PIRs have their limitations, don’t stop doing them! I suggest you start doing something else as well.

Continuous reviews are the future

I would advocate continuous reviews. Review project success on a monthly basis with the main stakeholder or key stakeholders. There’s a lot of talk about ‘engagement’ but not a lot of practical advice about how to actually do it – sitting down with your stakeholders and talking to them regularly is an easy (and cheap) way to build engagement.

It’s just about talking to people, so it doesn’t take up much time, but continuous structured reviews do give you data in the same way that PIRs do. On top of that, talking to them also gives you data you can actually act on, so you avoid this ‘how can I help you now it’s too late’ problem.

You can ask people to score how you are managing their projects on a scale from 1 to 10. You can ask them the same set of questions month-on-month so that you can see if you are doing better. You can monitor their feedback over the long term, tweaking your project management approach and the project deliverable themselves, so that at the end of the project the stakeholders actually get what they want, in the way that they want it.

That’s got to be better than saving all the feedback until the end, don’t you think?

Elizabeth Harrin is the co-author of Customer-Centric Project Management (Gower, 2012). She writes the award-winning blog, A Girl’s Guide to Project Management (www.GirlsGuideToPM.com) and is Director of project communications company The Otobos Group.

(Pic Courtesy: E. Harrin)