Risk Assessment in Software Projects: Part 1

A project on the brink of falling apart is the night-mare of every project manager. Either by mare misfortune or great-fortune, it just so happened that every project I have worked on until now inherently had a high risk of not-making-it-to the finish line from day one. Misfortune: I have lost all compassion for a failed-project. Great-fortune: It made me aggressively result oriented, “the show must go on”. I have heard the expression that strictly goal-orient people don’t tend to focus on the path of journey. Whoever said this could clearly use some experience in running a high-risk project. When the stake in hand is large you would need regular check points to assess the health of your project so that you can do what it take to bring the ship back home. One of the matrix of measuring project health is measuring the risk the project during multiple check-points. Question is what is risk?

Defining Risk

ISO 31000 defines risk as “effect of uncertainty on objectives”.

… uuhmm … okay … what does that mean? I interpret uncertainty as the probability of something happening and effect is the negative impact on your project. Actually it can be positive impact which is also known as opportunity. That means for a certain objective

risk-assess-1.1

For all the objectives associated in a mission

risk-assess-1.2

Assessing Risk

Quantifying risk is not a trivial task because essentially risk is something in future that may or may not happen. Measuring the negative impact is also more of a perceptive value looking out from the present. In the world of financial risk assessment the monetary impact is usually known ahead of time and the probability of an event happening can be calculated from the massive base of historical data available out there. In the case of software development the exact monetary impact is not always apparent and with all the new evolving technologies the historical data is not exactly available or applicable. Assessing risk in a software inclines more towards the project manager’s experience and perception rather than all concrete data.

In this article I am going to focus on software projects where developers and testers are onsite or collocated. In other words a detail breakdown of the engineering process and tasks are available along with the schedule and task owner. Although the principle is similar the out-sourced projects or project that has multiple dependencies on the engineering partner companies do not necessarily have the similar visibility into detailed schedule and task ownership to make some of the assumptions I made here (don’t worry I’ll talk about that too in the following articles)

Project Breakdown

Be it a completely new software or be it a new iteration for improving an existing software the breakdown is usually same across the board: There are bunch of top level features and engineering process tasks (like setting up or re-configuring the build system or release server). Then those features are broken down into smaller ingredient tasks including both developers writing code and QA engineering testing the content.

Assessing Impact

I start the impact assessment by first categorizing the features in terms of deliverable priority. Clear understanding of the product requirement is crucial in determining the priority. In my world I have three kinds of features

  • P0 – Ship Blocker: If this feature is not implemented then we can’t ship this baby. It could be because this feature is all your software has or you are doing this to prove that your product can catch up to the competitors.
  • P1 – Seriously Compromised Functionality: You can still sell your product or meet your major commitments but might not provide the premium user experience. For example, as far as I remember the first Visual Web Developer was released without a debugger.
  • P2 – Nice-to-have Features: These are features that will clearly make some difference but not likely to damage your brand if the feature doesn’t exist.

One thing to thing to keep in mind is that if there is dependency chain between the features like you can’t start writing the UI code until the UI Framework is ready. If so then the logical thing would be not assign lower priority to a base feature while the feature(s) depending on this are high priority. Just off-topic, through-out my experience as developer, as well as technical lead I have seen feature priorities getting “readjusted” in the middle and near the end of a project! Clearly there was significant lack of understanding the product requirements. After I started managing projects on my own, this is something I have been consciously trying to avoid.

For the ingredient tasks composing a feature I prioritize them similarly in three catagories

  • P0 – mandatory task: If this does not get fully implemented then it will tank the feature. Developer writing the code functional code belongs to this category. (dev task)
  • P1 – very important tasks: If this does not get fully/partially implemented then the feature experience will be crappy (you might get hammered in the review but you can still make the feature work). I usually view test task as P1 (I’ll explain later why it is not P0)
  • P2 recommended tasks: If this does not get implemented then you can still use the feature but might have to pay otherwise, like not having proper design pattern for coding will put serious friction in future extensibility and enhancement or doing poly check for legal concern.

Quantifying the Impact

As I mentioned before that risk analysis might require you to pull off a quite a bit of judgment call here and there. I personally see each P0 feature as impacting the entire project individually all by itself. So if you know that you overall project/product/company (in case if you are a start-up and the product is all you got) capital then not having the P0 feature will cost the capital in question. Quantifying P1 and P2 features are not so straight forward unless you know exactly how much you are getting paid for each of those features. In my cases there were no such clarity. That is why varying from project to project I have chosen different ways to quantify the P1 and P2 features. One way of looking at it could be that, all the P1 features together might sum in half of the entire capital. If that case if your project capital is a $1000000 and you have 5 P1 features then each P1 feature has the impact of $100000. The concept is similar for P2 features unless you know how much each of these features will earn you. I usually see the nice of have features all together having an impact 1/10th to 1/5th of the product/project capital which means 5 P2 features would have money impact of $20000 – $40000.

If you do not have visibility to the exact or direct monetary impact of your product then you can simply assume the value 1 as your entirety. (I have worked on products where they were to be distributed for free so that the platform they run on get sold more). In this case each of your P0 features would have impact of 1, each P1 would have impact of 0.1 and each P2 would have impact of 0.02 – 0.04.

Quantifying the impact of features’ ingredient tasks are also similar. Every single the P0 task would have impact of 1 on the parent feature (not the entire project). For P1 tasks, as I specified, in my world, I the feature’s end to end testing is one big P1 tasks of impact 0.75 – 0.9 which means all the composing mini-tasks all together measure up to that value. This value depends on the developers’ familiarity with the technology, the developers’ experience in similar line of product. The better the dev code is expected to be the lower the number will be. P2 tasks are similar, just with a lower total impact, somewhere 1/4th of the entire feature. Note that this is very context sensitive. If you are only P2 task in that feature is poly-check most likely there is no reason to have an impact more than 0.1 or less assigned to it.

Again just to re-iterate my point, there is no better way than to know the exact impact value of every single individual task. The numbers I am providing are heuristics only when your entire project is a big blurb of cash value and that might be the only info you have in terms of impact.

Notice that when you sum the impacts of all the tasks for a feature or the impacts of all the features on your project it might and most likely will be larger than the value of your projects entirety. This is basically my way of saying there is no compromise with P0 features and tasks.

Probability Calculation

Before calculating the probability let us take a step back and think what are the common things that might go wrong in our projects –

  • May not have enough time to finish the task
  • Dev/test “might” leave and there is no back up (eeek!)
  • A dev task is blocking validation or another part of coding

Now let us talks about the odds of those –

  • Dev/test “might” leave: This should not be left on probability. As a project manager you should know who is leaving and who is not. Let’s find that out and let’s get rolling. THE SHOW MUST GO ON!
  • A dev task is blocking validation: This should not be left on probability either. As an engineering the design dependency should be pre-known to you. If this is about finding out unknown pitfalls of the design then figure out what has been the chances that these kind of pitfalls have been discovered and how familiar the team is with the technology.
  • May not have enough time: The most common thing I hear every day and likely to be the most common issue across all software projects. I am going to focus on this one, finding out the probability of slipping the schedule

Chances of Slipping the Schedule

There is almost no way of assigning a perfect probability of these things happening. The formula that I use for probability calculation where a task is going side-ways is the following –

risk-assess-3

So if your original estimate for a task was for the dev task was 10 days and after 7 days only 40% was done then your probability of slipping the schedule is

risk-assess-2.1

After 9 days if you completed just 10% more (reaching to 50% completed task) then your chances of slipping the schedule rises to

risk-assess-2.2

It is just a heuristic measurement. This is not the best formula in town but it sure does give you an easy way to quantify your impending doom!

What does it really mean to slip a schedule? For lot of start-ups that means finding a job in taco-stand. For larger firms the story is differently. In fact for firms that can withstand schedule slip on products calculating the impact is a whole different beast. It depends on your relationship with your customers/engineering partners, your desperation for kicking ass of your competitor products, how much you would need those reviewers to say something nice about your product. That list can go on and on. Frankly enough I see the world of schedule slip as one big yes-or-no question (at least in my line of work there is no mercy).

Consolidating Risk Calculation

Now that you know how do you calculate the impact and the probability the rest is pretty straight forward –

risk-assess-4

How to Interpret the Risk Value

I like to categorize my risks into 3 buckets:

  • High Risk: Risk value is greater than 80% of your project capital
  • Medium Risk: Risk value is between 40% – 80% of the project capital
  • Low Risk: Anything below 40% means that this is a low risk situation and there are there are other higher priority tasks you can focus on.

In fact I have been in one project where the risk was more than 350% of the project entirety. I made a whole new bucket for situations like that. I call it the “Tony Montana” High Risk: “say halloo to my li’l friend”

Advertisements

2 thoughts on “Risk Assessment in Software Projects: Part 1

  1. Pingback: Risk Assessment in Software Development Project: Part 2 – Tips and Samples | Soumit's

  2. Pingback: Risk Assessment in Software Projects: Part 1 | Mindfulness

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s