In the software industry, the definition of "success" isn't necessarily the same as it is for the rest of the world. The customer asks for a complex system composed of many parts, with a specified budget, and a timeline. A software project is usually considered successful if any part of the system is developed, at any cost, at some time. Not necessarily the functionality requested, the budget, or the deadline.
So claimed Steve McConnell at a recent talk about the business value of software processes Timon and I attended.
Surprising? Shocking, even? Not if you've ever been involved in creating a moderately complex web site, or software project. All too often, people charge in and build something, never taking a moment to plan, never stopping to look at the big picture.
And, I'm embarrassed to say, I have been guilty of this too.
Here is what we're doing to make sure we don't do it again.
Rapid Iteration versus Engineering
There's a case to be made, most certainly, for charging in and getting something done, and then fixing it later. There's also a case to be made for meticulously planning a system before building it. These break into the two major camps of project management: Agile versus Waterfall.
With Agile development, the underlying assumption is you cannot know in advance what the system needs to do, until you can put a system in place and see how it's broken. It's far easier to apply software that already exists to a problem, and then identify what's wrong with it, what needs to change. If you go into a project assuming that the outcome is going to be wrong, you'll conserve cash to be able to fix it later. The downside to this approach is that with a bit more planning, you can uncover a lot of requirements, and keep out of quite a few possible dead-end approaches.
With a Waterfall process, development starts with a rigorous planning phase, followed by development, followed by testing, and then release. Nothing is ready to go until everything is. The problem with this approach is that it's big, risky, and can be completely derailed at huge expense by changing requirements.
A question of scale
In practice, if you look at a single cycle in an Agile project, it looks very much like a Waterfall process, composed of the same basic process: Plan, build, test, release, and celebrate. It's just that you focus on delivering the smallest possible project first, and plan for multiple projects. Let's take this to the physical world, and see how you might build a house, or a bridge.
If you were to build a house using an Agile process, the first project might be to lay the foundation. You can pitch your tent on that and have some assurance you won't drown in the mud, but it's not all that useful. Next might be to put up the frame, and from there it's pretty quick to add exterior walls and a roof. Now you can stay dry and sheltered -- but it's not really a home yet. If you were building Agile, you would then add plumbing, electric, and heating systems. Then you would finish out one room at a time. Talk to a contractor, and they'll say, well, that's how we build houses! How is that different?
The key difference between building a house using Agile, instead of Waterfall, would be that you would move in as soon as the roof is up, and start living there, while work was being done. The workers would have to work around you, and it would probably cost more in the long run, but you get the use of the house much sooner -- you're not out renting somewhere else. Maybe this doesn't make sense for a house, but it makes a lot of sense for a web application, especially if you aren't already living in another system.
If you applied Agile to a bridge, it looks even more silly -- while there are many individual parts to building a bridge, preparing the site, creating a foundation, building the spans, putting it all in place -- a bridge is completely worthless until you can cross it.
But if you change your perspective a bit and look at a different scale, you can see that Agile can be applied to an entire road system. You start by building a bridge in a city that will start using it the day it opens, and you build the rural bridges later. In this sense you get value out of the system, people can drive across the bridges as they are complete, before the entire project is complete. It took decades to finish the Interstate highway system, but people started getting the benefits of freeways in the first couple years. And maxing them out soon thereafter!
By the software industry's standards, the projects Freelock has taken on have been stunningly successful. The vast majority of our projects have launched, with some subset of the desired functionality, at some point in time, at some cost. We've had our share of problem sites, certainly, but we've built a business on taking over projects started by other people who have ended up walking away, and carrying them across the finish line.
Over the past 10 years, we have had a few sites fail, in that they never launched. Recently I analyzed these. A couple were killed by the customer because they no longer wanted to pursue the idea. The rest turned out to cost more than the client was willing to pay. In every case, the single reason we did not see these projects to completion boiled down to money -- to finish the job cost more than our customer was willing to pay.
Even that many failures is too much. So how do we prevent these going forward? The single key thing we can do is to verify whether or not the value to be gained by doing the project exceeds the cost of doing the work. What's the point of building a site that costs more than the site owner is ever going to see in terms of benefits? This is the favorite calculation of bean counters everywhere: what is the return on investment? How soon will this site pay for itself, in terms of increased business, lower costs, greater satisfaction?
It turns out that some projects are simply more valuable than others. For one prospective customer, we can save $30K a year on licensing costs alone -- even without calculating the time it will save its staff on manual data entry, we know that a $50K project will pay for itself in less than 2 years.
More often, coming up with actual numbers can be difficult to do. But if you're faced with a customer retention problem, having a system that helps you keep track of customer requests, set realistic scheduling expectations, and improve the quality of your service, that can lead to a huge amount of value in terms of happier customers who stay longer and refer more business your way.
To prevent failure, we need to make sure our customers have realistic expectations around cost, help them identify the true value of the systems we build for them, and help them squeeze the most possible value out of the systems we build. And whenever the cost exceeds the value, it becomes clear it's not a project for us to take on.
But we can do better than that. For years we have been trying to work on an Agile basis, keeping development cycles as short as possible, doing multiple iterations and growing sites organically. But there is more to Agile than lots of little iterations, and there are many ways we diverge from Agile, as well.
Agile describes a whole family of methodologies, and there are a number of techniques people refer to when talking about Agile development.
So while we've been trying to do Agile development for several years, we've spent the last few months refining that into some very specific practices, and it's come to define how we do business as a shop. Here's a peek under the hood, describing how we do it.
The very beginning of our process, we diverge from Agile. The very first thing we do is learn about the project, and create a plan. The Agile Manifesto values "responding to change over following a plan" but we find that having a plan is essential to success, even if you change it later. So the very first thing we do is come up with a rough outline of what the customer wants accomplished, what our starting point is, and at least an initial proposal about what we will do first.
If there is an existing site we're starting from, we make a copy of it and do some analysis on it to figure out if there are some risks specific to this project -- basically what the starting point is. If we're starting wholly from scratch, we identifying what base functionality, which collection of modules, and what architecture we're going to base the site on.
This inception process typically takes anywhere from 6 hours to 50 hours, depending on the complexity of the project and what we have to start with. We deliver a set of documentation describing the general architecture, the proposed underlying technical approach, an analysis of risk factors, and a roadmap identifying roughly the sequence of development.
Based on the roadmap, the next step is to agree upon a budget. The budget is not an estimate, or a commitment to deliver a set of features -- it is a commitment that we will devote a specified portion of our team's time and attention to the project at hand.
While everybody wants to know how much it's going to cost to get what they want, the reality is that with custom software, it is not possible to determine how long it's going to take in advance, until you've done it. And another reality is that your requirements for the software, by the time you use it, will almost certainly be different than you thought at the outset.
When discussing budget, we want to know that you have realistic expectations about what can be accomplished with $5K, $25K, $100K, or more. And we want to know what you're willing to spend over the time period we're discussing. That's how much time we will put on our work schedule for your project. We can give you some sanity checks on whether it's at all possible to deliver something useful for your budget, but the main thing we need to establish right up front is how much time to reserve for you, before we get into the details of what we're going to deliver.
Actual development gets done in sprints. These are budget- and time-bound mini-projects, typically running for 3 - 4 weeks per sprint, and anywhere from 20 - 60 hours per sprint. Within a sprint, we use a Waterfall process, starting with specific planning of the functionality -- and the most important part of planning is identifying the finish line. The sprint starts with the pre-determined budget, and a list of priorities from the roadmap. During the planning part of the sprint, we identify specifically how we will solve each of those cases, and scope out what we will actually deliver during that sprint. This is a collaborative process with our customer, involving fleshing out the details and identifying the criteria for success. Planning can take anywhere from 5 to 50% of the sprint -- in some cases, there's a lot more planning to be done than actual development!
Next comes development, executing the plan. We build out the functionality as described in the plan. That's almost the easiest part, if the plan was done well -- we find that the more time we spend planning, the less time is necessary for development and the more often we come in under budget instead of over.
After the functionality has been developed comes the Testing/Quality Assurance phase. We have someone else on our team review each case, and see if it meets the criteria for success defined in the plan. We also test the original user stories, look for ways of breaking the system that may not have been planned for, and in general identify things that might be improved in the future. And we let our customers in to see if it meets their expectations and provide feedback on the prototype. We conserve a percentage of the sprint budget for fixing issues that get discovered in this phase, and if no issues are revealed, might work a little further down the backlog of wishlist items, get more done than planned. If large issues are revealed, they get created as work for future sprints.
Finally we get to the release phase. In this phase, we wrap up the sprint. We document the reasoning behind major decisions, we re-adjust the plans to match what was actually released if necessary, we revise the roadmap with what has been completed and any changes of priority for future sprints. We release the code, and celebrate!
In between sprints, our account manager gets in touch with the customer, makes sure we're delivering, and sets up the budget and schedule for the next sprint.
As a team, we have anywhere from 6 - 15 active projects in any given week. Everybody has a list of cases to work on. It's critical for us to keep them flowing.
Every morning we do a stand-up meeting called a "Scrum." We go around the room and each person tells the group what they got done since the last scrum, what they plan to do next, and anywhere they are stuck or blocked. This quick check-in gives us an opportunity to prevent conflicts, make sure we haven't assigned the same work to multiple people, resolve things that are preventing work from getting done, and make sure everybody is on track with the plan.
Scheduling, and capacity
The biggest challenge of working on many projects at the same time, with many people on our team (we're currently at 10) is scheduling. We try to keep the time we can start a new project right around 2 weeks out -- we never want to run out of work to do, and it usually takes that long to close a deal anyways. But even so, many of our clients continue to spring new work on us without warning (and usually they want it this week, too!) This is why the budget becomes so important -- by having a firm budget commitment, we can fill up our calendars and determine how many hours to spend on each project each week.
We know what our capacity is -- our developers generally work full time, but not overtime. We under-book our developer's schedules, allowing room for projects to grow by a few hours here and there without throwing off our schedules -- and also to allow for training, contributing back to the community, and working on internal projects.
The challenge is figuring out the budget, when nobody knows what it will actually cost until we're well underway. The process of determining an accurate estimate takes time and research, and that planning can often be a substantial part of the overall budget. But not doing that means you're totally in the dark. That's why we start with the budget -- we will deliver the most value we can on your priorities, for that budget. We will spend part of that budget planning out the implementation. And if an estimate is important, we can spend more time developing an accurate estimate -- though in most cases, we already have budget set aside to do the work. Customers who need concrete, accurate estimates before committing budget for the work will generally have to wait longer for the actual project to get started, because coming up with these estimates is a project all by itself.
With agreed-upon budgets, and known capacities, I'm excited to report that after years of trying to get this working, in the past couple few months we've succeeded in wrangling our schedules into place, and we're finding our projects ticking right along, delivering what we've promised and more, within budget, on time!
That is our definition of success.
But what about quick requests?
Most of our clients stay on for at least months, and more often, years. We are in the site maintenance business, after all -- not everything is a project. While putting most of our work into sprints, we still have customers with very small budgets who just want to call somebody for help now and then. To be able to accommodate this unplanned work, we now have a request queue dedicated to these one-off requests. And we rotate the queue around our team -- each week we have one person designated to work on the request queue, instead of planned work. These tasks are simply done on a first-come, first-served basis, in the order we can get to them.
This also provides us some capacity for taking on emergency work, something we've done a lot of in the past, and it's been an important source of customers for us. If somebody is willing to pay our emergency rate, they get bumped right to the front of the queue, and it doesn't otherwise impact our schedule for planned work.
It's a whole new business
How can you make what you do better? That's the question we started with, and we've been coming up with new answers ever since. While I established the general guidelines for how we do business, it's really been my team that has led the way in creating a system to make it all work. Timon in particular has spent a lot of time hammering out the process, and getting everybody on board. We have developed a disciplined approach to getting results, and consistently delivering value to our customers. There's still plenty of room for improvement, but we're very excited to see how the hard work we've spent on improving our processes is already starting to pay off!
I would love to hear your comments about what we're doing, how this might apply to your business, or what you've done to make your business deliver better results!