What Makes The IT Market Want to Scream?
According to Roger Sessions, the enterprise architect and CTO of ObjectWatch, “the total annual cost of worldwide IT failures is $6.2 trillion dollars. According to the 2009 U.S. Budget, 66% of all Federal IT dollars are invested in projects that are “at risk”. A large number of these will eventually fail. I assume that failure rate to be around 65%!”
I have seen the lists of reasons for project failures from so-called experts, floating around in the industry. None of them mention the primary reason, the missing link between Requirements and Design. Today’s approach to system building, supported by modern methodologies, is akin to a builder scribbling an owner’s ideas about their dream home on a paper napkin and basing the integrity of each stage of construction on his client’s approval.
The panacea for these ills is to come up with new methodologies. Agile and Extreme Programming come to mind. They are the Buzz in today’s jaded IT marketplace. They are also the answer to every technician’s prayer. Real technicians don’t eat quiche..... and they don’t document! The average IT technician wants to be left alone in his cubicle where he can hold a one-man brainstorming session and code the results into latest and greatest code, while he empathizes with the Users’ plight and wants to do battle to promote their interests without giving a hoot in hell about the project timeline, project costs, delivery schedule and quality of deliverables.
He probably could get away with that approach in a government environment where the money supply is endless right up to the time the economy hits the floor and project failure rates of 66% are given scant attention. I have been aware of projects in the government sector that have gone on and on for 4 years with no end in sight, with a project team size of 1 technician.
Not so in private enterprise. In this sector it’s a different kind of chaos. It’s a huge stage play complete with prima donnas, work horses and social directors propping up the “Team Player” concept. Every human endeavor succeeds when there is one central idea to drive it. A single central idea has integrity. To help the reader visualize the concept, compare a streamlined skyscraper to the layered wedding-cake-layered abominations in Washington DC in which every architect promoting the Greater Good has seen his contribution added to the distorted concrete.
That’s the preferred approach to developing software systems in the private sector today. Agile (an abject misnomer) and Extreme Programming have unwittingly given the technician the weapon to wreck such havoc. With these approaches, the Technical Project Manager is a thing of the past, replaced by the Scrum Master with a “Black Belt” certification who hands out iterative chunks of “work”. Each iteration gives the technician as much slack as he can expend, in the 2 or 3 weeks it takes each iteration, to make umpteen visits to Users who magically drum up more and more “requirements” which the technician heroically fights to implement, without any concern for Scope Creep and budget blowouts.
Agile is driven by the define/build/test cycle, but it's the Definition step that makes or breaks an Agile project. The Construction and Test activities can be executed in bite-sized chunks, but it would be disastrous to apply that approach to the Definition. And why? Because during a demo of the results to Users there is always a concern that the Users may want more or simply change their minds on how they would want to see a process executed. The technician on the other hand will not be aware of stretching the timeline which is not apparent because of the bite-sized effort. It's the "quick" and constant "adjustments" to the Scope that delay projects and cause budget overruns.
At the Iteration Planning stage there is prioritization and estimation of tasks and negotiation of Scope. Scope should not be negotiated at a miniscule, task level. Iteration begins with expanding/elaborating the Requirements. How does that happen? What's the deliverable from such an activity? There is none. From the "expanded Requirements" the team plunges into the Design phase after which the Coding begins followed by a Build based on the Release roadmap, followed by Testing. And all the while there is tracking and adjustment at the Task Level!
What’s missing in software development is the creation of the Functional Specification (System Blueprint), the vital and logical next step after Requirements and before Design. That's the deliverable that should replace the Expanded Requirements activity and it should be created, assembled and ready before the Iteration planning begins. It has a specific format and calls for the inclusion of certain artifacts. The format lends itself to compartmentalization of the system and therefore the work, into Logical chunks. The Iteration steps mentioned in the above paragraph can then be applied without fear of Scope Creep, because the Users and Sponsor will have signed off on the Functional Specification, i.e. quantity of work to be done.
· 65% of all IT projects fail including Federal projects. That’s 65% of $6.2 trillion.
· Missing system blueprint
· New buzz “methodologies”
· Evangelism & Social Justice or “It’s only money - Give Them What They Want”
But everything has its use and Agile is no different. It’s a methodology that can work wonders in maintenance environments if it’s accompanied by an effective Change Control system.
We on the other hand decided to use both Waterfall and Agile methods in developing and delivering our projects. We use the Waterfall approach during the Initiation and Planning phases. The all time major deliverable from the Planning phase is the Functional Specification, the System Blueprint. We create it in such a way as to be able to chunk it into “iterations” in the Design and Code Construction phases by using the Agile approach.
There are 2 major benefits of this approach:
1. Users, Testers and Trainers are involved in the creation of the Functional Spec very early in the project and Requirements and Business Processes are identified and recorded cooperatively in JAD sessions with eager stakeholders willing to signoff
2. The Design flows easily from the Functional Spec, which is chunked into logical iterations that coincide with the Design. There is no need for technicians to visit Users during each iteration since all the Requirements and Processes have been identified and documented; therefore no Scope Creep and no budget blowouts.