Is Agile Really Better than Waterfall?

The simple and short answer to this question is "No".

There is no single methodology which can claim to be a panacea for the problems faced by software projects.

Evolution of Agile

Agile methods like XP, Scrum, FDD etc. have evolved based on the principles enshrined in the Agile manifesto which supposedly came into being as a result of frustrations experienced with the Waterfall method.

Agile has become quite fashionable since the last few years and as the die-hard Agile fans would love to hear "Agile takes care of all that is wrong with the monster named Waterfall".

However, reality is far removed from this statement.

One thought that often comes up is related to Agile and Fr(agile) and does accept the fact that Agile has its own set of challenges and shortcomings just like Waterfall or any other methodology.

In fact, in many ways Waterfall is much better than Agile and as some might say Waterfall Model, Original and Forever.

The "Waterfall"  Concept

Waterfall is based on the concept of staggering as it advocates starting an activity only when its precedent activities have been completed.

Agile is based on the concept of parallelism as it advocates performing all activities in parallel without bothering about their precedence relationships.

Agile approach is better when the product or technology is a new or innovative one and changes to try out several permuations and combinations is budgeted in its cost.

However, no organization can provide infinite budget for any project and hence projects are forced to do upfront planning to the extent they  have visibility.

On this front waterfall wins hands-down.

Agile says changes are welcome.

Waterfall also says changes are welcome but adds that changes don't come free of cost.

This is a good incentive for customer and developers to think through and ensure the high-level requirements and overall architecture is laid down before coding and testing.

The Agile Concept

In Agile method estimation, planning, requirements and design documentation, test planning, meetings, status reports are viewed as overheads.

And the focus is on delivering the working software to the customer day in and day out (or rather sprint in and sprint out).

It follows from above that Agile is focused on coding, testing and bug-fixing.

Code is continuously spewed out by the project, tested against the requirements and changed either due to design and implementation issues or due to changes in requirements.

Code changes are effected through re-factoring which essentially amounts to re-design and re-coding.

In this process the overall architecture is not focused upon and evolves along with the requirements.

This may lead to weak architecture and impact non-functional requirements like performance, scalability etc.

Waterfall versus Agile

It can be concluded that Agile seems to focus on the short-term at the cost of long-term.

For complex projects done by big, distributed teams Agile requires much higher rigor and discipline that what is advocated by the Agile purists.

In such situations Waterfall is a better choice.

It is also a fact that no one uses pure Waterfall.

Intermediate deliveries have been a common practice in case of Waterfall projects as well.

In the end, Agile seems to be more of a marketing ploy than just a well-grounded method.

It attempts to make a virtue out of lack of discipline by suggesting that trying to get upfront clarity in the requirements and approach in the beginning, to the extent possible but limited by visibility and constraints, is futile as they will change.

Thinking hard and having a good plan while remaining open for changes to the plan is a better stratgey than the above. 

Since, as is commonly accepted, well begun is half done.

In conclusion, it can also be said that "well planned is half execcution done"!