Agile Software Development and Mormon Doctrine

I’ve been an Agile proponent for over eight years now.  “Agile” is  an approach to software development based on lean manufacturing practices.    It’s very interesting and there is a lot of philosophy involved.   Most interesting to me, however, is the parallels I see between the tenets of Agile and what Joseph Smith revealed about the history of the earth and the purpose of life.   (This short essay has wider applicability than just software development, so please bear with me.)

First, let me describe a little about agility and it’s converse, waterfall.  Both of these are software engineering practices, but they have nearly opposite approaches.    The basic strategy of agility is to push authority down.  An agile team has a leader, who acts mostly as a coach, and the ordinary people are allowed to make their own good decisions, even if they are mistakes.  The theory is that when operating as a group, the effectiveness increases when individuals can act autonomously.  The basic strategy of  Waterfall is to push authority up.  Control is concentrated at the top where the leaders have the decision making power.  The ordinary people doing the work are encouraged to follow directions.   The theory is that limiting control to just a few smart people benefits the entire project through consistency.  Agile teams learn through individuals making their own mistakes (and  successes) and then talking openly about them.    Waterfall teams learn by having information disseminated from experts to novices. 

Steve McConnell gave a keynote address recently where he talked about “The 10 Most Powerful Ideas in Software Engineering.”   His comments do a great job of summing up what the Agile community has valued all these years.   His first comment is very profound and it reflects what I have always thought and felt about Agile development:

                Software Development Work is Performed by Human Beings

Many interesting conclusions can be drawn from this statement, so I’ll use it as a springboard for my thesis here.   It goes without saying that all work is performed by Human Beings.  Sure, there are robots, but the humans are the impetus.  Now,  the trouble we face, as practitioners of better engineering processes in general,  is overcoming human weakness.   It is not hard to get people to nod and agree with the good manufacturing/design philosophies.    Like David Parnas says, great engineering ideas aren’t on the horizon, they have always been with us and have been with us for years.  In fact, I believe they’ve been with us for thousands of years in the form that God has given to man by way of revelation and enlightenment.   But as with all gospel principles, they are easy to teach, but it has ever been difficult to get people to live them and apply them.

Think about it.  Because humans are always part of what we are doing, any process you or I might create, all of our best plans, will be assaulted by the usual human factors:  pride, selfishness, politics, evasion, laziness, group think, etc.    Overcoming these problems is extremely difficult and there are some shortcuts that appear promising and leaders are tempted to take them.  These shortcuts always involve the leadership “taking control”.  You see, software developers adopt waterfall not because it is better, but because it is easier.   Agile requires individual accountability on everyone’s part.  Waterfall puts the accountability in the hands of a powerful few, or worse, in the hands of a powerful committee. 

As a Mormon, I’ve notice from the beginning that the ups and downs of agile practices have a lot in common with the ups and downs I’ve had as a teacher of spiritual things.  Joseph Smith revealed that all of humanity lived with God before the Earth was created, and that there were two plans presented to glorify god’s children- one presented by Jesus and the other by Satan.    Satan’s plan is basically the waterfall model.   “If I could just have tight enough control”, he says, “I could give you everything you want.  I’ll save everyone and I’ll have the glory.”    The alternative presented by Jesus can be seen as agility at its best:   “I’ll give everyone complete agency.   Line upon line precept upon precept, you will gradually learn from your own mistakes, but the result if you do so will be something glorious.”    Satan’s plan is industrial.  God’s is organic.    People gravitate toward’s Satan’s plan because it is the path of least resistance.   Jesus’ plan, like Agile processes, require people to do pesky things like talk to each other, reach a concensus, be nice, and be accountable.   Agile is delicate, fragile, alive, easy to kill, and constantly changing.   Waterfall is a dead cinder block building- it has staying power and a high resistance to change.     The ultimate irony here, is that in spite of the promises for something better, waterfall processes squash happiness.  When it comes to how they feel, people *hate* waterfall processes and love Agile processes to the point of zealotry.   Case in point:

Search hits for “agile zealot” – 365
Search hits for “waterfall zealot” – 2

If waterfall is so great, why don’t people talk about “waterfall zealots?”  Instead, people talk (and often complain) about “agile zealots.”  The question to ask is, what fills them with such zeal?      The same question is asked about people who have enthusiastic about their religion.   Mormons get this a lot.   People often ask, “Why do Mormons have to send out so many missionaries?”   It’s largely because we feel strongly about what we have. 

So now let’s think about this more broadly.  This fight (agile vs. waterfall) is happening all around the world in all segments of life.  Franchizes vs. Single-owner establishments, corporations vs. mom and pop shops,  totalitarian governments vs democracies, standardized testing vs. individual excellence, and on and on.   The battle between Jesus’ plan and Satan’s plan is constantly at the fore.  If you are preaching agility in any kind of an organization, you are basically preaching the gospel.  And because you are doing this, you will basically have similar results to missionaries who do the same:

Matt 13:3-8  
3 … Behold, a sower went forth to sow;  
4 And when he sowed, some seeds fell by the way side, and the fowls came and devoured them up:
5 Some fell upon stony places, where they had not much earth: and forthwith they sprung up, because they had no deepness of earth:
6 And when the sun was up, they were scorched; and because they had no root, they withered away.
7 And some fell among thorns; and the thorns sprung up, and choked them:
8 But other fell into good ground, and brought forth fruit, some an hundredfold, some sixtyfold, some thirtyfold.

When teaching sound practices that consider the needs of people, your efforts will most often be rebuffed, but when the message is accepted, it is always worth the effort. 

The Black Swan and marriage

A friend and I have been talking about the Black Swan book and applying these concepts to marriage.  The idea of the black swan is that it is the outlier events that make life interesting and unpredictable.  Leaving room for outliers is an important investment if you want to be open to big dividends. 

So Here's a thought.  Use the stock market as a metaphor for a spouse.  Your personal capital (happiness, unity, joy, etc.) is the result of investments placed in this market.  Some investments are safe and will pay off a predictable dividend.  (Keeping your job, taking out the trash, making dinner, ...)    There are other investments which seem safe, but actually do poorly because of idiosyncrasies of the underlying market. These would be things like giving a gift to a spouse whose love language is encouraging words.  

But what about investments that pay off 100 fold?  In this light, a black Swan husband would be wise, after spending 95% of his capital in the safe investments (routines he establishes), to spend the other 5% he'll in crazy, new untried “stocks”.   Just like a financial investor who is getting creative, he does not want to do this in a random, haphazard fashion. No, but he does need to open up his mind to try an "interesting" investment.   I think this might have to do with paying attention to the tips and hints dropped by the wife.  These are clues for possible investments, but one has to be alert for them.  It might also have to do with reading creatively to get new ideas to try.

One interesting thing that I am observing is that both good and bad investments have the dividend of information.   If you try to bringing home a giant box of chocolates for your wife and it totally flops, chances are you will get to have a conversation in which she'll tell you something about herself that you never knew before.  Same thing if you pick something that works well.  The husband learns something new that makes him a better husband.  

Now for practicalities.  What does 5% mean in a relationship?  If we look at waking hours, that's about 5/week. So the Black Swan says that if we want to take advantage of outliers, we should spend a good chunk of at least one day every week trying something new with our spouse. I find that number very interesting because leaders in my church have long emphasized the importance of "dating" your spouse once/week.  What would happen if I looked at weekly dates as a way to try a new kind of investment?  Maybe we could discover that we both love bowling, or poetry, or going to art museums, or racing radio controlled boats in a local pond.