The Future of Programming - Adopting The Functional Paradigm?

The previous post covered some thoughts on the future-looking programming themes present at OSCON 2014.

Following that wonderful conference, long-time Open Source advocate, Pythonista, and instructor Steve Holden, was kind enough to host his third annual "OSCON Survivors' Breakfast" with tens of esteemed attendees, speakers, and organizers enjoying great company and conversation, relaxing together after the flurry of conference activity, planning a leisurely day in Portland, and – most immediately – having some much-needed breakfast.

The view from the 23rd floor was quite an eyeful, and the conversation ranged across equally panoramic topics. Sitting with Alex Martelli, Anna Ravenscroft, and Katie Miller, the conversation inevitably turned to thoughts programmatical. One thread of the discussion was so compelling that it helped crystallize this series of blog posts. That was kicked off with Katie's question:

Multiple points of discussion spawned from this, some of which still continue. The rest of this post explores these. 

What constitutes a large company? We settled on discussing Fortune 500 companies, which, by definition are:

Afterwards, I looked up the 2013 top 25 tech companies in the Fortune 500. I've listed them below; in parentheses is the Fortune 500 ranking. After the dash are the functional programming languages used on various company projects – these are listed only if I have talked to someone who has worked on a project (or interviewed for a job that used the language), or if I have read an article by an employee who has stated that they use the listed language(s) [1].

The companies which havecommitted to projects guessed to be of significant business value written inFP languages include: Apple, HP, and eBay. Possibly also Oracle and Intel. So,a rough estimate of between 3 to 5 of the top 25 U.S. tech companies have madea significant investment in FP.

Why notGoogle?

The next two sections offer summaries ofsome views on this.

Is an FP language suitable for largeorganisations? Are smaller companies better served by them? During breakfast,It was postulated that dealing with such things as immutable data, handlingI/O in pure FP languages, and creating/using higher order functions is easierfor small startups due to the shorter amount of time required to hire or traina critical mass of skilled programmers.

It iscertainly true that it will take larger organisations longer to train itspersonnel simply due to sheer numbers and, even with enough trainers,logistics. But this argument can be made for any corporate level ofinstruction; in my book, this cancels out on both sides and is not an argumentunique to hard topics, even less, specifically pertinent to FPadoption.

I've heard this one a bit: "Some peoplejust don't think in FP terms." They need loops and iteration, not higher orderfunctions and recursion. Joel Spolsky makes reference to this in his articleTheGuerrilla Guide to Interviewing. In particular, he says that "For somereason most people seem to be born without the part of the brain thatunderstands pointers." This has been applied to topics in FP as well asC.

To be fair, Joel's comment was probably madewith a bit of lightness and not meant to be a statement on the nature of mindor a theory of cognition. The context of the article is a very practical one:hiring. When trying to identify whether a programmer would be an asset foryour team, you're not living in the space of cognitive theory, rather youinhabit the realm of quick approximations, gut instincts, and fast, economicaldecisions.

Regardless, I find this perspective –Type Physicalism [3] – fairly objectionable. This is because I see it as akind of intellectual "racism." Early social sciences utilized this form ofreasoning to justify all sorts of discriminatory thinking in the name of"science", reinforcing a rigid mentality of "us" vs. "them." In my ownexperience, I've seen this sort of approach used to shutdown exploration, toenforce elitism, and dismiss ideas that threaten the authority of the statusquo.

Rather than seeing the problem of comprehending FP as aphysical limitation of the individual, I see instructionalfailure as the obstacle to overcome. If we start with the proposition thatcertain brains are deficient, we are essentially abandoning education. It isthe responsibility of the instructor to engage creatively with each student'slearning style. When adhering to the idea that certain brains are limited, onediscards creative engagement; one doesn't even consider working with thestudents and their learning styles. This is a view that, however implicitly,can be used to shun diversity and dismiss potential.

I believe the essence of what Joel was shooting for can beapproached in a much kinder fashion (adapted for an FPdiscussion):

None of us was born knowing GOTO statements,global state, mutable data, or for loops. There are many programmers alive, though, whosefirst contact with programming involved one or more of these. That's their"home town", as it were; their programmatic birth place. Having utilized – aswell as taught – imperative, OOP, and functional styles of programming, I donot feel that one is intrinsically any harder than another. However, they aresometimes so vastly different from each other in style or syntax or semanticsthat once a student has solidified around the concepts of a particularparadigm, it can be a challenge retraining to work easily inanother.

If both "ideal use case" and "brainfit" are given as arguments against adopting FP (or any other new paradigm) inlarge organisations, and neither are considered logically or philosophicallyvalid, what's at the root of the resistance?

It is notuncommon for changes in an industry or field of study to be met withresistance. The bigger or more different the change from the status quo, veryoften is proportional to the amount of resistance. I suspect that this isreally what we're seeing when companies take a stance against FP. There arevery often valid business concerns: "we've made an investment in OOP" or "itwill cost too much to train/hire/migrate to FP." 

I would remind those company leaders, though, that new sources ofrevenue, that product innovation and changes in market adoption do not oftencome from maintaining or enforcing the current state. Instead, that is anidentifying characteristic of companies whose relevance isfading.

Even if your company has market dominanceor is a monopoly, there is still a good incentive for exploring alternativeparadigms. At the very least, one can uncover inefficiencies and apply newknowledge to remove duplication of efforts, increase margins,etc.

As a manager, I have found that about half of the seniorengineers up for promotion have very little to no interest in taking ondifferent (new to them) programmatic paradigms. They consider current burdenssufficient (or too much) and would rather spend what little free time theyhave available to them in improving existing systems.

Senior engineers who have a more academic or research bent (orare easily bored) are much more likely to embrace this sort of change.Interestingly, senior engineers who have little to no competitive drive willmore readily pick up something new if the need arises. This may be due to suchthings as not perceiving accumulated knowledge as territory to defend, forexample.

Younger engineers with less experience(and less of an investment made in a particular school of thought) are muchmore willing to take on new challenges. I believe there are many reasons forthis, one of which may include an interest in becoming more professionallycompetitive with their peers.

Junior or senior, Ihave found that programmers who are currently looking to find new employmentare nearly invariably not only willing to take on the challenge of learningdifferent paradigms, but are usually going about that proactively and engagingin self-study.

Even if a companyis locked into well-established processes and views on programming, they mayfind it in their best interests to provide a more open-minded approach withtheir employees who would enjoy that. Their retention rates could very wellincrease dramatically.

Philosophy and hiringstrategies aside, do we – as programmers, software projects, or organizationsthat support programming – need to take on the burden of learning or adoptingfunctional programming? Quite possibly not.

IfGoogle's plans around Go involve building a new operating system (in thespirit of 1970s C and UNIX), the systems programmers may find pure functionstoo cumbersome to work with. FP may be too burdensome a fit for that type ofwork.

If one is not tied to a historical analogywith UNIX, as Mozilla is not with Rust, doing something like creating a newbrowser engine (or running a remote services company) may be a good fit forFP, especially if one has data showing reduced error counts when using typesystems.

As we shall see illustrated in the nextpost, the usual advice continues to apply: the decision of which paradigm toemploy for any given project should be dictated by the best fit and notideological inflexibility. The bearing this has on programming isinnovation: it is the early adopters who have the best chance ofleading us into the future.

Up next: Retrospective onProgramming Paradigms
Previously: Themes at OSCON 2014