Google Tech Lead: "Engineering Management Is Dying"

I recently made a career move from being a hands-off manager, to what Google calls a Tech Lead/Manager. In part, I did it because I love coding and felt the need for a change, but I’m also looking to the future.

Engineering management, as a distinct competence, is disappearing from the software world. It’s been undone by both technological and cultural change; by years of failure applying standard project management techniques where they don’t belong, and equally, from the widespread acceptance of Agile (however you define it) as THE one-size-fits-all way to do things. But mostly, it’s been undone by a continuing erosion of responsibility accorded to managers. 

To a degree, we had it coming: equating engineering management with project management and project management with PERT, we added little value for many years. The people who created Agile (Beck, Cockburn, Popendeck, etc.) realized that wasn’t the way. They thought deeply about process, motivation, and so on. Today they’re surrounded a vast army of scrum-masters and evangelists, recycling (competently, mostly) the same insights, and an even larger crowd performing half-assed renditions of steps learned by rote: “Yeah, we do agile. We have a standup every day.” Many of these renditions have the professionalism, but not the passion, of a small-town talent show. 

The growing acceptance of agile as the ‘one true religion’ means that, regardless of whether it works for us, we check the Agile box and stop thinking about process. It’s hard to add value that way.

The real problem, though, runs much deeper. 

No managers need apply

Certainly, outsourcing is a factor. If engineering moves overseas, engineering management jobs follow.  But the real issue is companies that do development on-shore, but do it with vastly reduced numbers of managers, who have vastly reduced responsibilities.

The 300-person game company Valve made headlines recently as a spotlight fell on their “boss-less” organization. Similar companies (GitHub, W.L. Gore, Morning Star) exist both inside and outside the software world. There are elements of this philosophy at work at Google, where it’s not unusual to see 200-person projects headed by an Engineer.

Drucker saw it coming. In the ’80s, he predicted the rise of the “information-based-organization”. He saw information as enabling an extremely flat structure, which like an orchestra, consisted of technical experts who perhaps dreamt of moving from second bassoon to first bassoon, or from a good orchestra to a better one, but had no interest in pursuing a career in conducting, a.k.a. management. 

In software this is clearly happening. In a well functioning team, most of the feedback (information) an engineer needs comes directly from the process: compilers with built in analytics, online, peer-driven, code reviews, continuous builds, and automated tests.

[Edit: As Josh Barratt commented, another important trend is that tools and services like Puppet, Chef, Capistrano, Github, App Engine, EC2, Heroku, etc., reduce complexity and thereby also reduce the need for engineering management.]

Management as social work 

The traditional definition of management is “getting things done through people”. Former Intel chief Andy Grove recognized this when he recommended we “measure managers based on their team’s output”. Drucker puts the same emphasis of accomplishment:

To be more than a slothful steward of the talents in his keeping, the executive has to accept responsibility for making the future happen. It is (this) that distinguishes the great business from the merely competent one, and the business builder from the executive-suite custodian.

Now engineering managers divide much of their time between status reports (which often say nothing of how the work is progressing), and a growing load of HR paperwork.  There is little that might be construed as “making the future happen”.  Instead, we tend to the career aspirations of our charges, show or feign an interest in them as people, provide formal feedback as needed for legal and compensation purposes, mediate disputes, put under-achievers on “performance plans” and carry out the occasional firing. All the while the hackers themselves tend to the business of building things. Our new responsibilities are valuable, but they’re not management. We have become social-workers.

One HR function managers may not get to perform is the one that counts: “We do everything to minimize the authority and power of the manager in making a hiring decision” a Google executive was quoted as saying and it’s certainly true. But it’s just the most articulate expression I’ve seen of a growing phenomenon. 

If there’s one thing about software engineering that everyone agrees on, it’s that nothing matters more than the people. Anyone who has built a team, scouring resumes, learning about the candidates, screening the best and selling them on the position knows what an intense experience it can be. It establishes a strong relationship between the hired and the hiring manager, and more importantly it establishes in the manager a deep commitment to the success of his people. Removing this task from the manager’s responsibilities further lessens the role.

Now what?

What we do now is a personal choice that each manager has to make for herself, based on her own talents and interests. As I mentioned above, I’m getting hands-on again. I think this will make me a better manager because I’m experiencing first-hand the troubles my team faces in their work. 

I’m also working on two other things that Drucker might recommend. He wrote:

Information is data endowed with relevance and purpose. Converting data into information requires knowledge. And knowledge, by definition, is specialized. The information-based organization requires far more specialists….

I’m becoming a specialist in two aspects of the software process that hackers ignore. The first is applying data, metrics and analytics to process improvement, and the second is structuring the work to enable people to have a better work-life, which I call positive software engineering. Not surprisingly, these are the two subjects I write about on this blog.