The folly of large teams

I came across this post on "The mythical man-month" (referring to a unit of labour) through twitter (via @finiteattention) and it really hit home. There has been a lot of complaining and hand-wringing and head-banging about the ECHOES project having grown completely out of control in a number of ways, and though self-labelled hacker-at-large Dhanji R. Prasanna wrote this post about a Google effort, it could easily have been written about all of us on ECHOES (except for the part about massive server traffic). For added fun, ECHOES also had the issue of the workers being physically in many different locations!

I would like to quote a few especially pertinent passages here:

"First, there was the dreaded endless meeting--they lasted for hours with very little being decided. Then, you started having to push people to provide...code changes that you desperately needed for your feature but that they had little to no interest in beyond the academic.
My style is to ask politely and then when I realize nothing is going to be done, to do it myself. This...does NOT work in large teams. There is simply too much system complexity for this to scale as a solution. Instead of shaving one Yak, you're shaving the entire Yak pen at the Zoo, and pretty soon traveling to Tibet to shave foreign Yaks you've never seen before and whose barbering you know little about."

So true. I'm laughing at that to keep from crying about it! Or how about this next quote? Replace "programmer" and "hacker" with "researcher"   and you definitely have ECHOES again.

"And this is the essential broader point--as a programmer you must have a series of wins, every single day. It is the Deus Ex Machina of hacker success. It is what makes you eager for the next feature, and the next after that. And a large team is poison to small wins. The nature of large teams is such that even when you do have wins, they come after long, tiresome and disproportionately many hurdles. And this takes all the wind out of them. Often when I shipped a feature it felt more like relief than euphoria."

I agree with the author that that really IS the key point about large teams-- they kill enthusiasm. It is impossible to congratulate yourself or your colleagues about any success because it took so miserably long to get there (and has likely created a wealthy of bad feeling and OTHER problems along the way). A success doesn't feel like a success any more. ON a bad day, I find it doesn't even feel like it was worth the effort to get there at all. Researchers and other people working in teams talk about "project politics" for a reason. There is not just because of the interpersonal issues that often get the label of being politics, but also the sense of protracted negotiations, constant friction, and people fighting their own disciplinary corner to the extent that they may lose sight of broader objectives.


When mini-teams are constantly barking "Fix this now! Turn attention level up to 11!" at other mini-teams, it seriously wears people down. Like in the human body, existing in a state of constant emergency contributes to the emergence of stress-related diseases (See Robert Sapolsky's excellent and entertaining book Why Zebras Don't Get Ulcers). For large teams, this takes the form of constricted communication channels, attacks of angry rhetoric, and dampened enthusiasm.


All of that said.... the author is also right when he writes of the potent blend of frustration and passion that makes such a team and project compelling and unforgettable. It's true, I totally get it. You look back fondly at the complete madness and the colleagues upon whom you wished painful deaths as you slaved away late at night, trying to fix problems and finish documents. I don't know why this should be so, as it seems like a completely insane and unreasonable way to feel about anything that has caused so much stress (for everyone!). Maybe it is some kind of cognitive dissonance reduction strategy? On the other hand, maybe it's not, I have used that term incorrectly, and Leon Festinger is now rolling over in his grave.

General reflections on the inaugural Turing Festival, Edinburgh

It is early on a Sunday morning at the end of August and most of the chairs upstairs at the Surgeon's Hall are empty. The repeated announcements that the last day of the Turing Festival was a sell-out suggest that, for many, late-night Fringe festival revelry was a more powerful draw than getting up for a third day of talks. I hoped that this third day of the festival would be more in the spirit of an amazing, geeky show-and-tell than the commercially-oriented programme of the first two days (which I had chosen not to attend out of equal parts disinterest and distaste for incessant start-up and commercialisation hype).

As the first speaker got started, I settled myself in with the hope of being excited and inspired by people doing innovative work. Ideally I would come away with some “ah-ha!” moments that I might apply to my own area of research, which is in multi-modal technologies for children with autism. Perhaps I might also get ideas for new ways to approach dissemination and public engagement. In any case, I wanted names to follow, links to click, food for thought. I had been happy to agree to provide some reflections on the event for the Interconnect newsletter,[1] and had some key questions jotted down in my notebook to guide my listening. Unfortunately, almost every single question remained unanswered at the end of the day.

While almost every speaker was passionate and knowledgeable, giving the audience plenty of food for thought and links to click, even talks in the same section of the programme felt entirely self-contained. There was little common ground between speakers, and little interaction with the audience. This audience was visibly varied and, from their questions, seemed to hail from many different disciplines and career trajectories. Their ages varied from postgrads to pensioners, and there was a better gender balance than I expected (about 1/3rd were women). Doubtless, the event might have been more dynamic if the audience were allowed more chances to interact with the speakers and content. Instead, there was a furious stream of #tfest tweets throughout the day to give some insight into how the politely applauding audience was really reacting. As I only got myself a “research Twitter” this summer (@a_m_alcorn0131), it was the first time I had attended an event that was being “live tweeted” as it happened. It was fascinating and frankly addictive[2] to be having a silent real-time conversation with the other audience members through the magic of smartphones and hashtags, and to see what complete strangers thought was most noteworthy about the presentations. Their tweets ranged from tweeting pithy direct quotes from David Britton, Tom Chatfield, and David McCandless, to arguing with Fraser Speirs over ipads in classrooms to the broad consensus that if only the Persuasion API speakers would stop giving us a boring and confused rehash of undergraduate social psychology, we could all go get some lunch. 

By the afternoon, the audience had filled in considerably for Tom Chatfield and David McCandless's talks. Apparently they have greater name recognition than many of the other speakers.

Overall, the quality and impact of the Sunday programme suffered considerably because it lacked cohesion and clear themes. Catchy but vague session titles like “Open Innovation at the Bleeding Edge” had little to do with the speakers' content. For example, the “Open Government” section of the programme might have been better titled something about personal data management. Although the speakers made interesting points about how much of administrative systems and data management are difficult to reform and streamline because they are indirectly—even accidentally--specified by other legislation, they were not government representatives. They promoted possible options for simplifying data management for both the government and citizens, but gave no information on what the government is actually doing (if anything) and what kind of results they aimed to achieve. These speakers might have been much more informative in chaired panel discussions on the title topics than they were individually, as was the case for most of the speakers all day. Only a few talks successfully stood alone--notably Sita Popat on robotics/dance, David McCandless on data journalism, and David Britton on the Large Hadron Collider—and seemed to have as their goal the celebration of innovations and collaborations which I had expected when signing up for the Turing Festival.

Dr. Sita Popat's talk on “Digital Bodies and Dancing Partners” succeeded especially well accessibly presenting fresh, interdisciplinary, and novel technology. Trained as a dancer, Dr. Popat entered academia after an injury and works at the intersection of dance, choreography, robotics, and performance. One video sequence showed a robotic arm as a dancing partner, which analysed a dancers’ movement and responded to its qualities (matching, opposing, following, etc.) to create a true impression of dancing with the human partner. Perhaps the most striking video examples showed how dancers have contributed to robotics research: engineers drawing madly as a dancer lay on the floor, moving her stiff limbs in an effort to imitate and extend the movements of a large insect-like robot beside her. The goal? To find out if the robot could use its legs in such a way as to stand. The exercise had been very successful, showing the engineers that their existing design was perhaps capable of much more than it had first appeared. Unfortunately, Dr. Popat said very little about how she arrived at (or created) this unusual field, and less about her personal views on these projects or involvement in science/technology more generally. 

Dr. Sita Popat describes how dancers have informed research into the potential movements for Zephyr, a stiff-legged and insect-like robot.

This idea of the STEM disciplines turning to the arts and humanities to inform their work was later continued persuasively by David McCandless, former TED-talker and self-professed data journalist of www.informationisbeautiful.net. His colourful and beautifully designed graphics and animations relate datasets to one another, so as to illustrate the differences in government spending in various areas, national debts, the relative size of countries’ armies, and other numbers that are usually presented in isolation from one another. This focus on the visual is not incidental, but the key to making sizes and relationships tangible and thought-provoking in a way that more traditional data presentations often do not. As McCandless noted, “Design is a powerful way to create trust and authority.” It is clear that regardless of your precise field or role, grasping how to deliver data in this way is an invaluable asset for communicating with colleagues, students, investors or the general public.

As a first-time event, Sunday’s Turing Festival had high points of insight, inspiration and humour, but was ultimately dragged down by uneven pacing and its complete lack of cohesion. Festival organisers, there are trade-offs between trying to have something for everyone and trying to have everything for a much narrower audience. For subsequent go-rounds, I hope that the organisers will either have a clearer and more obvious target audience, or will group (and name!) their content intelligently so that some sections are more clearly focused on development and commercialisation, public sector and non-profit groups, arts-related work, more “pure” research/science... etc. Subtitling your event “Edinburgh International Technology Festival” is so broad as to be in danger of foolishness: There is no way that everyone’s expectations can be met and interests can be catered for, unless you grow to be as big as the Edinburgh Science Festival. And that already has a lot of technology-related events. 

Despite my criticisms above, I think the event has huge potential and that there are many people in Scotland and further afield who have volumes to contribute in this area and should be encouraged to share their fascinating work with a wider audience of tech enthusiasts. In future years, a better understanding of the audience and a better structure (less lecturing, more discourse and interaction!) would doubtless make the event more compelling and better justify the price tag.


[1] Interconnect is an organisation for women in Scotland who are studying and working in STEM subjects. They put out a request on Twitter for someone to provide feedback about the Turing Festival event and I said I was happy to do so. Stay tuned to hear what happens to this piece and if it gets published!
[2] This second layer of the event was so addictive, in fact, that when my cell phone started dying I walked all the way across town on the lunch break in order to get my charger. In the rain! True story.