If you find this post interesting, please consider checking out Browserpath, a web performance tool I’m working on.

…

TL;DR: This blog post is about 4000 words. I gave a talk at a University to some students about how to do good software consulting projects. I discussed focusing client needs, working in an agile fashion, team and client communications, and more. I spoke about how all computer problems are people problems and how projects rarely fail for technical reasons.

…

I was recently invited by my Alma Mater (The University of Bristol) to give a guest lecture. I spoke to a number of second year Computer Science students taking the Software Project Engineering course. This post is a textlation summary of what I said to them. It’s also really really long. Sorry about that. You can always check out the slides on Speakerdeck. It’s worth noting, I only had about two days to construct the presentation. This blog post is a rough, in order, translation of that presentation. So it may seem a little all over the place.

…

I would like to believe that I am, at least, a moderately effective consultant. I’ve definitely learned a lot over the past couple of years running my business. It’s vastly changed my opinion about what’s actually important in the technology industry. I was glad that I was asked to come and give this presentation. My experience particularly dealing with clients applies directly to this style of course in a way that I think people who work for “product” companies won’t have had. In addition, running a business adds an additional layer of skills that one wouldn’t otherwise encounter.

The most important lesson

If I could teach one thing to all new developers it would be the following sentence. All computer problems are people problems. The interpretation of this sentence is in the eye of beholder, but another way of phrasing it is: No software project ever failed for technical reasons. Not. One.

The first time I heard this, I was very skeptical. I thought I understood what it meant to run a software project. I thought that there were indeed meaningful technical reasons that projects failed. I now, however, have very different feelings about this. An example of this might be “bad code”. When you see bad code, it’s very important to realize that that code probably isn’t there for technical reasons.

Behind all of the technical decisions made in a project there is a person. A squishy, fallible person. Behind that person is a team and behind that team there is a culture. Culture pervades companies and the industry and gives us our “good technical decisions”. Our “logical defaults” if you will. The thing that’s important to realize about all of this is that these defaults are nothing but the ideas of people that won, a long time ago.

There’s a reason different communities have different practices around things like testing, refactoring, code quality and even bigger decisions. That’s the people, not the technology. In those communities people are driving different ideas about how systems should work. Often, the most articulate of those people will drive ideas all the way back down into those teams, and those individuals. I guess what I’m trying to say here, is that it’s mostly false to assume that any particular technical factor in a system can be blamed for any kind of project failure. A more holistic view, that includes the people, the off days, the hangovers, the disagreements, and the budgets and deadlines is necessary to locate failure or success in a project. You can’t simply blame technical factors. That is never enough.

Getting started

At the beginning of every software project, everything feels very smooth. The analogy I like to use is “on the rails”. You’ll ship features and feel effective. You’ll feel like you’re making progress. Everything will be great. Unfortunately, this state will not remain. I have yet to work on a software project that did not end up at some point being a total train-wreck.

Train wreck

This isn’t to say that I think I’m a bad developer, the teams I’m working with are bad, or even a comment on the industry as a whole. It’s not a judgement on anyone who has ever had a train-wreck on one of their projects. It doesn’t happen due to any lack of communication with clients, stakeholders, product owners, or whoever might be driving the project. It seems to happen with all teams, regardless of skill, scope and context. It’s probably inevitable because computers are hard. For second year Computer Science students: it’s also not a problem if their project turns into a total train-wreck. I’d say that it’s almost inevitable. It’s their first group project with any kind of outside client.

One of the most obvious reasons this happens is that it’s often very attractive to trade speed now for technical debt later. As it turns out, good teams can do this basically forever. The upper bound on the amount of technical debt a team of skillful developers can work with is very very high. On younger teams, the ones I was talking to, that is absolutely not the case. Ensuring code quality is managed with feature delivery is hugely important, but I’m getting ahead of myself.

The software you write as a student is wildly different to Industry software

Here’s a point where I’m just going to quote Sarah Mei verbatim, and then include some of my own interpretation below.

I lived with it for 16 years, but I just realized today how weird academia’s concept of a project finished “on time” is.— Sarah Mei (@sarahmei) October 20, 2015

Anything not finished to their satisfaction is points off, like a project isn’t satisfactory unless it does everything its creator imagined.— Sarah Mei (@sarahmei) October 20, 2015

We tell students - for 16 years - that this is how to be successful. No wonder folks coming out can’t function on a real project.— Sarah Mei (@sarahmei) October 20, 2015

Real projects are a totally different mindset. Real projects are a constant negotiation around what finished means.— Sarah Mei (@sarahmei) October 20, 2015

This is one of the reasons why I think subjecting students to a “real” (Author’s note: I would use the word “commercial” instead of real here) project is a really good idea. By asking students to build a piece of software for six months in a group, we teach them a lot. By giving the students an external client to dictate the requirements, we teach them a lot.

Many universities (based on my extremely biased UK experience) require group projects from their students. Rarely are they put in this kind of commercial context though. Having an outside client is an interesting proposition.

I’m not going to discuss the ethical concerns of asking a bunch of second year Computer Science students to build software for other people in exchange for grades. However, I will say that it’s important that they’re aware of where they sit in the industry right now. It’s also important for them to understand how this project might affect them going forward.

I actually cut this section from the talk, because I had to fit Sarah’s wonderful twitter rant in. This section would have taken me over time, but it’s worth putting in the text version.

As a developer, it’s tempting to think you understand the law. It’s just another system. Systems are what we’re good at. Unfortunately, this couldn’t be further from the truth. Patrick Mackenzie has this catchphrase “charge more”. I think mine might be “hire lawyers”. Hire lawyers. Hire lawyers. Hire Lawyers.

The students that I was speaking to were building software for unspecified customers, which creates a whole host of legal problems. I don’t think most new developers know the default positions on these things, so I’ll briefly cover them here.

Intellectual property: you own it. Period. The end. Unless you sign the intellectual property over in some kind of contract, you own it. Here’s the exciting bit: I don’t think the students I spoke to signed any such contract. I also don’t think that the clients the students are working with have signed any contract with the University. I’ll let you draw your own conclusions on how much of an intellectual property trainwreck that could become in the future.

The other is implied warranty. Now, given the non-existence of any contract, I’ve got no idea how open to being sued any of the students might be when they hand over the software. Given the “everyone deserves their day in court” philosophy, it might be the case that the software they hand over could totally be used to sue them if it breaks down at some point. Who knows! Hire Lawyers.

Communicate with your clients

A second year Computer Science student is already a commercial proficient software developer. At Bristol, they’ll have already programmed a number of small (up to 1000 source lines of code) systems in C, Java and Haskell. Anyone who can do that can basically take any Junior Developer job anywhere in the industry. Given that and their non-technical clients, they are the technical experts on the project. They, second year Computer Science students, are literal expert consultants. That should horrify them. The first problem this gives them is making their clients believe that they’re actually getting some pretty adept people working with them. (I’m not saying that a second year Computer Science student is a fully equipped technical programmer or consultant. Merely remarking on the fact that the delta between them and their clients is huge).

One thing that I’ve learned from my 2.5 years of being a full time consultant is that managing clients is it’s own separate, complex, skill. It’s also the most important thing that one needs to be good at to be an effective consultant. The particular kind of consulting I do lends itself to a certain level of technical expertise, but that’s absolutely not true for the students. They just need to be able to build an application (which admittedly, is a tough, but fair, barrier for them).

That makes client communication the thing that they’ll need to focus on more than any other. There are myriad problems when it comes to working with clients that simply do not exist when building a solo project. When you have a client, the scope of the project is always expanding. This isn’t because of any explicit wish to expand the scope of the project on their side. It’s more that some part of the back of their brain will always be ticking over on this project, adding features, continuing until the logical completion of the project.

For any project with a fixed deadline (like this university one) it’s not reasonable to assume the students can deliver to this ever expanding scope. To that end, I told them that one of the most critical things you can do is establish clear scopes with your client. I also told them that it’s ok to say no to feature requests. It is so much better to deliver something small, tight, and working, than to deliver a huge unending mess that’s not quite functional.

One of the key reasons for this is that all clients ever want is to be able to show their software project to someone. As a first pass, from scratch, greenfield, you’re not going to get to commercially functional in six months. You’re hopefully going to get to “passable MVP”. As such, small and working is better than big and broken.

The other thing that you’ll have with scopes and clients is that the scope of the project will always be changing. Most of the time this also isn’t because your clients hate you. It’s because every time you talk to them, or every time they see the software you’re working on, their mind will change. This is natural. Our mental models of what the software will look like will be violated in an instant when we actually see it.

This is why waterfall doesn’t work (for some types of projects). A friend of mine, Stephen Best, likes to say “Waterfall is the fastest way to get to the wrong place.” What he means by that is that waterfall gives you a big, fixed, aim that’s specified in advance towards which you should work. When you get to that big fixed aim, you’ll discover that, while you’ve built exactly what the client asked for, you’ve not built what the client wanted.

The key here, is to establish clear short term scopes with your client. Deliver towards those short term scopes and then adjust them. Repeat until done. When planning these short term scopes you force two things:

  1. The client focuses on what they need right now
  2. It’s clear, not fuzzy what you want to deliver

It is OK to declare that an expectation from a client is unreasonable and strike it early. Some of the teams I worked with are, for example, asked to deliver both iOS and Android versions of an application. This literally doubles their workload for very little additional benefit for the client. All the client wants (right now) is to be able to show the thing to some people. There’s no need, in that context, for the application to be built on top of two wildly different mobile operating system platforms.

The point of all of this, is that communicating with a client about their expectations ahead of time can be of huge benefit in consulting. You can strike anything through features all the way up to entire platforms given the constraints of a project. If you explain why you’re working within reasonable constraints, no reasonable client can fault you for that. If they do, then they’re probably being an unreasonable client and you probably don’t want to work with them.

Turning unreasonable clients into reasonable clients is much more difficult, and left as an exercise to the reader.

Project execution

This part of the talk was focused on how one actually executes a project with small focused scopes. Now I’m not saying that I have a global panacea for running software projects, but this is roughly what we aim to do at Fun and Plausible. It’s also worth noting that you should be absolutely flexible with your project workflow. There is no “right” way to deliver software, you have to work out what works for you and your team.

At this point in the talk, I made a fairly controversial statement, which actually had one lecturer jokingly say “Don’t listen to him”. The statement is that you should not follow a particular “Agile methodology”. To which the audience giggled. I should be very clear about what I mean by this. I don’t mean “don’t do agile”. I mean don’t adopt a fixed set of agile practices like “Extreme Programming”, “Scrum”, or one of the “DD” methodologies.

Agile isn’t about picking a fixed set of practices and then using them forever. Picking any specific methodology locks you into the mindset that the particular methodology is “correct”. If it doesn’t work, it must mean that something is wrong with you or your team.

Nothing could be further from the truth. If you’re struggling with a particular agile methodology, it probably means it’s not right for you or your team. Instead, I think it’s better to view agile as a toolbox. It’s better to pick and choose from the different parts of “agile” that are right for you. In addition to this, you are allowed to adjust your project management method over time. This is perfectly fine. Sometimes you need additional structure, and sometimes you don’t.

During early phases of a project, it’s probably fine for your team to just have cards, move them around and explore. As you get further into your project or closer to your deadline, you might find that things like story points, burn down charts and dedicated sprints work for you.

An example

This was the point where I gave an example of the lightweight agile methodology I like to use. I by no means would attempt to declare this to be a universal project management structure. I also would say that I’ve found this to work well for client projects, but I can’t imagine it working so well at a product shop. I also imagine that if we had clients of a different size, scope or level of technical competence it wouldn’t work.

When I’m working on a client project that’s going to have a medium or long duration (i.e. more than a quarter) I like to do the following things:

Notably missing from this list is the presence of any kind of story pointing, or metrics over developer progress. I find this to be fine. It’s easy for clients to fall in to the trap of assuming that when you deliver 30 story points in one week, you’ll always deliver 30 story points. In fact, that may not be true at all. A lack of story points also means that you’re required to continue to break tasks down until they’re all roughly the same size.

One thing I find is that you can just declare a story to be “5 points” (or whatever your hardest size is) and then stop breaking it down. If you do that, your completion time on that task is going to be significantly higher than 5x on any 1 pointer you have. You don’t fully understand that problem.

I guess what I’m sort of getting at is that story points give you arbitrary metrics to hide behind. The problem is, I’ve typically found story points don’t hold the same meaning over time. 6 story points from 8 weeks ago will not mean the same thing as 6 story points today. This can be very difficult to explain to non-technical people, and I find it’s better to sidestep the conversation entirely.

For what it’s worth, Sarah Mei (remember her from earlier) likes story points a lot. We’re allowed to have that disagreement, it doesn’t mean we think the other is bad at software, it’s just different perspectives motivated by different working conditions.

Obviously, if you want to be able to measure velocity, the relative difficulty of tasks or something else, story points can be useful. The point I’m making here is not that story points aren’t good. Instead, I’m saying that they typically aren’t useful for the sorts of projects that I work on.

For the students I was lecturing to, there’s this concept of “client acceptance” that’s part of their marking (grading in 🇺🇸) scheme. Notionally, this has to be achieved by the project submission deadline. One thing that I’ve seen happen before is that people think that means they don’t need to show their clients things very often. So long as they do everything the client has asked for and they show them just before the deadline it will be fine.

To me, that sounds like a waterfall approach. Instead, I encouraged them to follow a two-week cycle structure like I provided above. The reason for this is that if you’re showing your client the project every two weeks, you are literally getting their acceptance every two weeks. When this happens you can be more comfortable in your assumption that the work you’re doing is useful and helping the client.

There is, of course, a big tradeoff to this approach. That tradeoff is that you’re going to have a lot more change requests than you would if you don’t have meetings often. You’re going to have to modify the direction the software project is taking a number of times. As it turns out, that’s fine. Waterfall is the fastest way to get to the wrong place. Agile takes more time, but it gets you to a destination that everyone’s happy with.

Tools

The situation being that I was speaking to students, I wanted to make sure that they were fully tooled up. Everyone has their favourites and I’ve already mentioned Trello. The other most important one is obviously GitHub.

GitHub has changed the way just about every developer I know works. One big part of that is pull requests. Pull requests give you a remarkable ability to get other people to read, validate and test your code before it hits your “master” branch. By doing this, you ensure that your master branch always works and that everyone is happy. It also means that you don’t have anything experimental in a place that’s easy to deploy outward, and demo to your clients.

There will, of course, be readers who leap into action that suggest that $THEIR_FAVOURED_GIT_HOSTING_SITE also has pull requests, a similar feature, or that pull requests are bad. However, GitHub is such a central tool, that I figured exposing students to it as a recommendation was a good idea. I was not there to present the politics of moving the industry to them.

Conclusions

It has never been easier to produce applications so quickly. This changes the way we work. This changes the way that we need to train our students. Client services presents its own set of unique challenges. I’m glad I got the opportunity to share some of my thoughts with the next generation of developers, and also all of you lovely readers.

Basically:

Let’s come back to that original point. All computer problems are people problems. All of the points listed above are specific instantiations of this core meta problem. By looking at yourself, your team, and your culture you’ll quickly see a minefield of problems. This is the core point of “all computer problems are people problems”. All of the issued I’ve addressed here are about communicating with your team and your client. These communication problems are at the core of every software problem. They’re how we get better. They’re how we do Agile, etc.

Thanks for reading. If you liked this post, please check out my upcoming SaaS: Browserpath.