Today was a lot of why I always enjoyed academic-style research but ultimately left it. We ran into an issue over the weekend that prevented us from getting something done before users hopped back on Monday morning.The task of the day today (Monday) was to get it solved so we don’t run into the same problem this weekend and could have the system ready by next Monday. 

This consisted of throwing around a lot of ideas, building prototypes and testing to see how well they worked and if they could get it done in time. I loved it. Designing experiments and gathering data is exciting and a big reason why I went to grad school.

But after working on it all day, we didn’t get convincing enough to be confident that, come Sunday, we wouldn’t have to tell the client that we’re going to delay again. We threw most of that work away (well, tucked it in the back pocket in case something similar shows up to take advantage of it) and ended up building a solution that will continuously until it’s done without affecting users, avoiding the requirement of running during off-hours over the weekend, but at the cost of potentially taking more than the weekend before completion. The users don’t mind not having the product done Monday or Thursday, but the stakeholders really don’t want to have to send out another email saying things are pushed back again.

Hooray! Solved! And without making promises that we aren’t confident about keeping! But I can’t help but feel a little bummed that most of the days work (all that exciting work!) was mostly for naught. This is one of the reasons why I found more frustration than fulfillment in academia; that a significant fraction of effort is spent on experiments and data that will ultimately go towards maybe a small paper or poster, but mostly own personal development.

In our case today, we could have avoided the “wasteful” work if we had realized that we did not want to promise then fail to deliver on that promised date. The only way we’d confidence to make a completion date promise was to have nearly zero uncertainty about completion date, which requires huge safety factors on projections or just doing the entire thing and well seeing when it completed. The cone of uncertainty can pretty wide.

Academia is really good at finding solutions to problems, but can be kind of terrible in identifying what the problems really are. If I was back in grad school with the same problem, I likely would have kept working on optimizing and shaving a hours, minutes, seconds off my projections until I had something within the target date with a safety factor of 2 (or should we do 3? Maybe 4 is enough). Academic Brett would have kept on working to solve that exciting, interesting problem, rather than think about what problem I’m really trying to solve. Which may be fun! Especially whenever things work, but can quickly enter the realm of very-not-fun when they don’t and that whole day/week of work felt wasted trying to solve the wrong problem.