As an entrepreneur, you are met with a barrage of decisions that you need to make daily. What color scheme should you use for your website? Which project management tool are you going to use? How many customer interviews do you need? How soon do you need them? How are you going to get them?
There is no one else to answer these questions for you. Worse still, there is no one to prioritize these questions. And most of the questions are too far outside of your realm of expertise for you to answer without extensive research — research that you don’t have time to complete.
It’s a scary situation. There are no defenses and no one else to take the fall. You have your mentors and your support network, but — at the end of the day — the buck can only stop with you.
And that’s okay.
You don’t have to be an expert in everything to be an entrepreneur. You don’t have to get every decision right. You won’t because you can’t. Nobody does.
The trick to being a great entrepreneur is not to be perfect — instead, just think like a scientist. Make every decision into an experiment. Make words like “metric” and “hypothesis” a part of your daily vocabulary.
Do you want an example? Let’s jump right into one.
I needed to create an introduction email for my mailing list recently. Nothing in my background had prepared me for that. I had a vague idea of what the purpose of such an email was, but I didn’t know how long the email should be, what to put in the subject line, or whether I should focus on my personal story or the value I provide.
So what did I do?
I followed my six-step process.
1. Figure Out How Reversible the Decision Is
Any decision that you can’t undo will require more devotion. This is particularly true if the consequences of getting it wrong are severe. In some severe cases, they are even best left to an expert. Examples of these cases typically include writing important contracts and filing corporate taxes.
Few decisions are completely irreversible or completely reversible, but some decisions have more lasting impact than others.
Accepting a job interview is usually not a binding decision. Telling your boss you are quitting is harder to come back from. Writing the words “I, Zak Kann, quit my job!” in flaming logs in your boss’s front yard is definitely not something you can undo.
For my mailing list, the introductory email newsletter was almost entirely reversible. I could easily change it at any point. This took away much of the risk of trying something bold or unique.
Those who received it before the changes would only ever see the old one, but that was a small fraction of my eventual userbase. I had no delusions about two million people joining my mailing list in the first week (fun fact: only 3 people joined, and that includes my girlfriend).
If the consequences for a wrong step are too severe and cannot be undone, stop the process here. I said I treated almost every decision as an experiment. You’ve found the first group of exceptions.
The vast majority of decisions don’t fall into that caveat though. For that larger group, we continue to step 2.
2. Determine How Valuable the Result Is
My mailing list is the most intimate pathway I have to communicate with my followers, so the mailing list itself was certainly of high importance. But was the initial email important?
I didn’t know the answer. The internet provided me with some guidance on the topic, but I didn’t find enough of a consensus to treat it as an answered question. So I deemed it a 3-out-of-5 on my importance scale. This meant it wasn’t unimportant, but it certainly wasn’t something worth spending days on.
Remember that experiments do take time. They need to be set up, maintained, evaluated, and acted on. This can be too much effort if there is little reason to believe that the underlying decision is important.
Some experiments are also going to take weeks or even months of wait time for proper data collection. In some cases, to quote General Patton, “A good plan, violently executed now, is better than a perfect plan next week.”
Ready to get technical? Let’s move to step 3.
3. Define Metrics
What makes for a successful introductory email?
In an ideal world, I would love to see my introductory email lead to replies or people setting up time on my calendar. But this is not an ideal world, and I knew those metrics would be too sparse for me to use this early in the game.
For the subject line, there was an obvious answer. I used the open rate. If your subject line catches their attention, they tend to open it. Sure, there are still complications based on whether it ended up in the spam or promotions folder, but those are things that I couldn’t easily account for regardless of my metric.
Determining whether the body of the email was working proved more complicated. I considered looking at the rate of open for the next email — figuring that a good intro email would make people more likely to stay engaged with the list. But that turned out difficult to calculate with my current tools.
Reluctantly, I chose to use the click rate. My intro email provides links to my three favorite blog posts, and a click on any of those counts as a success. This is far from the ideal metric, but I believe that it is much better than having no metric at all.
Incidentally, this is not always the case.
A bad metric can be worse than no metric at all. Vanity metrics can have you optimizing for behavior that is not useful. Worse, that behavior may be happening instead of the behavior you want to be driving.
So what makes an ideal metric?
- It should directly relate to your desired outcome. Every task has a desired outcome. That outcome is related to your business goals. If your metric is not related to that outcome, it is useless.
- It should be predictive. This is related to the first bullet. Website visits sound great, but if 90% of your users leave immediately, it’s not a good metric. Increasing bounce visits looks good on that metric, but it does nothing for your bottom line.
- It should be isolated. If other levers you are pulling impact this metric more than the test it is associated with, it’s either a bad metric for this test or this is a bad test.
- It should be easily measurable. Don’t jump through hoops for the measurement. I don’t use any measurement if I can’t automate most of its calculation. Most of the time, a complicated measurement can be replaced with a similar but more streamlined alternative.
Once you’ve got your metrics, it’s time to let the real fun begin. Step 4, here we come.
4. Run the Experiment… Maybe
Before deciding to run the experiment, remember that you have several other experiments you could be running as well. Alongside the email changes, I was working on improving my Instagram account, creating a YouTube channel, and too many other tasks to name here.
Here’s where I stood on the mailing list introductory email decision.
- It was mostly reversible, so I wasn’t worried about running an experiment.
- It was a 3/5 on my importance scale, so there were more important items to consider.
- I had a metric that I was happy with for the subject line, but I wasn’t particularly satisfied with the metric I was using for the body of the email.
I also realized that this experiment was going to take a long time. New readers were trickling into my mailing list, so I wouldn’t get any usable results out of this for several weeks.
My final decision was that the experiment was worth doing, but most of it could wait. The subject line was the easiest to create, the most readily measured, and the most impactful part of the entire thing, so I decided that was the only part I would experiment on just yet.
I created two subject lines, and I am currently in the process of measuring whether there is any substantial difference between the two. I expect this will take at least another month at the current growth rate of my email list.
Notice that I created only two subject lines. I could have chosen from a near-infinite number of options. So why did I pick just two? Great question. I had a couple of reasons.
- More options require a longer experiment. Let’s say I have 200 people sign up during the next couple of months. If I have two possible subject lines, then I have approximately 100 getting each subject line. If I have 10 subject lines, then only 20 people get each subject line in that same time. It will take me 5 times as long to complete the experiment.
- I don’t know if any of this matters. This is not unrelated to the first point. In truth, the biggest decision I’m trying to come to here is “Does changing the subject line make a substantial difference?” I need to find the fastest way of determining the answer to that question. I’ve chosen two drastically different subject lines. If there is no measurable difference between the two, then I know I can leave this experiment behind. Is this a perfect system? Of course not, but it’s better than wasting months on an experiment that either doesn’t matter or that I’m not yet equipped to properly undertake.
For the body text, I went with my gut and my internet research on what to include. I did start collecting data on click rates, so I’ll have a headstart when I decide to start playing with that lever. But I don’t have a second candidate to test against at this time, and I doubt I will until at least October.
All your decisions will fall into one of three categories.
- Worth experimenting with now
- Worth experimenting with later
- Not worth experimenting with
That 3rd category contains anything that is irreversible, too risky, or just not that important. For anything in the 2nd category — just like my body text — try to start gathering statistics now if it’s low enough effort. Sometimes, though, changes you make in the meantime will render those early results misleading or useless, so handle with care.
Most of the grunt work is now done. Sit back, let the data flow in, and try not to look at it every 20 seconds. *Guilty*. Once the data is all in, it’s time for step 5.
5. Analyze the Results
You’re in the home stretch.
Don’t get too fancy here. These should be simple A/B tests. Often, the software you are using will facilitate that with minimal effort. For instance, I use Mailchimp for my mailing list, and they have A/B testing built-in. I’m a stats junkie, but I’m not going to reinvent the wheel when a convenient solution already exists.
Ready for the last step?
6. Decide Next Steps
With the results in hand, here’s what you know now that you didn’t before.
- Which option performed better?
- How much of a difference did it make?
Should you always go with the better-performing option?
Nope.
Your metrics were about measuring a specific benefit. But most changes will have a broader impact than that. You also need to take into account all of the following that apply to your particular case.
- Is one option cheaper than the other? Sometimes you can build this into your metric — e.g., cost per click instead of total clicks. But in other cases, it is easier to evaluate cost as part of the results instead of using it in the metric directly. This is especially true if determining the cost is so complicated that you don’t want to do it until you know the final impact.
- Is one option easier than the other? Your time is valuable, as is your mental health. A minor improvement that is massively more complicated probably isn’t worth it.
- Is there a switching cost or risk? This one only applies if one of the options was already in use before the experiment. There is always a cost to making a change instead of maintaining the status quo, even if the thing you are changing to will be easier in the long term. As an example, I just moved my website to a new hosting service. The new service was cheaper, but it took me several hours to safely move my entire site.
I don’t have a magic formula for weighing these factors. The best I can do is to say that risk and time can both be assigned a dollar value. Try to determine what the expected cost and expected return of each option are, and make your decision from there.
Final Thoughts
I didn’t realize how complicated this was going to sound until I started writing it down as a process. That should tell you that the process is much easier than it sounds.
At first, each step will take a significant amount of thought and effort. But eventually, it will go from a conscious process to learned behavior. Running these constant experiments takes very little time out of my life, and it has been well worth any time I have devoted to it.
I make decisions faster because I have a framework for evaluating them.
I have more confidence in the decisions I have made.
I have reason to believe that confidence is earned. If I ever need to, I can back those decisions up with facts and figures.
Perhaps most importantly, each result guides me in prioritizing future decisions.
So don’t worry if this seems daunting at first. Don’t stop because it takes more time. I guarantee that, in the long run, you will save much more time than you lose.
Are you ready to be a sci-entrepreneur?