Test the Riskiest Areas of Your Business First

Share on facebook
Share on twitter
Share on pinterest


I love what Alistair Croll has to say in this Software Advice interview about Lean Analytics, particularly his thoughts on testing the riskiest areas of your business first. What a great opportunity to reflect on how we think about our business here at Keen IO.

“Test the riskiest areas of your business first”

Last year we built an architecture prototype for Keen IO, graduated Tech Stars, and immediately dug deeper into customer development and marketing. We had 8,000 ideas of what to build next, but we spent most of our energy soliciting feedback, listening, and building out analysis & visualization features only as demanded.

It was TEMPTING to build a massive architecture to handle 8 Billion events per minute, but we knew architecture wasn’t a risk area for our business. We know how to scale technology.

What we weren’t sure about was how to drive customers and traffic to our site, and what things those customers would demand. So we focused all of our energy on that first. Validating our assumptions about the market gave us the confidence to invest heavily in infrastructure in our next phase of our product growth. In Alistair’s words:

“The old way: If you build it they will come.

The new way: If they come you will build it”

“What is your most important business metric?”

Different metrics will be important to your business depending on your stage. Here are the phases we’ve gone through so far, and the important events we’ve tracked in order to measure our progress:

1. Signups & Followers — proof that people are interested in analytics by API

  • Beta signups
  • Twitter followers
  • Account creations

2. Visitors & Conversion — proof that we can drive people to our site at a meaningful volume, and certain percentage of them will signup

  • Pageviews including referrers (e.g. Hacker News, Twitter, etc)
  • Account creations

3. Engagement & Retention — proof that people are using the product and continuing to use the product

  • Number events being sent to Keen IO (Engagement)
  • Number of unique accounts sending events to Keen IO (Engagement)
  • Daily event volumes for top accounts (Engagement)
  • Number of Keen IO API analysis calls (Engagement)
  • Number of unique accounts performing analysis calls (Engagement)
  • Number of accounts that become active users (Conversion)
  • Number of accounts using the service X weeks after signup (Retention)

4. Revenue — proof that we can convert engaged users into paying customers

  • Payment revenue
  • Number of paying customers
  • Number of active accounts that become paying customers (Conversion)

Thanks to Software Advice for sharing the video!

Share this post with your friends

Share on facebook
Share on google
Share on twitter
Share on linkedin

Subscribe to our Newsletter

Leave a Reply

Your email address will not be published. Required fields are marked *