Flexible client-side caching with Keen

Flexible Client-side Caching

At Keen one of the things we're focusing on is decreasing "time to first insight" as we want to make it easy for you to deliver lightning-fast embedded analytics.

To do that we're making sure that you have the ability to accelerate and cache queries at all levels of your Keen embedded analytics stack.

We're happy to share that keen-analysis.js now supports flexible Client-side Caching. It allows you to cache query results client side in your user's browser so that follow on or repeat queries get answered fast and efficiently.

It’s easy to implement and flexible feature for you to leverage. You can choose to cache all queries, or just specific queries - and you can of course control how long to cache results for:

For more details check out the SDK documentation here.

So when might you want to use Client-side Caching?

It’s most useful in cases where users are latency sensitive or where you want to control how often a user can run a query.

For example, if you had a dashboard that showed "Ad Engagement in the last 7 days" you might want to ensure that you always have the latest data. But, what if you're also showing "Ad Engagement in the last 12 months"? That's likely to be a more costly query due to the long time horizon. You could opt to cache that query for 24 hours so it's not fetched as frequently.

The other scenario where Client-side Caching can come in handy is if you anticipate that you might need to show a user a different set of queries soon. You could run these queries in the background when the user loads your application and then have the results cached and ready should the user need them. This is a great way to provide a user experience where loading other metrics feels instant.

Client-side Caching along with our service side feature's of Cached Datasets and Cached Queries give you a rich set of features that you can leverage to deliver metrics and insights to your end users with lightning speed.

Giving our customers a voice - Keen

Giving Our Users a Voice

A quick note’s probably in order since I’m sort of a newbie on the team. My name’s Florian. I was fortunate to join Keen a few months ago as VP of Product Engineering. Some of you probably know me already from demos, support, calls, etc. I’ve been passionate about building analytics and metrics products for a long time. It goes back a decade or more. It started out with me trying to visualize the spread of SQL Slammer in school. It’s been a fun journey to get here - filled with time series databases, horizon charts, Holt-Winters Forecast’s, and bespoke cloud storage platforms. But I’m even more excited about what's to come and what I can do to help Keen make our customers' jobs easier.

We were acquired by Scaleworks a little over 8 months ago. During this time we’ve been hard at work behind the scenes. We’ve been building out our team, optimizing our infrastructure, and getting ready move the Keen platform forward to the next stage. Along the way we also took a bit of a detour to work on GDPR, that, of course, consumed quite a bit of time. Now we’re ready to start moving forward.

We may be a bit biased, but we think the Keen platform is fantastic (and it's clear you do as well). Of course, we also know that like any platform it has warts and shortcomings. Good news is that we have a lot of room to improve. Analytics is not a solved problem. Implementing any analytics solutions is still hard and still frustrating. We’re working to solve that and we’ll have more exciting news about it in the coming months! Through customer interviews, support tickets, and account reviews, our users have been fantastic about helping us start to plot out our future roadmap. However, we’ve felt like we’ve only been able to scratch the tip of the feedback iceberg.

Giving users a voice

I’ve was lucky to have been able to use canny.io previously and knew I wanted something similar for Keen customers.

Building a strong relationship between our user community and our team is critical to the ongoing success of Keen. To that end, we want to make sure that our users always have a voice.

Today we’re happy to be launching our product feedback board using canny.io’s awesome platform. feedback.keen.io is the place you can go to post your feature requests, tell us about things that would make your job easier, or see what we’ve got inflight. We’ll share mockup’s and use it as a sounding board as we turn ideas into actual features. Not everything here will get built but we promise to always keep the communication going and you’ll definitely have a voice as Keen grows.

Collaborate and Engage

Using our product feedback board will help us prioritize, balance, and clarify feature requests. It's a great way for us to spot what users need from us to make their day to day jobs easier. We can use it to validate new features we’ve cooked up internally, or get feedback on mockups as we iterate on UX. The collaborative nature means we can share and get feedback from Keen users much faster.

With API focused products like Keen, launching a new feature often means that end users have to do a bit of dev work to actually utilize them. The fact that we can give users a peek at what's coming, as well as let them preview how new features are being structured makes consuming them much easier. One of our driving goals is going to be to make the jobs of our users easier, and feedback.keen.io will let users adopt new features with a little less work.

Honestly, it makes our jobs a bit easier as well. Product boards like this make it easy for the Customer Success, Product, and Engineering teams to keep our customers up-to-date and engaged. CS can send follow-ups when you reach out with a feature request via support, our Account Managers will eventually be able to share a living roadmap during account reviews, I’ll be able to easily see which customers I can chat with after a feature launch to see if we hit the mark, and our engineers have a much more direct line to customers for clarification.

So, come and help us lay down a path for Keen:

  1. We’re planning support for AMP - AMP Support
  2. The ability to anonymize IP Address to help you be GDPR compliant is in progress - Anonymize IP Addresses
  3. We’d love feedback on what sort of Spend Limits/Alerting we should implement - Spend Limits
  4. And there’s more (from OR Filters, to SSO Logins, to Dashboard SDK updates), and whatever else you want us to know about!

(Hint: That exciting news we’ve got in store… that I mentioned earlier? Keep an eye on feedback.keen.io over the next few weeks and you’ll get early insights to it as we start adding cards/features to the board)

Keen and GDPR

You’ve probably heard all about the EU’s new regulation, the General Data Protection Regulation (GDPR). The GDPR applies not only to EU-based businesses but also to any business that controls or processes data of EU citizens. Not only is GDPR an important step in protecting privacy for European citizens, it also raises the bar for data protection, security, and compliance in the industry.

At Keen, we’ve been hard at work to ensure that our own practices are GDPR-compliant. A big piece of that is ensuring that our product makes it easy for you to use Keen to handle data in compliance with GDPR requirements. In March 2018 we published a blog post that detailed the steps we would take in order to accomplish this.

Since that time, we’ve accomplished the following:

  • Appointed a Data Protection Officer and a data protection working team
  • Built a formal data map
  • Performed internal threat modeling and gap analysis (and set up a recurring schedule)
  • Adopted and formalized written policies around core areas, including (but not necessarily limited to): data protection, data backup, data retention, access management, and breach management and reporting
  • Conducted formal data protection training for all Keen employees
  • Encrypted data at rest (still in progress for some data)
  • We’re working with a 3rd party auditor to schedule annual security audits
  • Completed legal paperwork to confirm that our Data Sub-processors (primarily Amazon) are GDPR-compliant
  • Offer a Data Processor Agreement to our customers upon request
  • Received Privacy Shield certification

There are several additional security enhancements that we will continue to iterate on and improve over time:

  • More granular access controls, allowing Keen employees to be granted access according to the Principle of Least Privilege
  • Full customer data access audit history
  • Lockdown of Keen employee devices, and/or limiting access to customer data to certain approved devices

** A note about data deletion **

During our many conversations with customers about their GDPR compliance efforts and concerns, the most common theme was the need for various types of data deletion. Some examples that we’ve heard include:

  • specific property removal from all events
  • deletion (or anonymization) of all events matching certain filters (e.g. all events with a specific user.id for “right to be forgotten” requests)
  • one-time deletions of all data before some time threshold
  • on-going “expiration” of data older than some horizon

While the Keen delete API endpoint can handle some of these at small scale, for larger use cases we felt that a more powerful toolset was needed. That toolset is now under active and on-going development, and is used internally. It can be run on customers’ behalf on a case-by-case basis. If you have GDPR-related deletion needs please contact us for more details.

Keep a lookout for more updates on our blog as we continue to make performance and security enhancements to Keen.