Flexible client-side caching with Keen

Flexible Client-side Caching

At Keen one of the things we're focusing on is decreasing "time to first insight" as we want to make it easy for you to deliver lightning-fast embedded analytics.

To do that we're making sure that you have the ability to accelerate and cache queries at all levels of your Keen embedded analytics stack.

We're happy to share that keen-analysis.js now supports flexible Client-side Caching. It allows you to cache query results client side in your user's browser so that follow on or repeat queries get answered fast and efficiently.

It’s easy to implement and flexible feature for you to leverage. You can choose to cache all queries, or just specific queries - and you can of course control how long to cache results for:

For more details check out the SDK documentation here.

So when might you want to use Client-side Caching?

It’s most useful in cases where users are latency sensitive or where you want to control how often a user can run a query.

For example, if you had a dashboard that showed "Ad Engagement in the last 7 days" you might want to ensure that you always have the latest data. But, what if you're also showing "Ad Engagement in the last 12 months"? That's likely to be a more costly query due to the long time horizon. You could opt to cache that query for 24 hours so it's not fetched as frequently.

The other scenario where Client-side Caching can come in handy is if you anticipate that you might need to show a user a different set of queries soon. You could run these queries in the background when the user loads your application and then have the results cached and ready should the user need them. This is a great way to provide a user experience where loading other metrics feels instant.

Client-side Caching along with our service side feature's of Cached Datasets and Cached Queries give you a rich set of features that you can leverage to deliver metrics and insights to your end users with lightning speed.


IP anonymization for Geo Enrichment with Keen

IP Anonymization for Geo Enrichment

Back in May, we detailed our accomplishments in terms of keeping your data GDPR compliant. Part of our progress included a powerful internal toolset which enables us to fulfill your data removal requests and this toolset works well for cleaning up data already persisted at Keen. But we believe that it’s better to prevent than to cure, and we asked ourselves what could we do to facilitate the use of our API while staying compliant with GDPR.

One valuable feature of the Keen Stream API is IP to Geo enrichment. This helps enrich your events to include geolocation data for your analysis and creation of map-based data visualizations. If you send Keen an IP address with the IP to Geo enrichment enabled, we can help give you more detailed geo data for analysis, like city, state/province, country, and more.

Given an event containing an IP address, you can define an addon to convert the IP address to a location. The IP address, however, stays in the event body. To activate the enrichment, include the keen.addons object in your data model as seen below:

Input event

{
  "ip_address": "${keen.ip}",
  "keen": {
    "addons": [{
      "name": "keen:ip_to_geo",
      "input": {
        "ip": "ip_address"
      },
      "output": "ip_geo_info"
    }]
  }
}

Output event

{
  "ip_address": "192.168.0.1",
  "ip_geo_info": {
    "city": "San Francisco",
    "province": "California",
    "country": "United States",
    "country_code": "US",
    "continent": "North America",
    "postal_code": "94122",
    "coordinates": [-122.42005, 37.77479]
  },
  "keen": {
    "created_at": "2012-12-14T20:24:01.123Z",
    "timestamp": "2012-12-14T20:24:01.123Z",
    "id": "asd9fadifjaqw9asdfasdf939"
  }
}

This is a powerful enrichment, however, IP addresses are considered personal information according to GDPR terms. Storing such information requires user consent and could also be subject to “right to be forgotten” requests. As such, it’s desirable to have the IP removed from the event just after being persisted. Up until now, it was only possible to remove the property holding the IP address using the Delete API or have us remove it for you. Either way, it was a burdensome process, which wasted your time and resources.

Introducing IP Anonymization

We’ve made changes to our public API which enable you to use our enrichment functionality while maintaining GDPR compliance. We’ve extended the `ip_to_geo` addon syntax to enable the use case where the field holding the IP address is removed after the IP to Geo enrichment takes place. Furthermore, this change is backward compatible so it won’t break your existing code.

To enable the feature just add the `remove_if_property` key to the `input` map with value `true`. After the IP to Geo processing takes place, the source property is removed.

Input event

{
  "ip_address": "${keen.ip}",
  "keen": {
    "addons": [{
      "name": "keen:ip_to_geo",
      "input": {
        "ip": "ip_address",
        "remove_ip_property": true
      },
      "output": "ip_geo_info"
    }]
  }
}

Output event

{
  "ip_geo_info": {
    "city": "San Francisco",
    "province": "California",
    "country": "United States",
    "country_code": "US",
    "continent": "North America",
    "postal_code": "94122",
    "coordinates": [-122.42005, 37.77479]
  },
  "keen": {
    "created_at": "2012-12-14T20:24:01.123Z",
    "timestamp": "2012-12-14T20:24:01.123Z",
    "id": "asd9fadifjaqw9asdfasdf939"
  }
}

Given that storing your customer IP addresses requires consent and imposes some legal requirements regarding data management, if it’s the location that matters in your event and not the IP address then this new feature of the `ip_to_geo` addon is your answer. It gives you an easy, time-saving way of maintaining your data GDPR compliant as you no longer need to manually remove it from events. Try it out and let us know what you think!


Giving our customers a voice - Keen

Giving Our Users a Voice

A quick note’s probably in order since I’m sort of a newbie on the team. My name’s Florian. I was fortunate to join Keen a few months ago as VP of Product Engineering. Some of you probably know me already from demos, support, calls, etc. I’ve been passionate about building analytics and metrics products for a long time. It goes back a decade or more. It started out with me trying to visualize the spread of SQL Slammer in school. It’s been a fun journey to get here - filled with time series databases, horizon charts, Holt-Winters Forecast’s, and bespoke cloud storage platforms. But I’m even more excited about what's to come and what I can do to help Keen make our customers' jobs easier.

We were acquired by Scaleworks a little over 8 months ago. During this time we’ve been hard at work behind the scenes. We’ve been building out our team, optimizing our infrastructure, and getting ready move the Keen platform forward to the next stage. Along the way we also took a bit of a detour to work on GDPR, that, of course, consumed quite a bit of time. Now we’re ready to start moving forward.

We may be a bit biased, but we think the Keen platform is fantastic (and it's clear you do as well). Of course, we also know that like any platform it has warts and shortcomings. Good news is that we have a lot of room to improve. Analytics is not a solved problem. Implementing any analytics solutions is still hard and still frustrating. We’re working to solve that and we’ll have more exciting news about it in the coming months! Through customer interviews, support tickets, and account reviews, our users have been fantastic about helping us start to plot out our future roadmap. However, we’ve felt like we’ve only been able to scratch the tip of the feedback iceberg.

Giving users a voice

I’ve was lucky to have been able to use canny.io previously and knew I wanted something similar for Keen customers.

Building a strong relationship between our user community and our team is critical to the ongoing success of Keen. To that end, we want to make sure that our users always have a voice.

Today we’re happy to be launching our product feedback board using canny.io’s awesome platform. feedback.keen.io is the place you can go to post your feature requests, tell us about things that would make your job easier, or see what we’ve got inflight. We’ll share mockup’s and use it as a sounding board as we turn ideas into actual features. Not everything here will get built but we promise to always keep the communication going and you’ll definitely have a voice as Keen grows.

Collaborate and Engage

Using our product feedback board will help us prioritize, balance, and clarify feature requests. It's a great way for us to spot what users need from us to make their day to day jobs easier. We can use it to validate new features we’ve cooked up internally, or get feedback on mockups as we iterate on UX. The collaborative nature means we can share and get feedback from Keen users much faster.

With API focused products like Keen, launching a new feature often means that end users have to do a bit of dev work to actually utilize them. The fact that we can give users a peek at what's coming, as well as let them preview how new features are being structured makes consuming them much easier. One of our driving goals is going to be to make the jobs of our users easier, and feedback.keen.io will let users adopt new features with a little less work.

Honestly, it makes our jobs a bit easier as well. Product boards like this make it easy for the Customer Success, Product, and Engineering teams to keep our customers up-to-date and engaged. CS can send follow-ups when you reach out with a feature request via support, our Account Managers will eventually be able to share a living roadmap during account reviews, I’ll be able to easily see which customers I can chat with after a feature launch to see if we hit the mark, and our engineers have a much more direct line to customers for clarification.

So, come and help us lay down a path for Keen:

  1. We’re planning support for AMP - AMP Support
  2. The ability to anonymize IP Address to help you be GDPR compliant is in progress - Anonymize IP Addresses
  3. We’d love feedback on what sort of Spend Limits/Alerting we should implement - Spend Limits
  4. And there’s more (from OR Filters, to SSO Logins, to Dashboard SDK updates), and whatever else you want us to know about!

(Hint: That exciting news we’ve got in store… that I mentioned earlier? Keep an eye on feedback.keen.io over the next few weeks and you’ll get early insights to it as we start adding cards/features to the board)