All Projects

Giving customers clear visibility and faster compliance reporting for the security data lake.

Team

Red Canary

Shipped

Q3 2024

Pain points

Customer data was not retained past a certain time period.
Customers had no way to access their data unless it was associated with a security threat.

My role

Led the UX strategy and design, collaborating with product and engineering.
Defined workflows for monitoring, exporting, and querying data lake content.

Outcomes

SDL shipped on time with strong customer praise, especially enterprise accounts.
Reduced compliance-driven churn risk.

Pain points

Customer data was not retained past a certain time period.
Customers had no way to access their data unless it was associated with a security threat.

My role

Led the UX strategy and design, collaborating with product and engineering.
Defined workflows for monitoring, exporting, and querying data lake content.

Outcomes

SDL shipped on time with strong customer praise, especially enterprise accounts.
Reduced compliance-driven churn risk.

Summary

We built the Security Data Lake (SDL), a new feature enabling customers to query their Red Canary data in real time. This tackled compliance challenges, elevated customer satisfaction, and brought immediate value to users demanding more transparency and control over their data. SDL addressed a competitive gap that could have otherwise lost us customers.

The problem

Customers struggled with lack of visibility into the vast amount of data they sent to Red Canary. There was no easy way to query, analyze, or even view the data beyond Red Canary’s detections. This limitation created two big problems: compliance headaches (especially for enterprise customers in regulated industries) and frustration over not being able to dig into their own data when investigating security events.

"How do we handle search quotas?"

When giving customers the ability to query massive amounts of data (meaured in TB), how do we keep customers from eating up their query limit in one search?

Give the customer an estimate of how big a query will be before they run it.

This is not a real problem, customers will know how big a query is on their own.

If a query is too big, stop it once it reaches a certain threshold.

Let the customer set a rate limit for query size.

Key Decision

Inform customers of how much data is used for every search they run.

The solution

SDL had two major components: a usage dashboard and a SQL query tool. We chose a phased MVP approach:

Ship the dashboard first to address compliance and visibility needs
Follow with the SQL query tool for deeper investigation

What went right

Scope changed a lot here but we managed to piece it out well enough to ship on time.
The Data Lake Search feature received instant positive feedback and won us a few new contracts.

Tough spots

Designed the Data Lake Search was tricky because we needed a LOT of new design patterns.
Alert customers to "big queries" before they execute them to avoid hitting search quotas too fast.

How we solved it

2 Months

Design time

14

Early access customers

8

Major iterations

Identifying Customer Needs

We prioritized customer input from support tickets and recurring feedback themes. It became immediately clear that users needed real control and access to their data—not just limited visibility through pre-set dashboards.

MVP Approach

We decided on a phased prioritization: first, a dashboard providing a high-impact “snapshot” view of critical data usage and compliance at a glance. Next, we would deliver a SQL Query Tool as a secondary release with powerful search functionality for more technically advanced users.

Design and Testing

I designed a dashboard that offered at-a-glance metrics like data usage (broken down by integration), historical trends for compliance reports, and intuitive export functionality.
For the SQL Query Tool, I leveraged familiar UX patterns from existing query tools to design a straightforward and functional experience, ensuring customers could execute, save, and manage queries with ease.
Throughout the build process, I worked closely with engineering to validate feasibility, and we tested prototypes with a group of early adopters to refine usability.

Lessons learned

Graphs and charts are fun, but make sure you're using the right one for the data you're presenting. The wrong style can communicate the wrong message.
Sometimes it's best not to reinvent the wheel. Its ok to use familiar UXs from other products if the pattern is universally recognized (SQL querying).