Co-owned the modernization of LaunchDarkly’s Experimentation Platform, strengthening competitiveness in enterprise deals.

Problem

We were losing enterprise deals to competitors whose experimentation products were more mature and carried features customers had made non-negotiable. While engineers felt comfortable with implementation, project managers had difficulty creating experiments, and data scientists couldn't fully trust the result analysis. Leadership's take in 2024 was that our feature gap against competitors was behind by about eighteen months.

During our research and customer interviews, four themes were identified: Users could not build an experiment quickly, run one efficiently, fully trust the analysis, and share results amongst their stakeholders.

Building and setting up: Setup was the entry barrier. Metrics and experiment configuration were cumbersome for new users, there was no way to draft an experiment, refine it, and get approval before it went live, and a minor change to a design forced a completely new iteration. Users also had no accurate way to estimate how long an experiment would need to run before committing to it.

Running it efficiently: The time it took to reach a trustworthy result was the deterrent. Fixed traffic allocation wasted exposures when a winner was already emerging or when several variations were in play, and concurrent experiments conflicted with each other, with no reliable holdout for measuring cumulative impact.

Analyzing results: Inconsistent language and poor data presentation made results hard to parse, and there was no transparency into how a number was calculated. The analysis ran too shallow for teams with mature data practices who needed advanced statistical methods, and there was no robust way to export results into a customer's own environment without configuring events inside LaunchDarkly.

Sharing and deciding: There was no quick, insightful summary of what an experiment actually found, and no way to share a result with stakeholders who did not have access to the platform.

Solution

We created an extensive experimentation roadmap that set out to revolutionize how users design, execute, analyze, and integrate product experiments, equipping them with cutting-edge tools to drive more effective, insightful experimentation.

1. Design and flexibility: focus on configurability, ease of use, and advanced features for experiment design
2. Advanced metrics and data analysis: deeper insights and more precise data handling by allowing users to define and analyze metrics dynamically, offering detailed segmentation, and real-time data analysis
3. Data warehouse: enhance data export capabilities and integrate advanced features that enable users to manage experiments directly within their data warehouse environments
4. Enhance user experience onboarding: build a streamlined onboarding processes and delightful, experimentation-first UX, that enable users to start experimenting quickly and effectively
5. Comprehensive Results Sharing and Insights: add results sharing capabilities and AI-generated experiment reports that increase collaborative decision-making for product teams

Impact

By late 2025 we closed the feature gap and the platform competed head-on with Statsig, Amplitude, and Datadog Eppo, while warehouse-native support reached parity across the big four warehouses. Experiments created per month roughly doubled and the users creating them more than doubled over the period I designed the surfaces. Deals significantly increased, including a recent $1.4M annual contract.

• Eurostar cut experiment setup from weeks to days and reported a 36% lift in bookings for their business
• Rivian reported a 10% lift in mobile demo-drive conversions