RyanShrum

Henry Meds

From first click to first order

Three projects from my time as a lead engineer, and later the engineering manager, of the Growth team at a fast-growing telehealth company. Growth owned the path a patient takes from first click to first order, so the work came down to two questions: what are patients actually doing, and how do we change it safely?

Team projects, described by what I owned. Teammates built much of the rest, and I’ve said where.

01 / 03

First-party event tracking

A tracking pipeline I conceived and built at a hackathon, recording funnel events from the app itself.

The problem

Our funnel analytics ran as a third-party script in the patient’s browser. Ad blockers and privacy features quietly dropped events, and what got through was coarse. We were deciding what to test and fix from data we couldn’t fully trust, on the steps that mattered most: choosing a time, entering details, paying.

What I owned

I built the first version, which we called Sherlock, end to end at one of the engineering org’s monthly hackathons. It was in production within weeks.

The client

An “evidence” recorder wired through every onboarding page: an event, a category, and open-ended clues. One session follows a patient across subdomains, with the treatment and customer attached once known and campaign parameters captured automatically.

The pipeline

A first-party endpoint on a Cloud Run function that validates the required fields and writes each record to BigQuery.

The analysts

Analysts specified the clues for each event, and a standing check-in every two weeks confirmed each flow was instrumented as intended.

Teammates later moved the client into a shared package for the forms app and extended it.

Results

Analysts built conversion-funnel dashboards on it, showing drop-off by page and by individual field. As teammates extended it, it came to capture more than 7% of traffic that ad blockers and privacy features used to hide.

From a click to a row in BigQuery The onboarding app sends an evidence record to a first-party Cloud Run function, which validates it and writes it to BigQuery, where analysts build funnel dashboards. Below, the old path: a third-party script that ad blockers and browser privacy features interrupt. Onboarding app evidence record Cloud Run validates, first-party BigQuery one row per event Analysts funnel dashboards Third-party script ✕ blocked by ad blockers and browser privacy
The path a funnel event takes now, and the one it used to take.
{
  "event": "DateSelected",
  "eventCategory": "ButtonClick",
  "eventSource": "onboarding",
  "sessionId": "<uuid>",
  "clues": {
    "order": 2,
    "appointmentCount": 14,
    "deltaInDays": 3,
    "treatment": "<treatment>",
    "searchParams": { "utm_source": "<source>" }
  }
}
An evidence record, simplified. Field names follow the real shape; the values are invented for illustration.

Decisions worth telling

Open-ended clues instead of a schema

Every record carries a few required fields, and everything else lives in clues. A new question from the analysts meant one line of instrumentation, not a migration.

Keep the test traffic out from day one

The day after launch I stopped automated and internal test traffic from being recorded. It looked like real patients, and would have skewed exactly the numbers the system existed to fix.

Tell intent from inertia

On the appointment step, each record notes which slot a patient picked, how far out it was, and whether it was simply the default. That separates patients who take the earliest time from those choosing what works for them.

React / TypeScript Cloud Run BigQuery Datadog

02 / 03

A standard for experiments

Choosing the A/B platform, building the first integration, and making experiments safe for every front-end team to run.

The problem

We needed experimentation we could rely on: tests that were safe to run in a patient-facing flow, results tied to what patients actually bought, and a way to tell whether a test had reached significance and passed its goal. Without that, a front-end team could ship a variant and still not know what it had learned.

What I owned

I evaluated LaunchDarkly, VWO and Kameleoon, ran the proof of concept, and built the first integration in the onboarding app.

The integration

Feature flags, an A/A test to confirm the setup produced no false signal, and client-side conversion goals tied to the checkout.

Safe to live with

A non-production flag menu, built at a hackathon with teammates. Bots and vendor outages get the control experience instead of an error, so the app keeps working if Kameleoon doesn’t load.

Clean endings

Every flag goes from launch to general release to deletion, so finished experiments don’t pile up as dead branches.

The analysts

I worked with the data analysts to get whatever they needed to call an experiment: significance, and pass or fail against the goal.

Teammates later reworked parts of the integration.

Results

The integration became the standard across front-end teams, I became the team’s go-to on it, and Kameleoon named me a 2024 Top All-Team Experimentation User. Two of the tests it ran changed how the team thought about the funnel.

The plain list that kept winning

Patients picked their state from a list of buttons. The team tested it against a state typeahead, with and without geolocation, and ran the test twice. Both times the list of buttons won, though it looked like the weaker design, and it stayed.

Most patients keep the default

A test of how subscribe-to-save memberships were ordered, read alongside first-party tracking of which options patients touched, showed most patients went with the option already selected. The order mattered more than the options.

Kameleoon certificate naming Ryan Shrum a 2024 Top All-Team Experimentation User.
Kameleoon’s 2024 recognition.
The life of an experiment Six steps in order: validate with an A/A test, turn the flag on for a slice of patients, measure against the goal, call it, roll the winner out to everyone, and delete the flag. Underneath, two safeguards apply throughout: tests can force a variant, and bots or a vendor outage get the control experience. A/A test Flag on Measure Call it Roll out Delete the flag no false signal for a slice against the goal pass or fail to everyone no dead branches Tests choose their variant An override makes end-to-end tests deterministic Fail to control Bots and vendor outages get the default experience
The path every flag takes, and the two safeguards around it.

Decisions worth telling

Make tests deterministic

End-to-end tests were being bucketed into experiments they weren’t written for, and failing at random. I built an override that lets a test choose its variant, so the suite could check A or B on purpose, and never by chance.

Prove the setup before trusting it

The first test was an A/A test: two identical experiences that should show no difference. Only once it came back clean did real results count.

Fail to the control experience

If Kameleoon doesn’t load, or the visitor is a bot, every flag reads as off. A vendor problem costs us an experiment, never a patient’s checkout.

React / TypeScript Kameleoon Feature flags Playwright

03 / 03

Onboarding that guides

Led the rewrite of onboarding around durable state, so patients are never asked for what we already know, and a return or refresh lands on the right step instead of the start.

The problem

The onboarding flow was a chain of pages, each holding its own state. A patient could land on a later step without the data it needed, which in turn let them create duplicate appointments or subscriptions. Returning patients were treated as brand new: with no good path to place a second order, every order looked like a first visit, and duplicate accounts followed.

What I owned

The rewrite

I led the move to a central store for appointment, checkout, provider and auth state, working with two teammates. Non-sensitive answers are saved for the session, so a patient who comes back doesn’t redo a step.

Routing that follows the data

It started as a hackathon proof of concept and shipped: each step declares the data it needs, and the router works out where a patient belongs. Anything already in the store is never asked for again, so known steps are skipped, and a step missing what it needs sends you back to the one that has it.

Sign-in for checkout

Firebase sign-in, built on the platform team’s token exchange and released behind flags, plus a pattern that keeps protected GraphQL queries apart from public ones in the generated client.

Recognizing a returning patient

When an email already belongs to a customer, the patient is asked to sign in before anything about the account is shown. I worked with product and design to close gaps in the original sign-in design, so the first version could be extended without a redo.

Results

Fewer duplicate appointments and subscriptions, less to redo for a patient who leaves and comes back, and a foundation the team kept building on, including bulk ordering and new treatments.

How onboarding decides the next step Five steps in order: treatment, state, appointment, contact details and payment, all reading from one central store saved for the session. A patient who reaches payment without an appointment is sent back to the appointment step. At contact details, an email that already belongs to a customer means signing in first, with nothing about the account shown until then. no appointment yet? sent back Treatment State Appointment Contact details Payment Email already has an account? Sign in first. Nothing about it is shown until then. One central store, saved for the session. Each step declares what it needs.
The route follows the data: what’s already known is skipped, a missing answer sends you back, and a return or refresh lands on the right step.

Decisions worth telling

Prove the rewrite with an A/A test

A rewrite this deep had to cut over all at once. I put the old and new flows behind a flag as an A/A test: identical to patients, completely different underneath. We only switched once the two converted at the same rate.

Let the route follow the data

Instead of each page guessing what came before it, every step says what it needs and the router decides. Duplicate appointments and subscriptions came from patients acting on pages with nothing behind them, and this closed that door.

Recognize without revealing

The existing-account lookup only triggers the sign-in prompt. Typing in someone else’s email tells you nothing about them.

React / TypeScript Redux Toolkit Firebase Auth GraphQL Playwright

Beyond the code

Observability and incidents

I co-owned observability for the team, then owned it outright as manager. I decided incident severity, commanded incidents for service issues, wrote the incident reports and drove the follow-up fixes. I set alerting by value: how often something fires against how risky it is, so the alerts that matter stay loud.

Growing the team

I interviewed candidates and reviewed code assessments for the nine people on the Growth team, mentored the engineers who stayed, and helped prepare one for promotion to Senior. I started as the first engineer dedicated to Growth, served as the first manager’s tech lead, then took over as manager.

Testing as a team habit

I took the team to ownership of the onboarding end-to-end suite, taught engineers to write and debug Playwright tests, and kept a good working relationship with the engineers in test.

A component library, directed

I directed the team’s component library, an implementation of the Untitled UI design system in the Material UI ecosystem, built mostly by teammates.

What ties them together

Trust the data before you optimize

Each project started from not being sure what we were looking at: a funnel, a test result, a rewrite that had to convert like the original. Proving that came first.

Make the safe path the default

Flags that fall back to the control experience. Routes that send a patient back instead of letting them proceed. An existing-account check that reveals nothing until sign-in.

Own the follow-through

Incident reports and fixes, flags cleaned up after launch, a standing sync with the analysts. The ending of a piece of work matters as much as shipping it.