When your board asks how the education program did last year, what number do you reach for first? For many of us it's registrations, or completions, or total hours of content consumed, because those are the numbers the LMS hands over without any extra work.

Those numbers aren't useless. They tell you whether the marketing worked and whether people stayed until the end. What they can't tell you is whether anyone did anything differently once they got back to work, and that's the question that should matter the most — after all, association education should be helping your audiences learn how to apply what they've learned to improve their sector and organizations.

I don't think most associations skip outcome measurement because they don't care. Evaluation takes time and money, and education teams are already stretched thin. I've outlined five things associations, especially small associations, can do to measure the impact of their learning programs.

A quick refresher before we start, though! Kirkpatrick's four levels are:

1. Reaction (did they like it?)
2. Learning (did they learn it?)
3. Behavior (are they using it?)
4. Results (did it make a difference?)

A lot of association evaluation stops at Level 1. The approaches here roughly follow Levels 3 and 4, and they borrow one question from Phillips's ROI methodology and one suggestion from Robert Brinkerhoff's Success Case Method.

This is Kirkpatrick Level 3: are learners using what they learned once they're back on the job?

Send a short survey 60 to 90 days after the course. That's long enough for people to have had a chance to apply what they learned, and recent enough that they still remember it. Then ask about specific behaviors. So in place of "How satisfied were you with this course?" you'd ask something like "Since the course, how often have you used the resource [name it, e.g. risk checklist] in client engagements?"

Two things make these surveys more useful:

• Write the questions from the learning objectives. Each question should map to a skill the course taught. If an objective won't turn into a behavior question, that tells you something about the objective.
• Ask supervisors too, when you can. Their view of what changed is a helpful check on what learners report about themselves.

This one moves into Level 4, results. Before the course, ask each learner to name one work metric they hope to move. It should be something they or their employer already track, such as error rates, time to close, audit findings, client retention, or compliance incidents. Then follow up later and ask them to report the change.

Is self-reported data imperfect? Yes. But a metric the learner committed to in advance is far more credible than a vague "was this useful?" rating, because they named the target before they knew how things would turn out.

Then add the isolation question from Phillips: "What percentage of this improvement do you attribute to the training?" Other things changed at that learner's workplace over those months (new software, a new manager, a slow quarter), so this question keeps you from claiming credit for all of it. A modest number you can defend will serve you better in a board report than a large one you can't.

Associations have unusually good access to this kind of data. Your AMS and LMS hold results: education completion, certification records, and member profiles, so we can look at outcomes a commercial training provider would have to guess at:

• Exam pass rates for course-takers compared with non-takers
• Time to certification
• Recertification rates
• Promotions or role changes reported in member profile updates

Employers value this data because it links learning to workforce capability, and that's a much easier case to make to the person approving the training budget.

One caveat worth stating when you share the results: people who sign up for a prep course may differ from those who don't, so I'd present a pass-rate gap as a strong signal and stop short of calling it proof.

The first three approaches measure value to the learner. This one measures value to the association as well, which matters to the board.

Compare members who completed meaningful learning against similar members who didn't, and look at:

• Renewal rates
• Engagement depth
• Upgrades, such as premium tiers, conference registration, or volunteer roles

The word "similar" matters. If you put your most active learners up against the whole membership, the education program will look better than it may actually be. So match on things like member type, tenure, and career stage where your data allows, and decide up front what counts as "meaningful" learning (a certificate program, say, as opposed to one free webinar).

The four approaches above give you numbers. Robert Brinkerhoff's Success Case Method helps you understand the reasons behind them.

Interview a few of your most successful learners and a few of your least successful ones. (The behavior survey from the first approach is a good way to find both groups.) From the first group you're looking for concrete, documented results, something like "we cut onboarding time from 6 weeks to 4 using the framework from Module 3." From the second group you're trying to find out what got in the way.

It's inexpensive, and it pays off in two ways:

• You get compelling stories for marketing and board reports, with evidence behind them.
• You see where support after training needs to improve, whether that's job aids, a community of practice, or something else.

Decide on the outcome metric before you build the course, and write it into the needs assessment and the learning objectives.

Measurement is much easier when the course was designed to move a specific number. So if the goal is fewer audit findings, the needs assessment says so, the objectives name the skills that reduce audit findings, and the 90-day survey asks about those same skills. When the metric gets chosen after the course has launched, you end up trying to work out what a finished course was supposed to change, and that's a much harder job.

You don't have to do all five at once. If registrations and what more typical course evaluations cover (satisfaction levels, etc.) are what you have today, pick one course and try this:

1. Choose the one outcome the course is meant to move.
2. Check that each learning objective maps to something a learner would do on the job.
3. Ask learners at registration which work metric they hope to improve.
4. Send a short behavior survey at 60 to 90 days, with the isolation question included.
5. Interview a few people from the top and the bottom of the results.

Registrations will still belong in your board report, but they can be supplemented with learner indicators of the impact of the training.

So which of these are you already doing, and what has gotten in the way of the rest? I'd like to hear what's working for you.

Cheers, Cathy