Guila Muir and Associates

Home » Text Book

Category Archives: Text Book

How Do You Know They Know? Evaluating Adult Learning

How can you evaluate training effectiveness? “Happy Sheets” don’t go far. Outcome-based evaluations meet criteria in Kirkpatrick’s evaluation model.

Learn more about evaluating adult learning

Mature students learning computer skills

How Do You Know They Know? Evaluating Adult Learning

by Guila Muir
info@guilamuir.com

I con­tinue to be sur­prised at the use of “Happy Sheets” as eval­u­a­tion tools in train­ing. Beyond let­ting the trainer know if he or she was loved and if the room was too cold, what else do they tell us?

In 1959, Don­ald Kirk­patrick devel­oped his famous model of train­ing eval­u­a­tion. Since then, it has pro­vided basic guide­lines to assess learn­ing. Experts have found that 85% or more of all train­ing pro­grams use “Happy Sheets,” which reveal noth­ing about actual learn­ing. And because data is much harder to col­lect and attribute directly to the train­ing the deeper you go, fewer than 10% of train­ing pro­grams use a Level 4 evaluation.

Level

Issue

Ques­tion Answered

Tool

1

Reac­tion

How Well Did They Like The Course? Rat­ing Sheets

2

Learn­ing

How Much Did They Learn? Tests, Sim­u­la­tions

3

Behav­ior

How Well Did They Apply It To Work? Per­for­mance Measures

4

                  Results What Return Did The Train­ing Invest­ment Yield? Cost-Benefit Analy­sis (Return on Investment)

Outcome-Based Eval­u­a­tions

By cre­at­ing and using an eval­u­a­tion based on the course’s learn­ing out­comes, you may get closer to an hon­est answer to the ques­tion, How Do You Know They Know? which is eval­u­a­tion at Level 2 of Kirkpatrick’s model. Typ­i­cally, the outcome-based eval­u­a­tion would ask par­tic­i­pants to rate their own abil­ity to per­form the learn­ing out­come, as in the fol­low­ing example:

As a result of this train­ing, please rate your abil­ity to do the fol­low­ing action from 1 (“I can’t do this at all”) to 5 (“I feel totally con­fi­dent doing this”): “I can explain at least five fea­tures of the Get Fit pro­gram with­out using notes.”

In many cases, the outcome-based eval­u­a­tion would also ask the par­tic­i­pant to list or explain those five features–in this way, act­ing as a test.

Keep in mind that unless you ask addi­tional ques­tions, you are still sim­ply col­lect­ing data on your par­tic­i­pants’ per­cep­tions of their own learn­ing. Sadly, those per­cep­tions of learn­ing are usu­ally much higher imme­di­ately after the train­ing ses­sion than a few days or weeks later. This is why follow-up train­ing and rein­force­ment is so important.

Nonethe­less, using an Outcome-Based Eval­u­a­tion can pro­vide infor­ma­tion on:

  • Per­for­mance issues about which the par­tic­i­pants feel less confident.
  • Issues you could improve or clar­ify for the next round of training.

All of this data is valu­able to you as you (1) improve the class itself, and (2) fol­low the par­tic­i­pants into the work­place to observe and sup­port them. We invite you to down­load free exam­ples of Out­come Based Eval­u­a­tions from Guila’s book, Instruc­tional Design That Soars.

Get your participants involved with this tip

A great article about how to get people involved immediately in your training sessions and presentations:  What is a hook?  Keep you presentations exciting, engaging, active with Guila Muir’s Kite Method of Instructional Design. Instructional Design That Soars is the textbook of choice when learning how to develop presentations, webinars, classes that will inspire your students.iStock_000013091322XSmall-256x300

These characteristics can be developed through attitude, habit and discipline

instructional design qualities

If you are an emerging leader, read more about the mind-set of these phrases.

You don’t need a certificate in instructional design to create great classes.

Instructional Design That Soars: Kite Method of Designing Classes

You don’t need a certificate in instructional design to create great classes. The Kite Method makes it easy. http://ow.ly/vKi7A71XkqAZsd-L

The kite flies because it has a tail

active closure

Subject Matter Experts

subject matter experts

This book will make developing and presenting easy. The Kite Method by Guila Muir makes it easy to turn your presentations into inspiring and engaging learning lessons.kite-book-1

Learn more here

See how the Kite instruc­tional design method works in real life!

Case Studies Using the Kite Instructional Design Method

See how the Kite instruc­tional design method works in real life!

“Hav­ing our cur­ricu­lum on paper means that we main­tain con­trol over it. It pro­vides con­sis­tency in our train­ing pro­gram, no mat­ter who the instruc­tor is.”
– Bar­bara Court­ney, Exec­u­tive Direc­tor, Artist Trust

Artist Trust (AT) is a medium-sized non-profit that deliv­ers infor­ma­tion ser­vices to artists through­out Wash­ing­ton State. AT selected Guila Muir’s KITE process to develop 44 hours of busi­ness skills train­ing for approx­i­mately 250 artists over a five-year period. The cri­te­ria for the train­ing, to be devel­oped by four con­tracted art pro­fes­sion­als with no back­ground in train­ing, included:

  • The final cur­ricu­lum had to be con­sis­tent so that each cohort received the same infor­ma­tion over a five-year period;
  • The train­ing ses­sions had to be inter­ac­tive and participatory;
  • The final cur­ricu­lum had to be writ­ten so that any sub­ject mat­ter expert could eas­ily deliver the training.

Guila helped AT staff deter­mine the con­tent and flow of the cur­ricu­lum. She then pro­vided the 2-day Train­ing Design class to the 4 sub­ject mat­ter experts. At the end of these two days, the par­tic­i­pants had devel­oped a draft of their first train­ing session.

Guila pro­vided the devel­op­ers with 30 hours of input and con­sul­ta­tion over the next 6 weeks. By the end, they had final­ized a total of 18 mod­ules, vary­ing in length from 1.5 — 6 hours long. (Note: all the devel­op­ers were employed full-time in other pur­suits. Work­ing with KITE was not their only task.)

In less than 6 weeks, the SMEs developed:

  • 18 dif­fer­ent Instruc­tor Guides–one per train­ing mod­ule. (The Instruc­tor Guides pro­vide the trainer’s “script,” includ­ing con­tent and clear instruc­tions that describe how to con­duct exer­cises and activities.)
  • Par­tic­i­pant work­sheets, hand­outs, and other sup­ple­men­tal learn­ing materials.
  • All nec­es­sary visual aids.

ROI

Because the experts them­selves wrote the cur­ricu­lum, Artist Trust paid less than one-quarter what they would have if an out­sider had done so, even at below-market con­sult­ing rates. Devel­op­ment time was cut by a third.

Out­comes

“Hav­ing replic­a­ble train­ing ses­sions on paper makes it eas­ier to eval­u­ate our pro­gram.”
–Bar­bara Court­ney, Exec­u­tive Direc­tor, Artist Trust

Although Artist Trust peri­od­i­cally brings in new instruc­tors, the train­ing remains con­sis­tent, facil­i­tat­ing pro­gram eval­u­a­tion. Artist Trust is now con­sid­er­ing mar­ket­ing all 18 mod­ules nationally.

More KITE suc­cess stories!

Do the fol­low­ing sound like “dry” sub­jects to you?

  • Con­tract Monitoring
  • Audit­ing
  • Con­tract law
  • Con­tract Management
  • The Com­pet­i­tive Process
  • Essen­tial and Advanced Con­tract Drafting

Wash­ing­ton State’s Depart­ment of Social and Health Service’s Cen­tral Con­tract­ing Ser­vices (CCS) deter­mined a need to develop train­ing ses­sions on these top­ics that would keep peo­ple awake! The ses­sions would be pre­sented statewide by a num­ber of dif­fer­ent sub­ject mat­ter experts.

In less than 3 months, using the KITE process, six Con­tract­ing Spe­cial­ists with lit­tle or no train­ing back­ground devel­oped 8 writ­ten cur­ric­ula. The ses­sions are inter­ac­tive and lively, and par­tic­i­pants have writ­ten on their eval­u­a­tions com­ments such as “I learned a TON. I never thought con­tract mon­i­tor­ing could be so fun!”

idtsworpress

3 Tips to Motivate Adult Learners

motivate adult learners

3 Tips to Motivate Adult Learners

Why bore your students with lectures and slideshows? Why not energize them as they learn?

Flying High With The Kite Method Of Instructional Design

Fly your kiteWhy bore your students with lectures and slideshows? Why not energize them as they learn?

Instructional Design That Soars: Shaping What You Know Into Classes That Inspire enables you to design lively classes and Webinars. It takes you step-by-step through the Kite method of fast, fun instructional design. Packed with downloadable templates, worksheets, checklists, exercises, and resources, you’ll return to this guide again and again.

This summer I won the Harvard Summer School Excellence in Teaching Award. Part of it was due to my course design. You gave me one of the only applicable skill sets in grad school. I loved the pedagogy, but your was news we could use. Cheers, Guila!

Cynthia Weintraub

Receive Two FREE Resources With Your Autographed Copy

You’ll save $3.00 by ordering through our website. We’ll also include two invaluable checklists:

  1. How to Build an In-House Training Team
  2. Criteria for Selecting In-House Trainers

Cost: $21.95 (usually $24.95) plus shipping and handling charges. All taxes are included. We’ll send your books USPS First Class. No international orders, please.

Order Your Copy Today! 

Special price with 2 FREE Resources $21.95 USD

Instructional Design That Soars is not yet available on Amazon.com.

Persuasion is the key

convince

Learn more about Instructional Design That Soars here

%d bloggers like this: