Book Results

The Results Assessment System

The evaluations used by human resource development (HRD) professionals have needed improvement for a long time. Human resource development managers usually don’t even measure the results of their departments’ work in organizational learning and performance. Perhaps this is because - until now - the available evaluation and assessment resources have been incomplete and barely usable.

However, numerous organizations are employing this new Results Assessment System successfully. The system is not only highly usable, it’s also accessible, and consistent with current research and theory. The five-step process includes a straightforward planning mechanism and practical measurement tools. The system is designed to assess the performance, learning and perception results from HRD programs and from other developmental efforts, whether spearheaded by leaders, managers or work groups. Other professions concerned about performance, learning and perception results will also find the system useful.

"If human resources development is to be a core organizational process, it must act like one and hold itself accountable for obtaining results through its interventions.

The five steps of the Results Assessment System are:

1. Analyze - Specify expected results.
2. Propose - Plan assessment of results.
3. Create - Develop measures of results.
4. Implement - Collect and analyze results data.
5. Assess - Interpret and report results assessment.

“Despite decades of preaching, cajoling and encouraging, the HRD profession still typically does not assess the results of HRD interventions (in general or specifically, within organization development or personnel training and development).”

To be most effective when assessing results, first clearly define and analyze your goals. These five steps form a logical progression of activity and end with definitive action. The final outcome enables you to make decisions and take action to maintain, change or eliminate programs. The ultimate purpose of the Results Assessment System is action, not research.

Performance and Mission

Performance outcomes are directly connected to the system’s mission. Every organization has a mission. The development of strong, competitive, growing organizations requires focus on performance results. These can be categorized as system results or financial results. Thus, this assessment system looks at results and measures performance. For example, Ford makes cars and the Union Gospel Mission saves souls. The measurement of the performance of such diverse systems is based upon three classifications:

1. The organization - Measuring the performance of the whole system.
2. Subsystems - Measuring the performance of the work processes within the system.
3. Individuals or work groups - Measuring performance at the contributor level.

A performance measurement for Ford might look like this:

• System level - Number of cars sold in Europe.
• Work process level - The time it takes to complete the final inspection process working with a completed car at the end of the assembly line.
• Individual or work group level - The number of patents received by the engine system research and development group.

“All organizations rely on human knowledge and expertise in order to establish and achieve their goals. Thus, the concern for learning is fundamental. HRD professionals and organizational leaders should not wavier on this fact.”

A performance measurement for the Union Gospel Mission might look like this:

• System level - The number of clients who are sober and self-supporting.
• Work process level - The number of meals served by the meal planning, preparation and serving process.
• Individual or work group level - The number of people responding positively to the one-on-one counseling program.

“The corporate school and human relations models of development practice that exist simply because they are ’good for employees’ are outdated.”

Since most organizations already have methods in place for measuring their core production systems, human resource development professionals only need to assess processes that are involved in systems or subsystems that HRD is trying to improve.

Within the Results Assessment System, learning results are divided into knowledge and expertise. When testing to measure knowledge, stick to multiple choice questions and avoid true-false questions, because a person has a 50% chance of choosing the correct answer without actually knowing anything.

Measuring expertise requires that a person demonstrate that expertise in a real or simulated setting. In some cases, the limited role-playing and simulation experience at the end of a HRD program are enough to measure expertise. In other instances, supervised fieldwork is the best measurement method.

Measuring Perception

How can you measure a person’s perceptions? A perception is not objective. It’s very subjective - the personal feeling, impression, belief or comprehension of an object, person, event or quality that may or may not be factual.

“Why assess and report results? Imagine for a moment an organization that does not measure its financial results. How would its members know if the organization were successful?”

The Results Assessment System classifies results concerning the status of (or a change in) perceptions in two categories, participant perceptions and stakeholder perceptions.

1. Participant perceptions - These are the perceptions of people who have firsthand experience with systems, processes, people, goods or services.
2. Stakeholder perceptions - These are the perceptions of those who lead systems and people. They have a vested interest in the desired results and the way of achieving them.

“Performance drivers are vital to organizational success.”

Obviously, personal perceptions of employee performance and of the outcome of a class or seminar aren’t good, reliable measurements of actual performance and results. However, all organizations are influenced by the power of human perceptions, even when credible, objective measures of performance and learning are available.

Companies must pay attention to the perceptions of their employees and stakeholders, even though those perceptions may be less important when it comes to assessing facts. Organization leaders and HRD professionals need an overall measurement and assessment system for evaluating perceptions and results - and any changes in them - and then interpreting their impact.

“Our original motivation for writing this book was to try to do something significant about improving the state of evaluation practice in human resource development.”

One of the purposes of a magazine such as Consumer Reports is to counter false perceptions with facts. The same can be said of the Results Assessment System.

Keep on Trucking

When Lee Iaccoca became the president of the Chrysler Corporation, the facts clearly showed him that Dodge trucks were as durable as his competitors’ trucks. However, potential customers perceived Dodge trucks as less durable, and thus less desirable than the competition’s trucks. When Iaccoca looked into customers’ perceptions, he found that they had very positive feelings about the old Dodge symbol - the ram’s head. He also discovered that they perceived older Dodge vehicles as more durable than new models.

“Professionals in the business of developing good measurement devices are focused on the reliability and validity of measurement instruments. A reliable instrument yields consistent results.”

To solve the perception problem, Iaccoca brought back the well-regarded Dodge Ram symbol by placing a ram hood ornament on each Dodge truck. He also began a new TV ad campaign in which he personally talked to viewers about how tough Dodge trucks were. As he spoke, an actual Dodge truck was dropped from above onto the street behind him. It came crashing to the ground unharmed as Iaccoca lauded its toughness.

As a result of this ad campaign, Dodge truck sales quickly rose significantly. Customer perception changed - Dodge trucks were much more durable than they previously thought. However, the trucks never changed. And their actual durability - which was always excellent, whether the customers perceived it previously or not - had not changed.

Perceiving Is Not Believing

Perceptions are important, but obviously they have their limitations. Therefore, it’s important to be absolutely clear about them. For example, if you want an assessment of air quality, measure actual air quality, not perceptions of it. If you want to know if an organization performed better as a result of a particular learning program, don’t ask people’s perceptions of performance change. Instead directly measure and assess performance before and after the learning program.

“Perceptions of performance and learning outcomes are not good measures of actual performance and learning outcomes.”

Assess both perceptions of process and perceptions of outcomes. "Process perceptions" about ongoing processes are commonly collected from participants such as employees, customers and others who are directly involved. Sometimes you may also collect perceptions from stakeholders such as managers, shareholders and citizens. You might want their perceptions of a work process, a team-building session or a program. These measurements shed light on people’s enjoyment and satisfaction, or on a program’s usefulness and applicability.

“Effective communication of results assessment findings is essential if the findings are to be supported and used by decision makers outside the HRD function.”

"Outcome perceptions" concern actual outcomes, not expected ones. These are the most important perceptions to collect from stakeholders, who are usually the people most interested in such information. When participant perceptions are collected from people who have firsthand experience, they feel that they are speaking with authority.

Elements of the Results Assessment Report

A Results Assessment Report is a user-friendly summary report designed to be useful to decision-makers, managers and participants. This report has eight sections: a heading identifying the organization and the program, the program’s purpose, the program’s description, the summary of results, approval (to be signed by the authorizing executives), a list of recipients (distribution), assessment results and a proposal for improvements.

Implementing the results of a systems assessment must be managed as an organizational change process. Begin by implementing what the culture will allow. Build acceptance by selling the incentives and rewards, and above all, make it a learning process that can benefit everyone involved. Remember that sound decisions about programs stem from more than just assessment data.

When human resource development professionals implement the Results Assessment System, human resource development will be taken seriously as an organizational partner dedicated to producing positive results. This will certainly benefit every aspect of the company or organization. Results assessment builds confidence and integrity, and proves that learning and other development activities can improve performance. Once their efficacy is proven, such human resource development programs will carry more weight and will no longer be seen as frivolous extras.