Cognitive Ability Testing

Talkoclock Flickr Creative Commons

by William E Hamilton @ Iman Fiqrie

Knowledge and skill (KS) in a workplace-oriented assessment are generally used to help improve job performance. This includes careers, KS, abilities, and competencies. Getting at a reliable and valid assessment, be it work-oriented or personality-oriented can be more than a challenge, requires skill, and more than likely, the assistance of a subject
matter expert (SME). Russel (2010) discusses KS differences in individual performance and has three determinants; declarative knowledge (DK) (the facts we know), procedural knowledge and skill (PKS)(i.e., knowing what and how to do something), and motivation (reflecting choice and effort).

Complicating this, however, are “…[i]ndividual differences in general cognitive ability, personality traits, interests, education, training, experience, commitment, and values…[as] determinants or predictors of DK, PKS, and M” (Russell, 2010, p. 142). The problem with assessments is validity and reliability and is a give and take between the expense to “properly assess” an impact on job performance, operational and practical ability to conduct the assessment.

Analyze the use of assessments of knowledge and skill to determine declarative knowledge, procedural knowledge and skill, and motivation

As mentioned, DK assessments that use multiple-choice has proven to be both reliable and valid and “is more than a simple memory for facts. It requires an understanding of facts and principles and how they fit together” (Russell, 2010, p. 142). Tests for declarative knowledge lie on a continuum from comprehension to that requiring analysis and synthesis
at levels of cognition from knowledge, comprehension, application, analysis, synthesis, and evaluation (Russell, 2010). The process involves weighting that information and obtaining the advice of a subject matter expert (SME) “…to make adjustments in the final weighting” (p. 143). Even using the job analysis straight may come with issues, e.g., analysis
rating of some domains may be overemphasized, some items not testable, and generally may be either too specific or not specific enough to be usable (Russell, 2010, p. 143). The use of True/ False in DK assessments presents problems, e.g., if the person is a good guesser, they would at least have a 50% chance of obtaining the right answer. Multiple-choice, however, can yield reliable scores “…commonly in the .8 to .90 range…” (Russel, 2010, p. 148).

As to Procedural Knowledge and Skill (PKS), this “requires examinees to demonstrate what should be done and have the skill to do it…including, work samples or simulations, situational interviews, performance tests, and situational judgment tests (SJT)” (Russel, 2010, p. 150), “…work samples are high-fidelity measures simulations”(Russel, 2010, p. 150). Some trade
-offs with these two methods are that knowledge-based instruction “… tend to result in greater resistance to faking and [have] somewhat higher correlations with a general cognitive ability… ” (Russel, 2010, p. 152). The validity and reliability with SJT suggest a moderate (r = .34) ability to predict job performance…”(Russel, 2010, p. 160).

Circumstances Assessments are Appropriate

Assessments are appropriate when job performance is of primary consideration and when there are data points that are observable and measurable, although Russell (2010) suggests there are many other uses for tests, e.g., “… licensure and certification tests serve the purpose of protecting the public from incompetent or unsafe practice… commonly used for selection or promotion” (Russell, 2010, p. 141).

on content validity”(Russell, 2010, p. 162).

SJT assessments “… are highly versatile across situations. Work samples can be difficult and expensive to develop and costly to administer. This makes them best suited to situations where candidates must come to the job with a particular skill or must be able to learn that skill in a short period of time… are also often a method of choice if the selection system must rely  on content validity”(Russell, 2010, p. 162).

Circumstances Assessments are Inappropriate

There are challenges in DK testing, “…enhancing authenticity and minimizing obsolescence… a failure to reflect a real-world application of knowledge and skill. The argument is that simple declarative knowledge items may not adequately reflect the complexity of the work environment… items can be made more authentic by writing items that require the examinee to apply knowledge” (Russell, 2010, p. 148).

“Even the most valid and reliable test may not be useful in a particular setting. Operational concerns such as test development, administration, cost, the potential for coaching, and the need to develop parallel forms are important to consider in selecting a measurement method” (Russell, 2010, p. 158).

Complex simulations can have very few data points or opportunities for scoring and subsequently “…very few data points on which to base estimation of reliability”

Conclusion

If workplace assessments are to help ensure effective job performance, i.e., be reliable and valid—then all the stakeholders should be using the same “playbook”. According to Passmore, Santos, Malvezzi, & Kraiger (2014), “Workplace learning” is a term open to wide-ranging interpretation… there is neither a singular definition nor a unified approach to what workplace learning is, what it should comprise, or for whom it is or should be intended. It is all the more imperative that assessments follow a disciplined framework and process from job analysis to work-oriented and personality-oriented methodologies that consider a process that leads to both moderate to high reliability and validity, refer workplace image near the introduction.

=============================================================

References

Passmore, J., dos Santos, N. R., Malvezzi, S., & Kraiger, K. (2014). The Wiley Blackwell Handbook of the psychology of training, development, and performance improvement. Retrieved from O’Reilly Safari Books Online

Russell, T. L. (2010). Knowledge and skill. In J. C. Scott & D. H. Reynolds (Eds.), Handbook of workplace assessment: Evidence-based practices for selecting and developing organizational talent (pp. 141-164). Retrieved from https://redshelf.com

Please follow and like us:

No Comments Yet

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Making your blog better! $5 Dollars please!

My 3D WP Tag Cloud

Kuala Lumpur
29°
broken clouds
humidity: 70%
H 32 • L 29
26°
Tue
30°
Wed
32°
Thu
32°
Fri
Weather from OpenWeatherMap