Lecturer, Malaysian Maritime Academy
A.A., B.S., MBA., ACB
“You can’t train your way to an outstanding business with bad fundamentals and habits, you have to be competent, have sound fundamentals and the right attitude”.1
This article is a continuation of parts I and II from the Reimagining Talent and Development series and proposes that the proper collection, analysis and evaluation of data are paramount in order for Talent Development Human Performance Improvement (HPI) solutions to be effective.
Providing talent and development solutions through data mining
For several articles written by the author in the Globalmet Newsletter, the last two specifically, the collection, measurement, analysis, evaluation and use of data to aid in providing Talent Development solutions has been either referred to or at least eluded to for several important reasons; but, how to use that data in any great detail has never really been expounded upon until now!
This article then, intends to inform in as much detail as this short venue permits and show how Talent Development (TD) Professionals (TDP) use data in the course of providing HPI solutions and business results to the organization.
Establish good data collection habits for talent and development
To begin with, the TD professional must establish the purpose of data collection, method, analysis and report format when it comes to solving TD issues and presenting them to stakeholders. Figure 1 below gives some examples of the nature of the data to be collected; Bassi and McMurrer refer to these as “disparate pieces of data” and should be part of the organization’s core data knowledge management system from which drives success and data requirements. However, to analyze and evaluate this data requires putting forth sound research design and methods (often referred to as Quantitative Research Methods) and the kind of rigorous design that ends up providing unbiased effective solutions based on the business drivers and needs of the organization—not emotion and office politics. The research process itself entails rigorous data collection, analysis, description, inference and evaluation in order to yield fruitful business and talent solutions as required by stakeholders. This process must be void of bias or risk the results becoming unreliable; the data must be filtered, grouped, categorized, described and inferences made as this is of paramount importance to the TDP and subsequent results.
Knowledge management data usage for talent and development
In an article written by Galagan, the author suggests that it isn’t easy to achieve a big impact using data– but that “…most companies fear doing it or restrict it…[and that] [m]any companies profess to be on board but are frustrated at the lack of results or change in practices”.
A crucially important key is that the “population” (data source) from which the sample of data is to be drawn and its methodology (how it is to be collected) must be done using the science of quantitative research methods if they are to be taken seriously; which entails the important concept of quantitative sampling methods (e.g., random, stratified, etc.)
Start by validating the business drivers for
talent and development
“The most important step in establishing measures for an evaluation strategy is validating the drivers” (Biech, 359). These include both the organization’s internal and external business drivers and forces eluded to in parts I and II (newsletters 44 and 45) and helps an organization to achieve, surpass and even sustain their strategic outlook, business goals and performance needs.
Seven steps of data collection
Discussing data and making it interesting is certainly a large topic taken on in such a small venue as a newsletter, none-the-less it must be emphasized as a mandatory, potentially daunting, but necessary part of the TD professional’s competency and responsibility. In simple terms, a seven step data collection plan is elaborated upon here:
7 step data collection plan for the Talent and Development professional
- Collect the right type of data so it can be analyzed correctly;
- Understand the intent and usage of the data collection so one can collect the data from the right population.
- Know the correct sampling technique so as not to bias the data.
- Understand the mechanics of research design, e.g., inferential (i.e., to infer or deduce with some degree of significance about what the future might entail).
- Understand how to derive the correct research question; e.g., disprove the null or false statement and to “what degree of confidence” (i.e., 99%, 95% or 90% confidence) and the consequences thereof– for example, the research question (statement) that all dogs have one leg; if we find one dog with four legs (null is true), then our original question is indeed false.
- Use proper analytical methods to derive the outcomes
- Brief the proof and profess a solution to a high degree of “statistical significance”.
Data collection models to consider before collecting the data
Once the business drivers have been identified, a model selecting the types of data to be collected can be selected. One such model is Donald Kirkpatrick’s Four Levels of Evaluation and refers; (1) to a learner’s reaction to the training (happy, glad or otherwise); (2) measurement of the learner’s behavior in the classroom and training environment (i.e., how they did on the assessment); (3) the individual’s behavior change on the job; and lastly, (4) the business results (increased sales, production, quality and such); Referred to as Level 1, 2,3 or 4 as not all of the levels must be used at any one time.
Don’t forget the purpose of the data collection
In the process of formulating a plan and collecting the data, it’s important not to forget the specific purpose of doing so and the necessity for accurate information and results. And according the Biech, this purpose may be to:
- Determine the current level of training and in particular skills
- Identify optimal performance levels and gaps compared with current levels
- Conduct needs and training requirements analysis
- Determine whether a course provided the required learning
There are many methods and models with which to help formulate more than a good research question that addresses the right requirements and subsequent solutions; a few were just discussed, e.g., Kirkpatricks Four Levels, obviously something executives and managers care a lot about. There are also other methods to assist in ROI evaluation like Philips ROI Methodology, Balanced Scorecard and a few others. As long as the TD professional “drives the process” methodically as discussed here and not become too overwhelmed by the process (and therefore bypass it) and not let the process drive the TD professional —the end result yields better business and financial results.
Why is data really important for executive stakeholders?
What data does or can do, is show correlations between one or two variables, suggest answers such as is there any significance between pre and post training or HPI indicated from the data; but, most importantly, how the TD professional might use the information to help improve business performance (HPI); e.g., predicting turnover, business impact of leadership development programs, effectiveness of one’s efforts to improve performance and many other areas as well (Biech, 391).
The TD professional must interpret, present and report the results of the HPI solution in a meaningful way that meets and exceeds stakeholders’ needs and requirements. Often times, data may not be presented taking into account the specific expectations, needs and requirements of executive stakeholders; an error here in process, communication and reporting may be tantamount to professional irresponsibility and not in keeping with the highest standards of the TD professional.
Conclusion on talent and development
In conclusion, TD professionals must provide quality, timely reporting and solutions to stakeholders that answer the question! The use of the right data is a key element in that process. And according to Biech, the TD professional should not necessarily expect praise, especially in an environment fraught with office politics, “rice bowls” or otherwise; however, testimonials from managers and senior executive can help save a mediocre presentation and one’s reputation.
- C= (K+S+E)*(A), where C equals competency; K is knowledge; S is skill or practical; E is experience and A is attitude. Note what happens when A is very low, zero or even negative, no matter what knowledge, skill or experience one has, one is still considered to be not yet competent.
Biech, Elaine. (2014). ASTD Handbook: The Definitive Reference for Training &Development (2nd Edition). Alexandria, VA: ASTD Press.
Bassi, L., & McMurrer, D. (2015, May 27). 7 Steps to Using Analytics to Improve the Evaluation of Learning. Retrieved May 30, 2015, from https://www.td.org/Publications/Blogs/Learning-Executive-Blog/2015/05/7-Steps-to-Using-Analytics-to-Improve-the-Evaluation-of-Learning
Galagan, P. (2015, April 17). Four Ways to Get Big Data Working for You-Instead of Frightening the Frontline. Retrieved May 30, 2015, from https://www.td.org/Publications/Blogs/Learning-Executive-Blog/2015/04/Four-Ways-to-Get-Big-Data-Working-for-You-Instead-of-Frightening-the-Frontline