The real-life on-the-job performance outcomes or low predictive validity of the S.T.A.R (Situation, Task, Action, and Result) method used in the Behavioral Interviews have led to major concern about the effectiveness of the S.T.A.R method which is prevalent in various organizations. For Human Resource (HR) professionals, particularly for the Talent Acquisition (TA) team members and the hiring managers, this is the opportune time to re-evaluate the efficacy of the S.T.A.R approach, considering the assessment process’s end objective. Needless to say, the end objective of any assessment method in hiring is to get the candidates who will be successful on the job. Figure 1- Limitations of S.T.A.R method in behavioural interviews.
Figure 22 here, delineates various assessment methods and their respective predictive validity. This is based on the fundamental research done by Spencer and Spencer who are pioneers and world’s top authority in Assessment methods. In fact, the word “Competency” was coined by them when they used to work with David McClelland (Harvard University Psychologist) in the Hay Group. Based on my research (post-doctoral work at MIT Sloan School Management on Assessment methods) and practice on various Assessment methods used during the pre-hiring and the post-hiring phases for decision-making and developmental purposes in organisations globally over the last 25 years, I have readjusted these validity coefficients. In this write-up, you will find the predictive validity (correlation with future on-the-job performance outcome) of Behavioral based Interview which applies S.T.A.R method is found to be low (.32) and lots of organisations nowadays are using this method of assessment during the hiring process. If selection or rejection or elimination decisions are primarily made based on this method, it could be highly erroneous. There is a high possibility of selecting the wrong candidates (α, Alpha error) or rejecting the right candidates (β, Beta error). Figure 2 – Predictive Validity is the correlation coefficient between the results of assessment tools used and on-the-job performance ratings/outcomes.BEI (Behavioral Event Interview) is significantly different from Behavioural-based Interview which is commonly used through the application of S.T.A.R (Situation, Tasks, Action and Results) approach. The cognitive component of BEI in terms of an Individual’s “Feelings, Wants and Thinking” differentiates it from the S.T.A.R method based Behavioural Interviews. The Behavioural Based Interview process is the derivative of the Critical Incident Technique (CIT) process which was developed by Flanagan in the early ’50s.
Various assessment methods
The following are the brief descriptions of various assessment methods mentioned in the above graph (Figure 2) however, the main focus of this write-up is to highlight the limitations and misdirection of S.T.A.R method.
Assessment center/development center (AC/DC)
“An Assessment Centre is a comprehensive & standardised procedure to assess multiple managerial & leadership competencies with the application of multiple simulation-based assessment techniques by the multiple trained assessors to have high predictability for managerial success for potential roles. This method can be used for selection, promotion, job rotation, career & succession planning and training & development needs identification of the incumbents in an organisation” (Priyadarshi 1999).
Simulation/work-sample tests (Inbox/In-basket exercise)
An in-basket or Inbox exercise is used as a simulated assessment tool to establish how well the candidates/applicants perform job-related tasks within a certain specified period. This particular type of simulation exercise ascertains how well various strategic and/or operational tasks are prioritized in a logical sequence and how to move to successfully manage them.
Behavioural event interview (BEI):
The basic purpose of BEI is to get the interviewee/candidate to describe in details four to six complete stories of critical incidents the candidate has experienced in the job- two or three “high points” or major successes, and two or three “low points” or key failures. To get the complete story, the answers for the following specific questions are necessary (Spencer and Spencer, 1993):
What was the situation? What events led up to it?
Who was involved?
What did you (the candidate) think, feel, or want to do in the situation?
What did you (the candidate) actually say? Here, the interviewer should be interested to know the skills the candidate showed.
What was the outcome? What happened?
The above BEI questions, the number 3 and number 4 make BEI unique as they focus on the cognitive part of candidates in terms of their “thinking, feeling and motivation”. These cognitive components of BEI differentiate it significantly from the S.T.A.R method based Behavioral Interviews.
Cognitive ability test (reasoning tests)
Cognitive ability tests assess abilities involved in thinking (e.g., reasoning, perception, verbal and mathematical ability, critical thinking and problem-solving ability). Such tests are designed to assess candidates’ abilities to use mental processes to solve work-related problems or to acquire new job knowledge. These tests aim to measure cognitive skills rather than acquired knowledge. By evaluating innate abilities, such as numerical and verbal aptitude, these tests help predict an individual’s potential success in tasks requiring intellectual prowess.
Personality tests
Personality tests are systematic assessments that aim to measure various aspects of a candidate’s personality, such as self-concept, internal locus of control, values, temperament, introversion and extroversion etc. These types of assessments have been around for decades. Employers often use personality tests during the hiring process; however, it is strongly recommended to map personality factors with the competencies required for the roles, otherwise, Personality tests have limitations in the right hiring decisions. The personality tests could be highly beneficial for employees’ developmental purposes.
Behavioral based interview (based on S.T.A.R approach)
The initial section of this write-up provides a reference to the S.T.A.R method and its challenges. One of the major challenges associated with Behavioral Interviews is the validity of the Situations (S) described by the candidates. There is a good possibility of the concoction of examples of past Situations by the candidates and it has been very difficult to verify those examples as the data or examples provided by the candidates can not be corroborated by other facts or sources. For example, during the S.T.A.R-based Behavioral interview, the candidate narrated a very convincing example of Team Leadership in a job situation in spite of always being the Individual Contributor in the past job. He mentioned the Tasks, Actions and Results related to guiding the team members. The example initially sounded very impressive, however, during the second level interview when probed deeper, the candidate accepted that he never managed a team before and the example provided to him was not the true example. The important point to reiterate here is on the basis of the S.T.A.R method only, he moved to the second level of the interview.
The other issue related to the S.T.A.R method is its limitation in assessing the subject or deep domain knowledge of specific functions. These Behavioral interview questions, in general, are equally applicable for any position irrespective of the nature of any function such as Legal, Marketing, Sales, Finance, HR, Technology etc. In other words, the knowledge of functional areas is not the key focus which is critical for success on the job. Moreover, it has been found that some of the interviewers use or misuse S.T.A.R method as an easy passage to conduct interviews without even reviewing the resumes of the candidates and making decision in terms of selection or rejection or elimination of the candidates.
In a nutshell, the S.T.A.R method lacks scientific rigour of assessment and that’s why its predictive validity is found to be only .32 as mentioned in Figure 2.
Biodata/resume
Use of Biodata/Resume in the selection process is self-explanatory.
References
The use of references in the selection process is a common practice in many organizations. However, it’s often observed that even average candidates receive positive references, as these references are typically chosen by the candidates themselves based on personal relationships.
Unstructured interviews
Unstructured interviews do not use any standardized approach or set questions, instead, the interviewer asks open-ended questions based on his/her areas of interest. The interviewers modify their questions to suit their own expectations in a non-standardized manner. The validity and reliability of the unstructured interview are found to be the lowest.
Notes
Himank Priyadarshi, 1999, Head Individual & Organisational Assessment Centre (IOAC), Academy of HRD published journal.
Spencer, Lyle and Spencer, Signe (1993):” Competence at Work: Models for Superior Performance”, John Wiley & Sons