Discussion: View Thread

Aptitude Increase with Training

  • 1.  Aptitude Increase with Training

    Posted 08-21-2024 15:15

    Good afternoon,

    I was recently asked by a colleague about guidelines for increasing aptitudes through training. They were informed that all aptitudes could be increased by one level with appropriate training and that aptitudes V, N, and S cannot increase above the G aptitude level. I am exploring this topic further and would appreciate your insights and any resources you might recommend.

    My understanding is that aptitudes represent innate abilities and are generally stable over time, with minimal impact from training. However, I recognize that training and practice can improve performance, especially in specific areas like math or language. I suspect that such improvements may be more noticeable at lower levels of aptitude (e.g., from low to low average), while the impact might be negligible at higher levels (e.g., from high average to high).

    What are your thoughts on this? Have you encountered any research or practical experiences that address this question?

    Thank you.



    ------------------------------
    Francois Paradis
    Certified Vocational Evaluator
    francois@career-options.ca
    Toronto, ON Canada
    ------------------------------


  • 2.  RE: Aptitude Increase with Training

    Posted 08-21-2024 15:22
    Hello,

    I would speak with Doctor Lee at EDITS publishing, San Diego, CA  as she has much research on abilities testing and the EDITS test she created.  





  • 3.  RE: Aptitude Increase with Training

    Posted 08-21-2024 18:00

    Great idea!  Dr. Lee is always so responsive and willing to answer questions.  You can reach her at lisalee@edits.net.

     

    Edits is an exhibitor at the IARP Fall Forum and IALCP Symposium. For those attending be sure to stop by, say hello, and ask questions. 

     

    Best regards,

     

    Michelle McBroom Weiss, MA, CRC, CCM, NCC, MSCC, ABVE/D, IPEC

    5543 Edmondson Pike, Suite 128

    Nashville, TN 37211

    mcbroomweiss@mcbroomweiss.com

    (P) 615-834-0186

    (F) 615-831-5274

    (C) 615-308-6395

     

     

     

     






  • 4.  RE: Aptitude Increase with Training

    Posted 08-21-2024 19:22
    Thank you. I have contacted Dr. Lee.

    Sincerely,

    François Paradis






  • 5.  RE: Aptitude Increase with Training

    Posted 08-22-2024 09:24

    Francois ... here is a great definition from the 1991 Revised Handbook for Analyzing Jobs defining many nuances of what is our traditional understanding of the concept of Aptitudes. https://skilltran.com/rhaj/rhaj9.pdf#page=4

    I recall a statement from the late, great Gale Gibson - Founder of Vertek, Inc. and its various software products which are now a part of SkillTRAN. His simple phrase was "Skills are aptitudes realized." I love that simple statement because it goes to the heart of it ... it is really difficult to master any kind of skill, however acquired, if you don't have the "knack" for it ... the potential (aptitude) to learn it. 

    We are all created equal before a far greater authority. But we each have been gifted differently.



    ------------------------------
    Jeff Truthan, MS-Rehabilitation Counseling, CVE
    President - SkillTRAN LLC - https://skilltran.com
    Spokane Valley, WA 99206
    (800) 827-2182 (Voice & Fax)
    ------------------------------



  • 6.  RE: Aptitude Increase with Training

    Posted 08-22-2024 09:34
    We really see aptitudes displayed in the Valpar Work Samples test.  That may be a resource you could cite. The "attention to detail" sample (I don't have the number memorized) is difficult for those individuals who do NOT have an aptitude for clerical work.  Perhaps practicing would improve their score, but I would argue that if they have worked with their hands their entire lives because that is where their aptitudes lie, making an adjustment to Sedentary, clerical work would be more difficult for them than for someone who did possess an aptitude for paying attention to detail.
    Best,
    Michele
    --
    Michele Erbacher, MS, CRC, ABVE/F
    Erbacher Rehabilitation & Consulting
    Cell:  (716) 807-6708






  • 7.  RE: Aptitude Increase with Training

    Posted 08-22-2024 09:35
    Jeffrey,

    I love Gale’s short, but sweet, and oh so true, explanation for aptitudes. As OG CVEs, you & I have probably given hundreds of thousands of aptitude tests (gotta love that GATB) and know that some people have a knack for verbal, some numerical, etc. Explaining that to the client was always rewarding as we narrowed vocational options. Thanks for your input!!


    Renee B. Jubrey, MS
    Certified Vocational Evaluator
    Diplomate/American Board of Vocational Experts
    RBJ Vocational Experts LLC
    Tele (860) 707-2220
    Fax (860) 651-7750




  • 8.  RE: Aptitude Increase with Training

    Posted 08-22-2024 11:46

    Thank you. Your comments go along my thinking that training would likely have little impact on aptitude test scores, except perhaps at lower aptitude levels, as previously mentioned. I hope to get some feedback from EDITS.



    ------------------------------
    Francois Paradis
    Certified Vocational Evaluator
    francois@career-options.ca
    Toronto, ON Canada
    ------------------------------



  • 9.  RE: Aptitude Increase with Training

    Posted 08-23-2024 01:59

    Bad decision to abandon GATB. But in reference to the question: If G is essentially similar to IQ (and IQ is essentially, as some say, little more than the ability to do well in school) and G is made up of V, N and S,  and all of these are very obviously developed and practiced and burnished in school, work OJT (training), I do not think it is as simple as you make it out to be. 



    ------------------------------
    Scott T. Stipe, MA, CRC, CDMS, IPEC, D/ABVE
    Certified Rehabilitation Counselor
    Board Certified Vocational Expert
    Scott Stipe & Associates, Inc.
    DBA Career Directions Northwest
    4110 SE Hawthorne Blvd
    #188
    Portland, Oregon, 97214
    (503)807-2668
    email: sstipecrc@gmail.com
    ------------------------------



  • 10.  RE: Aptitude Increase with Training

    Posted 08-23-2024 11:00

    I have always thought the following:

    Aptitude as the potential to learn/perform
    GED-RML as a measure of academic training
    SVP as the demonstration of proficiency through learning and/or work experience.



    ------------------------------
    Jeff Truthan, MS-Rehabilitation Counseling, CVE
    President - SkillTRAN LLC - https://skilltran.com
    Spokane Valley, WA 99206
    (800) 827-2182 (Voice & Fax)
    ------------------------------



  • 11.  RE: Aptitude Increase with Training

    Posted 08-23-2024 11:36
    Agree, however, a hypothetical:

    Assume Joe had 9th grade education and was administered aptitude testing back in 2018. G, V, N scores were below average, level 4. 

    Then assume that in 2024 he completed significant GED preparation courses primarily in math, language, reading. 

    Assume Joe successfully completed GED in late 2023. 

    Assume Joe has been taking prerequisite math, English and other courses in 2024 in order to enter a community college engineering technology program and doing well. 

    Assume that Joe qualifies for a program which will require that he take aptitude testing in late 2024.

    Would you expect that Joe's G, V, and N scores in late 2024 will be the same as they were in 2018?







  • 12.  RE: Aptitude Increase with Training

    Posted 08-23-2024 12:24

    Assuming Joe is a younger individual, implied by 9th grade education and aptitude testing back in 2018. Failed completion of high school in the typical age range implies any of a wide variety of issues likely occurring during that time that may have suppressed performance on the aptitude testing as well as completion of HS in traditional timing.

    Aptitudes and interests generally stabilize by the mid-twenties. Subsequent completion of high school via GED and  interest and engagement in community college training are terrific accomplishments for Joe.

    Subsequent testing may well reveal an upward bump in G, V, N scores partly from additional education but likely due to better attention span and motivation at an older age. If achievement testing would be done, would certainly expect solid gains here as well.



    ------------------------------
    Jeff Truthan, MS-Rehabilitation Counseling, CVE
    President - SkillTRAN LLC - https://skilltran.com
    Spokane Valley, WA 99206
    (800) 827-2182 (Voice & Fax)
    ------------------------------



  • 13.  RE: Aptitude Increase with Training

    Posted 08-23-2024 13:14
    This goes along my way of thinking Jeff. If there are any aptitude gains to be made with training, they are more likely, in my opinion, to happen at lower aptitude levels, maybe up to the average level. I think aptitudes at lower levels are more sensitive to literacy/numeracy upgrading.

    As Scott pointed out, this may be a simplistic view and reality is likely more complex or nuanced but I don't think the principle is wrong. I don't have any back up from literature, so I just call it a hunch for the time being. Perhaps EDITS Will be able to provide more clarity on this question. I will post it here if I hear back from them.

    François






  • 14.  RE: Aptitude Increase with Training

    Posted 08-23-2024 14:39

    No implication Joe was that young. He had simply dropped out of school following 9th grade. Very common scenario in WC and other injury cases with blue collar workers. Let's say he was 45-ish at time he took first aptitude test. With all other events same and with his further education, what, would none of that much matter and with another go round of aptitude testing half a dozen years later would GVN merely mirror those before?



    ------------------------------
    Scott T. Stipe, MA, CRC, CDMS, IPEC, D/ABVE
    Certified Rehabilitation Counselor
    Board Certified Vocational Expert
    Scott Stipe & Associates, Inc.
    DBA Career Directions Northwest
    4110 SE Hawthorne Blvd
    #188
    Portland, Oregon, 97214
    (503)807-2668
    email: sstipecrc@gmail.com
    ------------------------------



  • 15.  RE: Aptitude Increase with Training

    Posted 08-23-2024 16:21

    Mid-life testing I would expect more stability in aptitudes G V N S P. Maybe slight decline in Q and motor skills K F M E. 



    ------------------------------
    Jeff Truthan, MS-Rehabilitation Counseling, CVE
    President - SkillTRAN LLC - https://skilltran.com
    Spokane Valley, WA 99206
    (800) 827-2182 (Voice & Fax)
    ------------------------------



  • 16.  RE: Aptitude Increase with Training

    Posted 08-23-2024 16:44

    Another way to look at this is we are likely to get an underestimate of a person's G, V or N aptitudes if said person has limited formal education, literacy or numeracy skills. Training such skills likely will result in an increase of related aptitude scores. This does not mean that person has increased those aptitudes but rather, we are getting a more accurate measurement of their true aptitude levels by removing said literacy/numeracy barriers. 

    I have come across a couple research papers discussing the impact of coaching on aptitude scores. The first paper is dated but still informative and concludes that, on average, a person receiving coaching can expect an increase in aptitude score by up to .73 standard deviation. https://files.eric.ed.gov/fulltext/ED235195.pdf

    The second paper is more recent and also support the positive impact of coaching on aptitude scores. https://www.researchgate.net/profile/Sheeja-Krishnakumar/publication/381672268_PRE-PLACEMENT_TRAINING_AND_IMPROVED_APTITUDE_SCORES/links/667abb33d21e220d89d29a08/PRE-PLACEMENT-TRAINING-AND-IMPROVED-APTITUDE-SCORES.pdf?origin=publication_detail&_tp=eyJjb250ZXh0Ijp7ImZpcnN0UGFnZSI6InB1YmxpY2F0aW9uRG93bmxvYWQiLCJwYWdlIjoicHVibGljYXRpb25Eb3dubG9hZCIsInByZXZpb3VzUGFnZSI6InB1YmxpY2F0aW9uIn19

    While coaching is different from GED training, I think they are complementary factors, so I would not be surprised to see a similar impact for GED training.



    ------------------------------
    Francois Paradis
    Certified Vocational Evaluator
    francois@career-options.ca
    Toronto, ON Canada
    ------------------------------



  • 17.  RE: Aptitude Increase with Training

    Posted 08-25-2024 12:34

    Assuming substantial additional training/education, I too would expect stability in aptitudes S P. and as you say, maybe slight decline in Q and motor skills K F M, but probably not GVN. Q is iffy. I would expect much higher GVN if academic deficits were addressed over time. V and N are markedly impacted by academics. If Joe hated school, never could read or do math too well and dropped out to work construction, his GVN quite likely would not have been so hot at 20 or 30, or 40. We often don't really know but sometimes see it in ASVAB scores. But if he gets additional training (in what V and N are all about) such training improves those aptitude scores



    ------------------------------
    Scott T. Stipe, MA, CRC, CDMS, IPEC, D/ABVE
    Certified Rehabilitation Counselor
    Board Certified Vocational Expert
    Scott Stipe & Associates, Inc.
    DBA Career Directions Northwest
    4110 SE Hawthorne Blvd
    #188
    Portland, Oregon, 97214
    (503)807-2668
    email: sstipecrc@gmail.com
    ------------------------------



  • 18.  RE: Aptitude Increase with Training

    Posted 08-30-2024 15:13

    I thought I would share a conversation I had this week with the kind folks at EDITS. The bottom line being that aptitudes themselves don't increase significantly with training. Rather, a person may underperform when weak literacy or numeracy skills get in the way of assessing their true ability levels. As always, feel free to comment!

    On August 21, 2024 at 12:51 PM, francois@career-options.ca (francois@career-options.ca) wrote:
    Good afternoon,
    I am hoping you can address this question regarding the impact of training on aptitude scores. I am a vocational evaluator based in Toronto and a colleague of mine recently asked a pertinent question as follow:
    Are there any guidelines for increasing aptitudes with training? My colleague reportedly received training suggesting that suggested that all aptitudes can be increased by only 1 level (with appropriate training), and that V, N, S cannot be increased above the G.
    I provide below my thoughts on this and was told there is a Dr. Lee at your office that might have more insight:
    This is a complex question without a straightforward answer. Aptitudes, as I understand them, represent innate abilities that are generally stable over time, with only minimal impact from training. However, training and practice can indeed improve performance, particularly in specific areas like math or language.
    Aptitude tests, such as the GATB, CAPS, CareerScope, or CFIT, typically focus on speed and accuracy rather than a broad knowledge base. For instance, Numerical aptitude tests often consist of arithmetic items and don't include higher-level math questions.
    I believe that training might lead to noticeable improvements in aptitude testing at lower levels (e.g., from low to low average or to an average level). However, the impact of training is likely negligible at higher levels (e.g., from high average to high).
    While I don't have specific research or literature to reference, I think it would be reasonable to consider raising a person's aptitude score if they successfully undergo related training, but I would caution against raising it beyond the average level.
    I should add that I am the person in charge of professional development with CAVEWAS, the Canadian association of vocational evaluators. Your input would be of help to multiple people. Thank you in advance for your help.
    Sincerely,
    Francois Paradis, M.A., CVE, CCVE, ICVE
    From: EdITS Customer Service <service@edits.netSent: Wednesday, August 28, 2024 4:35 PM To: francois@career-options.ca <francois@career-options.caCc: Lisa Lee <lisalee@edits.netSubject: Re: The impact of training on aptitude testing
     
    Francois,
    Thanks for reaching out! My name is Andrew Tricarico and I have discussed this with Lisa, my direct supervisor. I have a master's degree in industrial-organizational psychology (where I accumulated a solid amount of testing and measurement expertise), so we both can offer some insight. 
    My background is in Industrial Psychology, and we typically classify job-related attributes as "knowledge," "skills," "abilities," or "other characteristics." Below are some relevant definitions as I understand them: 
    • Knowledge is an attribute that was purely developed through learning (biology, physics, the English language, etc.).
    • A skill is an attribute that improves with practice (persuasion, judgment and decision making, time management, etc.).
    • An ability is a more innate quality that can't as easily be practiced (physical flexibility, deductive reasoning, reaction time, etc.).
    • Aptitude (in terms of psychology) is closest in definition to ability, but specifically concerns one's potential. 
    While I am not intimately familiar with the GATB, I was able to find some literature that discusses its origin and structure which may have direct bearing on the training that your colleague participated in. Scores on V and S certainly couldn't be higher than G, as V is measured by Test 4 and S is measured by Test 3. G is simply a composite score of Tests 3, 4, and 6. N is measured by Tests 2 and 6, so someone could potentially score very high on Test 2 and surpass G, but it is probably a rare occurrence (and could be impossible depending on how the scores are calculated). Perhaps some math was done in the test development process to ensure that neither V, N, nor S could not surpass the G composite score, but I do not have access to a manual. 
    I agree that the question of aptitude versus ability and the possibility of improving scores is a very complex question. From my perspective, I think your definition is closest to "ability." Though again, aptitude and ability are quite similar. I browsed a couple abstracts, and did find a meta-analysis discussing how education can increase general mental ability (Richie & Tucker, 2018). Kulik et al. (1984) specifically focused on participants taking practice versions of aptitude tests, finding that ability moderated the effect of practice tests on aptitude test performance. Specifically, those with high ability scores experienced higher score gains, opposite to what is proposed in your email. Of course, even if it is a meta-analysis, it's an older one and I have not read enough to offer a thorough critique. Additionally, a method of training that isn't general schooling or a practice test certainly could moderate the relationship differently… I would need to compile a true literature review to know for sure. Also yes, numerical aptitude assessments are typically focused on speed and accuracy regarding lower-level problems, because using higher-level math problems would start to lean the assessment towards a measure of high-level math knowledge rather than general math-related aptitude.
    I don't understand what was meant by "it would be reasonable to consider raising a person's aptitude score, (but) would caution raising it beyond the average level," though. To me, the wording implies modifying scores on an individual basis after an assessment has been taken, which should never happen (unless you're deleting their data entirely because you know they cheated, didn't finish the assessment, or something along those lines). The only way to determine if one's aptitude (as measured by a particular assessment) has been modified is to have them take the same assessment again. Practice effects can influence the validity of these assessments for sure (particularly if assessment-specific training is received), but editing results is a guaranteed threat to validity. The CAPS (and most other aptitude assessments), are designed to take a snapshot of a participant's aptitude at a specific point in time.
    A participants' score is just what the assessment gives you- it's your *interpretation* of that score that should change. If you know the participant has undergone training (and especially if that training is specific to the content of a particular test), you also know that their results *might* be a little less valid. On the other hand, especially if the previous administration of the assessment and subsequent training had been years prior, perhaps the training did manage to increase one's aptitude (or, simply, more experience was gained in general). 
    In short, if you're leveraging a validated psychometric instrument like the GATB or the CAPS, one shouldn't ever consider raising (or lowering) scores due to outside factors because they were not designed for post-hoc modifications. If someone who scored far below average later scored high above average on an aptitude assessment, I'd certainly look into it, but I would always be using the scores the assessment presented as the basis of my interpretation.
    Overall, test batteries such as the CAPS offer a fairly accurate measure, at this point in time, of a client's ability levels. Test scores can improve through additional training, but we have no way to tell just how much improvement can be achieved. 
    I hope I have addressed your question! Regardless, I am happy to discuss this further over the phone or through email.
    References:
    Kulik, J. A., Kulik, C.-L. C., & Bangert, R. L. (1984). Effects of Practice on Aptitude and Achievement Test Scores. American Educational Research Journal, 21(2), 435–447. https://doi.org/10.2307/1162453
    Ritchie, S. J., & Tucker-Drob, E. M. (2018). How Much Does Education Improve Intelligence? A Meta-Analysis. Psychological Science, 29(8), 1358-1369. https://doi-org.libproxy.sdsu.edu/10.1177/0956797618774253
    Sincerely,
    Andrew
    _________________________
    EdITS LLC
    Educational & Industrial Testing Service
    On August 30, 2024 at 9:53 AM, francois@career-options.ca (francois@career-options.ca) wrote:
    Good morning Andrew and Lisa and thank you Andrew for your detailed response. I am including one of my colleague, Laura, who initiated this discussion. Laura, please feel free to jump in on this conversation. I think it is an important one and I wish our forum members to benefit from it.
    Regarding my previous statement ("it would be reasonable to consider raising a person's aptitude score, (but) would caution raising it beyond the average level," ), it was not meant to say that we would raise a person's test score. Rather, we may recommend an occupation one level above a person's scores if we have reasonable expectations those scores are an under-representation of their true ability levels due to literacy/numeracy issues. This does not mean that person has increased those aptitudes but rather, we are getting a more accurate measurement of their true aptitude levels by removing said literacy/numeracy barriers. 
    Let's imagine for a moment a typical scenario to illustrate this: an immigrant with a bachelor's degree in engineering and previous work experience as a mechanical engineer comes to Canada and wishes to get certified as a mechanical engineering technician. He now has to complete 3 years of college studies in engineering technology to qualify for a license here. Let's say this man's English literacy skills are at about a grade 8 equivalent. Let's also assume we gave him an aptitude test and that he achieved a low average level of verbal aptitude and an average level of general learning ability. We know, from the occupational profile of a mechanical engineering technician, that a high average level of general learning ability and an average level of verbal aptitude are required. Can we say that this person is likely to fail in becoming an engineering technician in Canada? I think not. Rather, we can expect that his true aptitude levels are higher than demonstrated on testing, given his background. His scores are depressed by his lower literacy skills. In this scenario, we are not raising his aptitude scores but assume those are under-representations of his true ability levels. We may recommend a career requiring one aptitude level above his current scores, with the assumption that he successfully complete academic upgrading to bring up his literacy level to a point where he can succeed in an engineering technology program. 
    If we were to test this person again after completion of his training, it would be reasonable to expect that his G and V aptitude scores have increased to the required level for his occupation. So, to be clear, we did not raise his initial test scores but made an assumption that his true ability levels would be revealed once the literacy barrier is removed. 
    In summary, I would advise against recommending occupations above a person's aptitude scores, unless there are specific circumstances and remediations to support it. 
    Thank you for your time and insights!
    Sincerely,
    Francois Paradis, M.A., CVE, CCVE, ICVE
    Francois,
    I totally understand what you're getting at now! I was looking at it from the wrong angle- test bias/type II error obfuscating a true score is a massive issue.
    In short, Lisa and I 100% agree with you guys here. We recommend that our CAPS examinees draw a line to the stanine above and below their given score, creating a crude "confidence interval" that is probably more representative of their true ability. Of course, language difficulties (affecting the non-language aspects of the assessment) may contaminate an observed score more than "typical error" would.
    This all emphasizes the importance of making an effort to ensure that assessments are, at the very least, culture-fair. It also certainly helps to get assessments back-translated and available in various languages so these threats to validity can slowly be stomped out (but of course, doing so is quite expensive).
    Regards,
    Andrew Tricarico
    _________________________
    EdITS LLC
    Educational & Industrial Testing Service


    ------------------------------
    Francois Paradis
    Certified Vocational Evaluator
    francois@career-options.ca
    Toronto, ON Canada
    ------------------------------