I came up in private rehab, WC. I and everyone else I knew/know used GATB and later the AP since choreography back to DOT was seamless, exquisite. On occasion if we saw some unusually low, unexpected whatever we might use more than one aptitude test but that was rare. Over the decades in forensics, I also generally only see one aptitude test administered but unfortunately see many (to perhaps most VEs) do no testing at all (even when they have opportunity to, probably due to lack of competence or worry over results throwing a wrench into the gears, flummoxing favorite RTW options). In my state in WC we had to do a training plan documenting that Joe or Sally met the OAP cutting scores or came dang close with SEM for the proposed occupational goal or it did not fly, period, start over. It was the same in OWCP which I did for years. There was no untimed anything in terms of aptitude testing. Giving the same test "with repetition" or untimed..............wouldn't almost anyone with practice effect or more time to hem/haw/review improve one's score? As far as norm groups, wasn't GATB normed on general population. Does not that general population have folks with myriad physical/mental this or that going on?
Maybe this will help:
What I am seeing is something akin to this, the facts are changed to protect the (not so) innocent. Let's say Joe was bus mechanic for years. Let's say he dropped out of high school 20 years ago and later got a GED. He did lousy in math, science, and English before dropping out to go to work. He did well in shop and PE. Aptitude testing shows N and V level 4, low, reading is not bad, almost grade 12. VE says if was given aptitude tests "untimed" he may well do better (duh, who wouldn't?). And IF (really, really big if) so could go to college and become a computer programmer or mechanical engineer. I guess anything is possible, but who cares, I thought experts deal in probabilities and have foundations. So, I am wanting to know if there are actually aptitude tests designed to be taken untimed and what the validity/reliability of such instruments is compared to all aptitude test I know of which are timed, like work and school and life is timed.
------------------------------
Scott T. Stipe, MA, CRC, CDMS, IPEC, D/ABVE
Certified Rehabilitation Counselor
Board Certified Vocational Expert
Scott Stipe & Associates, Inc.
DBA Career Directions Northwest
4110 SE Hawthorne Blvd
#188
Portland, Oregon, 97214
(503)234-4484
(503)234-4126 fax
email:
sstipe@careerdirectionsnw.comwebsite:
www.careerdirectionsnw.com------------------------------
Original Message:
Sent: 08-30-2022 18:36
From: Renee Jubrey
Subject: "Untimed" Aptitude Tests
Phil,
I too would draw a line under the last completed test item when the allotted time was up and then allow an evaluee to complete on an untimed basis. This was done while working at the school for the Deaf where I would evaluate 3-4 individuals every other week for 16 years. Some were students (up to age 21 years) and others were adults whose jobs had been downsized or who came from another country. We did use the Valpar (green monster) norms specific to the Deaf and other norms specific to the test at hand. Our reasoning was to determine if the individual could actually accurately score on the test without the pressure of timing. We found that generally individuals could pick up speed with repetition over time and this gave us a better idea of which training program or job they may be most successful in.
I too used the GATB but never gave one test to determine an aptitude (or as Jeff so nicely put it "a knack"). For each of the aptitudes tested we used a minimum of two test products and mostly three. Triangulating the results as best as possible and found many of our placements (either outright into competitive employment or into a short training program) were quite successful.
I have used the Valpar 300 Series as well and find the criterion reference/MTM to be most useful as I agree it can be difficult to match an evaluee with a norm group. Anyway, as I prepare to retire I am looking to liquidate my professional testing supplies. If anyone is interested in the Valpar Series 300 email me privately.
------------------------------
Renee Jubrey
Certified Vocational Evaluator
CertifiedVE@gmail.com
East Granby, CT United States
Original Message:
Sent: 08-30-2022 18:14
From: Phillip Boswell
Subject: "Untimed" Aptitude Tests
Methinks that we're coming at this from different angles and that's O.K.
Tests are aids to clinical judgement and aptitudes represent a small component of work. Aptitudes as described in the RHAJ represent about 15% of the 72 work characteristics identified in it (or 12.5% in Canada since we don't use colour discrimination or eye-hand-foot coordination in our NOC codes).
When a standardized test is administered and the resulting score interpreted with reference to the norms provided for the test, a number of assumptions are made. One of these is that the test in question has been administered under the same conditions as obtained when the test was administered in the standardization program – the same directions, the same time limits, etc. It is also assumed, of course, that the tests have been scored accurately and that interpretative scores have been read correctly from the table of norms.
Therefore, it follows that tests that are not normed against people with disabilities shouldn't be used with them because we're not comparing apples to apples (likes to likes). Be we do. This supports the use of criterion referenced assessments where a specific content domain is the frame of reference for interpretation. The focus is on what the person knows and can do, not how this person compares with other people. It is designed around a specific task or series of tasks and the steps (criteria) are pre-determined and the person's performance is compared against the criteria.
To your point Ron, I interpret what you're talking about as the concept of general productivity (i.e., the individual's ability to perform work tasks with sufficient speed, quality, and consistency to allow the employer to profit from his/her employment). This then naturally leads to a discussion around the concept of competitive versus sympathetic employment.
Interesting stuff...
Phil
------------------------------
Phillip Boswell
Vocational Evaluator
phil@wcve.ca
Comox, BC Canada
Original Message:
Sent: 08-30-2022 17:49
From: Ronald Smolarski
Subject: "Untimed" Aptitude Tests
Time is very important to a business man/ woman. One can either flip 10 pizzas in an hour or not. In this pizza example it may be the point of making a profit or not...our capitalistic method of making money means the person must work at a competitive and sustained rate and have the competencies to complete the tasks to the job. Time is one element of making money, therefore the timed test means something no different than in baseball...can the runner on first make it to second before the catcher can throw the ball to second base – that task involves a time element.
Ronald T. Smolarski, M.A.
Certified Life Care Planner
Forensic Economist
Certified Functional Capacity Evaluator
Vocational Expert
ron@beaconrehab.com
www.beaconrehab.com
(800) 821-8463
Ann Arbor Michigan, USA
DISCLAIMER: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. Additionally, this communication and/or attached files may contain protected health information, which is governed by HIPAA regulations. If this information is intended to be forwarded or shared, you and your entity are responsible to assure HIPAA regulation and guidelines are followed. If you are not the intended recipient any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by e-mail and destroy all copies of the original message.
Original Message:
Sent: 8/30/2022 2:47:00 PM
From: Phillip W. Boswell
Subject: RE: "Untimed" Aptitude Tests
Scott:
Untimed aptitude tests are indeed a "thing". Valpar Pro 3000 allows the evaluator to choose to either time or untime sub-tests. Personally, I untime tests and chose this particular aptitude test because of this feature (the program is no longer available). Consider that one of the most common accommodations we offer students with disabilities in academic settings is extra time for tests. Why would this be any different in the work-place?
While it is important to acknowledge that in some instances it is helpful to understand how an individual is functioning compared to the "average" population when assessing for competitive employment. However, most norm referenced tests do not use representative norm samples that include individuals with disabilities. This often results in labels of disability and the identification of weakness.
When I first started testing aptitudes, I was certified to use the GATB. I'll just say that I wasn't a big fan of the timed format. On more than one occasion, I felt that the GATB test results unrepresented the evaluees abilities, especially when triangulated with other tests. I would make my thoughts known in my report and characterised them as 'clinical judgement'.
I see this as perfectly logical from an evaluation perspective. I'm sure there are others out there in cyber-space that can provide more sciencey, albeit dated, data regarding the reliability and validity of this software.
Hopefully this is helpful. I'd be interested to hear what others have to say.
Regards,
Phil
------------------------------
Phillip Boswell
Vocational Evaluator
phil@wcve.ca
Comox, BC Canada
Original Message:
Sent: 08-30-2022 13:45
From: Scott Stipe
Subject: "Untimed" Aptitude Tests
I am seeing some VEs indicate that while an individual did very poorly on certain subtests of an aptitude test, that if he/she had taken such testing "untimed" that they would possibly have done better and thereby would have aptitude to perform or learn to perform various skilled occupations. Such strikes me as illogical and without any sort of foundation. It seems to me that work involves timed tasks. School projects and testing involves timed tasks. Does anyone have anything about "untimed" aptitude testing, reliability and validity, etc.? Is such a standard method?
Thanks all
------------------------------
Scott T. Stipe, MA, CRC, CDMS, IPEC, D/ABVE
Certified Rehabilitation Counselor
Board Certified Vocational Expert
Scott Stipe & Associates, Inc.
DBA Career Directions Northwest
4110 SE Hawthorne Blvd
#188
Portland, Oregon, 97214
(503)234-4484
(503)234-4126 fax
email: sstipe@careerdirectionsnw.com
website: www.careerdirectionsnw.com
------------------------------