Three is the charm: Certification by Examination
By Rosalye A. Settles
Originally published in “ACA News,” December 1999.
I attained certification by examination in 1993 after two unsuccessful efforts. In my first two attempts, I scored just slightly below the pass point. After my second failure, I began to question myself and the whole test process. How could I have failed after having worked in the field for a number of years and in variety of settings? Who were the people who created the exam and what were thinking about, anyway? With two almost passing scores under my belt, I was to say the least, disappointed.
But, I remained undaunted. Why? Because I believed certification was important to my professional growth and development. Upon reflection I came to view the test process as a means of self-assessment. With an adjustment in perspective, I realized that I needed to expand and solidify my archival knowledge. Despite years of work experience and a masters degree in American History, there were aspects of my chosen field I needed to learn more about. So, I began to study and educate myself in specific archival subject domains.
Somehow I knew my efforts would lead to passing the certification exam the next time. My educational pursuits eventually lead to a second masters degree in library science from Columbia University in New York City, in preservation. Preservation had always been of interest to me and, yes, it was a subject area I needed to learn more about.
In 1993, my refusal to accept defeat manifested into benefit in the form of a postcard notifying me of my passing score. (One could say three is the charm). I was happy to have finally achieved certification, unaware that my goal to be certified would lead to other professional opportunities directly related to the Academy of Certified Archivists (ACA) certification examination.
From 1996-1998 I served on the ACA Examination Development Committee (EDC), the body tasked with the development of the certification examination. During that time, I was the only female Committee member who had been certified by examination. As I assumed my role as a EDC member, I shifted from being a former test candidate to test creation participant. I would observe the test development process and get the chance to see just how those questions I labored over found their way into the examination.
While working on the EDC I contributed to the development of a certification examination and the creation of the Handbook for Certification. I reviewed question statistics, made recommendations for the removal of questions from the item bank and created new items for two of the examination’s subject domains. My Committee members and I often engaged in spirited debate on the appropriateness of a particular question. How many people would know the correct answer? Would test takers really have access to the references we cited? Was the knowledge a question sought to test too obscure? Should a question be eliminated or salvaged by revision?
Working in tandem with our psychometrician (an expert in statistical testing), I observed how the EDC ensures that questions and the actual test itself are valid. Each year after the test is given, the test is evaluated. Each question is analyzed to see how well it performed i.e. did a large percentage of test takers fail to select the correct answer? If so, why? After the test is reviewed, the passing score is established based on statistical standard errors of measurement. This close scrutiny delays the ACA’s notification to test candidates, but it ensures both the validity of the questions and the examination itself.
The creation and analysis of a professional test instrument is a rigorous and thoughtful series of tasks. It is a continual process of evaluation that is no way capricious, arbitrary or frivolous. My EDC experience provided me with a new perspective for the examination that I had worked so diligently to pass. The ACA’s test of archival knowledge is done on a high and demanding level. Not achieving initial success upon taking the exam is not an indictment or statement of one’s capability; rather, it is an opportunity to continue the on-going learning process and strengthen one’s intellectual prowess relative to archives.
From 1998-2000 I will assume the role of Regent for the EDC. I bring a dual perspective to this new position, analytical archival knowledge and practical experience as a former test candidate. My fellow Committee members, along with our psychometrician, bring talent, experience and commitment to the process of examination development. As we confer, debate and finally concur, we will ensure that the ACA certification examination is both statistically valid and a true reflection of the breadth of knowledge possessed by certified archivists.
Obstacles to success are a means by which we develop ourselves. Often a shift in perspective is necessary for us to see just what we need to see; namely, that delay does not mean denial. Often we need to take a new approach, work from a different vantage point and work a little bit harder. Then if we make the necessary adjustments, we can see our goals realized before our very eyes, sometimes in the form of something as small as a post-card in our mailbox.