StPaulRecruiter Since 2001
the smart solution for St. Paul jobs

Director of Examination Requirements and Psychometrics- hybrid

Company: American Registry of Radiologic Technologists
Location: Saint Paul
Posted on: August 7, 2022

Job Description:

Do you have analytic and leadership skills, an interest in health care, and the ability to help improve and simplify complex processes? If so, we'd like to talk to you.

The American Registry of Radiologic Technologists ( ARRT ) is about to embark on our second century of promoting high-quality patient care by credentialing and certifying individuals in medical imaging, interventional procedures, and radiation therapy. We're looking for a Director of Exam Requirements and Psychometrics to lead the team that creates, reviews, and updates our examinations. One major goal of the position is to help identify, institute, and maintain the integrity of state-of-the-art credentialing processes. This Director reports to the Senior Director of Credentialing Operations and works closely with our CEO and Board of Trustees.

Ideally, you'd hold a Ph.D. in psychometrics or I/O psychology and have leadership experience or have a master's degree in any field and experience in the credentialing field.

ARRT has headquarters just south of St. Paul, Minnesota. We offer market competitive pay and top benefits and emphasize work/life balance. Because of the role's leadership and security responsibilities, we need someone who can work in the office at least 50 percent of the time. And because this person will interact with our Board of Trustees and professional committees, we require COVID vaccination.

We've listed an overview of the departmental responsibilities below. We haven't differentiated the activities you'll perform and those you'll delegate. That's because this position is ultimately responsible for the performance of the team and for the quality of our assessment development process and products.


Master's degree plus experience in credentialing OR

Doctorate in Psychometrics, Quantitative Methods in Psychology, Educational Measurement, Industrial Psychology, or related field.


3-5 years' experience in Exam development and/or psychometrics is preferred.

3-5 years' Supervisory/Managerial experience is preferred.

Department's Key Duties and Responsibilities

General Managerial Responsibilities

As the head of your department, you'll be responsible for managing staff, assessing departmental staffing needs, hiring, training, performing employee performance reviews, and completing other general managerial duties.

You'll also be responsible for department budgeting and contributing to strategic planning for the department and organization.

Assessment Development (Includes examination and structured self-assessments)

Plan and conduct practice analyses projects

Develop/revise content specifications and competency/clinical experience requirements based upon practice analysis results

Administer program of item development (i.e., item writing, pilot testing, and calibration)

Plan and conduct standard setting projects

Responsible for all the steps in the Structured Self-Assessment (SSA) process

Assembly and Production

Assemble assessment forms and other materials for committee meetings

Schedule meeting dates, plan, and coordinate activities for committee meetings

Conduct meetings of committees

Produce electronic assessment forms to send to CBT vendor

Conduct quality control checks on electronic assessment forms assembled by CBT vendor

Maintain item banks

Maintain Assessment Development Procedures Manual

Scoring, Analysis, and Reporting

Review item analysis and assessment participant comments

Run programs and interpret data to process assessment results

Maintain equating mechanism

Maintain quality control checks on overall process

Prepare reports summarizing assessment results

Maintain sections of Assessment Processing Manual which cover procedures for which Psychometrics Services is responsible

Examination Security

Maintain the validity of the assessments by employing the latest procedures to identify: Those using others to complete the assessment for them, examinees with pre-knowledge of the assessment content, or copyright infringement

Develops policy recommendations and is responsible for the executing security policy for all content contained in assessments

Chair of the Assessments Security Team (AST) and recommends corrective action for security incidents based on AST investigations

Executes policies for procuring legal protection for ARRT intellectual property (copyright of item bank, trade secrets)

Maintain confidential and accurate records when engaged in a legal action

Effectively represent ARRT in depositions and/or when asked to testify in a court of law


Assure procedural integrity for the administration of ARRT examination requirements and continuing qualifications requirements

Work with Strategic Communications to develop informational materials (e.g., articles, news releases, slide shows, Certification Handbooks (Primary Pathways, Post-Primary Pathways, and R.R.A.), Handbook for State Licensing Examinees, Examinee Handbook for Limited Scope, Item Writing Manual) on assessment development, assembly, scoring analysis and reporting

Prepare project and technical reports on examination development, assembly, scoring, analysis and reporting

Respond to questions/complaints on examination development, assembly, scoring, analysis, and reporting

Identify issues requiring Board policy decisions on exam development, assembly, scoring, analysis and reporting

Provide staff support on psychometric issues at Board Meetings

Interact with Information Technology Department on development and maintenance of computer programs for psychometric services


Collect, analyze, and report data as input into decision making processes

Provide technical assistance in research design and data analysis to staff

Participate in scholarly research as time permits

Keep informed of issues and new developments in certification and licensure testing

Cross Departmental Responsibilities

Work with our Strategic Communications department to develop informational materials (e.g., articles, news releases, presentations, and handbooks)

Partner with our Information Technology department to develop and maintain computer programs for psychometric services

Review a full job description, which includes additional information on duties and responsibilities, skills and abilities, core competencies, and physical demands. This fulltime position is a hybrid- located onsite at our office in St. Paul, Minnesota with potential for partial telework (approximately 50%). Relocation assistance may be available.

The American Registry of Radiologic Technologists (ARRT) is the world's largest organization offering credentials in medical imaging, interventional procedures, and radiation therapy. We certify and register technologists in a range of disciplines by overseeing and administering education, ethics, and examination requirements. Visit for more information.

The Twin Cities (Minneapolis and St. Paul) is a vibrant and lively metropolitan area. It consistently appears on national lists of best places to live in the U.S., and we couldn't agree more. We're home to six professional sports teams, an impressive arts and theater scene, and numerous restaurants with James Beard-winning and -nominated chefs. We also have an abundance of lakes, trails, and outdoor activities, as well as dozens of higher education institutions within the metro area.

This position is considered public facing for ARRT and requires the individual to be fully vaccinated for COVID-19. Accommodation requests will be reviewed on an individual basis. Contact Human Resources for more information.


Keywords: American Registry of Radiologic Technologists, St. Paul , Director of Examination Requirements and Psychometrics- hybrid, Executive , Saint Paul, Minnesota

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category

Log In or Create An Account

Get the latest Minnesota jobs by following @recnetMN on Twitter!

St. Paul RSS job feeds