Manager, Testing Operations & Evaluation (Johns Hopkins Center for Talented Youth)

We are seeking aManager, Testing Operations & Evaluationwho will support the administration, evaluation, and improvement of CTY's assessment programs. Reporting to the Director of Research and Assessment, this role oversees testing logistics, trains and supports proctors, manages secure test materials, and conducts applied data analyses that inform program effectiveness. The manager will also collaborate with senior leadership and researchers on studies of reliability, validity, and student outcomes.

This is an excellent opportunity for someone with a background in psychometrics, quantitative psychology, or educational measurement who enjoys combining hands-on operations with meaningful data analysis and research collaborations.

Specific Duties & Responsibilities

Testing Operations

• Manage logistics for the administration of CTY-owned tests, including secure storage, inventory, and distribution of materials.
• Train and support internal and external test administrators (proctors) to ensure compliance with CTY policies and the Standards for Educational and Psychological Testing.
• Update and maintain CTY's testing manuals, proctor training materials, and operational procedures.
• Serve as vendor point of contact for CTY testing solutions
• Use software tools (e.g., Remark, SPSS) to score assessments, process results, and generate reports.
• Beta test new computer-based forms and assists with troubleshooting.

Evaluation & Data Analysis

• Collect and manage evaluation data on student progress and program effectiveness.
• Conduct applied analyses (e.g., descriptive statistics, regression, basic psychometric checks) to inform program decision-making.
• Develop evaluation instruments and surveys in collaboration with other relevant departments across the organization.
• Maintain records, databases, and documentation related to testing and evaluation.
• Prepare concise reports and data visualizations for CTY leadership and stakeholders.

Collaboration & Client Support

• Serve as a resource for families, schools, and contracted partners regarding testing logistics, score interpretation, and best practices.
• Support communications with partners on license renewals and testing agreements.
• Collaborate with senior leadership on validation studies, norming projects for current SCAT, and the development of new assessments.
• Manage external testing projects of varying scale, including those with international organizations, and oversee contract staff as needed.

Other Duties

• Assist in cross-functional projects and research initiatives as assigned.
• Participate in ongoing training to stay current with testing best practices.
• Stay current on CTY's strategic priorities and orientation in the field.
• Participate in academic conferences as budget allows.
• Other duties as assigned.

• Master's Degree in Psychology, Educational Measurement, Psychometrics, Quantitative Psychology, or related field.
• Three years of experience (inclusive of doctoral-level training) in testing operations, assessment administration, or applied educational measurement. Experience supporting norming studies, reliability/validity research, or item bank development.
• Additional education may substitute for required experience and additional related experience may substitute for required education beyond a high school diploma/graduation equivalent, to the extent permitted by the JHU equivalency formula.

• Communication - Strong written and verbal skills, with ability to translate technical results into accessible reports for non-technical stakeholders.
• Organization - Strong project management skills; ability to manage multiple priorities and meet deadlines.
• Values - Commitment to diversity, equity, inclusion, and anti-racism in testing and evaluation practices.
• Evaluation - Experience designing and implementing evaluation instruments, conducting multivariate analyses, and developing dashboards or data visualizations.
• Testing Operations - Experience training and managing proctors, overseeing secure test administration, and maintaining compliance with testing standards.
• Collaboration - Demonstrated success working with schools, families, or external stakeholders in test administration contexts.
• Professional Growth - Interest in contributing to psychometric innovation, with potential to grow into higher-level leadership in testing and evaluation.

Technical qualifications and specialized certifications

• Proficiency in R or SPSS, with ability to write and edit scoring programs; familiarity with software such as Remark, Qualtrics, or similar survey/evaluation tools.

Classified Title: Manager, Testing Operations & Evaluation
Role/Level/Range: ACRP/04/ME
Starting Salary Range: $64,600 - $113,300 Annually ($90,000 targeted; Commensurate w/exp.)
Employee group: Full Time
Schedule: M-F, 8:30-5
FLSA Status:Exempt
Location: Hybrid/Mount Washington Campus
Department name: CTY Research
Personnel area: Academic and Business Centers