Panel Discussion/Debate

Presented by:

• Stephanie Dille, Meazure Learning
• Rachel R. Schoenig, Cornerstone Strategies, LLC
• Donna McPartland, McPartland Privacy Advising
• Kellie Early, Examining Solutions LLC
• Isabelle Gonthier, PhD, PSI Services

The opt out movement is in full swing in the education space. From K-12 to admissions, critics are vocally urging leaders to forego standardized testing and their advocacy is seeing results. Today, more colleges and universities are test optional than at any time in the past. Does this foreshadow changes in the credentialing and licensure space? Is it possible that testing will become just one of several options individuals can use to demonstrate their qualifications for a job or profession? As we look to the future, how will verification of skills and knowledge will evolve - will apprenticeships and diploma privilege render testing obsolete? Or will the intersection of the worlds of academia and credentialing include more certifications as part of a course or degree requirement? And perhaps most importantly, are there steps we can take as testing professionals to make the path ahead one of sunshine and roses, rather than doom and gloom? Join seasoned professionals for a spirited debate on the future of testing. Explore these issues and leave with key considerations to help shape the journey ahead for the assessment industry. It’s a session you won’t want to miss!

E-Poster

Meet the Authors:

• David Chen, PhD, Meazure Learning
• Jackie Holt, Meazure Learning
• Todd Galati, American Council on Exercise
• Natalie Rose, Certification Commission for the National Association Medical Staff Services
• Meghan Leeming, University of North Carolina at Greensboro and Meazure Learning

Live remote proctoring offers candidates with a new, flexible alternative to traditional in-person, proctored testing. However, testing programs must provide validation evidence that these exam administration modalities are comparable. This requires an analysis of both candidate scores and item performance. This poster presents statistical analyses and psychometric considerations when providing support for the use of both exam modalities. If statistical differences exist between modalities, steps to address these differences are shared.

Concurrent Session

Presented by:

• Gabriell Darby, Meazure Learning
• Cory Clark, Meazure Learning
• Simon Welham, Mulesoft
• Alison Kramer, Caveon Test Security

Cheating and content theft can seem like a never-ending cat-and-mouse game for global, high-stakes programs. Online exam security is critical to prevent proxy testing, identify potential misconduct, and protect content from being compromised or shared, but security doesn’t begin and end with the proctor. To ensure exams are protected, consistent and innovative security processes must be used and evaluated for effectiveness. Collaborating with several responsive and forward-thinking exam security providers, MuleSoft deployed a constant monitoring and enhanced security strategy that allows program leaders to not only catch “cheaters” but also identify integrity breaches and continually develop new tactics.  Hear a case study in which Meazure Learning’s enhanced security program detected data points identifying potential collusion, which led to a deeper analysis by Caveon’s data forensics team. We’ll state the analysis outcomes as well as MuleSoft’s response and policy changes given the advancements in available detection tools. Hear firsthand how the collaborative approach between three industry leaders provides a balance of security and IP protection while also focusing on the overall candidate experience. These organizations work closely to identify security threats and create scalable solutions with highly defensible outcomes. Attendees will get a peek behind the enhanced security curtain and a broader understanding of the importance of partner collaboration when it comes to protecting global exam programs.

Panel Discussion

Presented by: 

• Sarah Mastrianni, Meazure Learning
• Alexandra Link, American Council on Exercise
• Nikki Klaus, Medical Group Management Association (MGMA)
• Cindy Lemek, Wound Ostomy Continence Nursing Certification Board (WOCNCB)

Any given item can go through round after round of editing and still not be quite right. So how do you go about limiting the number of edits needed when authoring new items? Join this session for an interactive discussion around recruiting, training, and retaining subject matter experts. ACE, ACMPE, and WOCNCB will share their approaches to maintaining a top-notch SME pool. Applying one or all of their strategies can save time, money, and the headaches that accompany endless review cycles. Additional topics of discussion will include how to incentivize subject matter experts when compensation is not an option, how to keep SMEs engaged during long remote meetings, and how to meet a program’s item development count goals and deadlines.

Concurrent Session

Presented by:

• Stephanie Dille, Meazure Learning
• Larry Lynch, National Restaurant Association
• Joel Oliva, National Commission for the Certification of Crane Operators (NCCCOS)

Over the past two years, the credentialing industry experienced rapid growth of online testing. Many programs were forced to quickly pivot the delivery of their exams online to continue providing their credential(s). For some, this meant transitioning candidates from test centers to at-home testing. For others, it was moving from in-person or classroom testing solutions to online solutions. Many were challenged even further by test-taker populations not accustomed to using computers for their day-to-day work or that may not have access to adequate technology. In addition to test-takers, other stakeholders are impacted by these changes. Despite these challenges, many credentialing programs quickly introduced a new exam delivery modality to a wide variety of stakeholders. And like with any new challenge, learned a lot. This session will highlight the experience of two large, high-stakes testing programs that have transitioned diverse test-taker populations to online delivery. Hear about the importance of a thoughtful communications plan to introduce a large program change to all stakeholders. Understand what to consider about your audiences if you have recently introduced online testing or plan to in the future. Examples of successful communications tactics for test-takers, educators, and board members will be shared, along with the results (what did and didn’t work). Throughout the session, the presenters will highlight their experience educating a diverse population of test-takers. Hear some concrete ways to communicate a transition online to non-technical test-takers, and review the importance of test-taker choice and feedback as part of the transition to online delivery.

Roundtable Discussion

Presented by:

• Chris Beauchamp & the R&D Committee 

Panel Discussion

Presented by:

• Chris Beauchamp, PhD, Meazure Learning
• Adrienne Cadle, PhD, Professional Testing, Inc.
• Guillermo Ortiz de Zarate, Lineup
• Nicola J. Wright, Board of Canadian Registered Safety Professionals

Credentialing and assessment exams are notorious for perpetuating unintended unfair impediments that prevent access and opportunity for traditionally underrepresented identity groups. A panel of experts will examine and discuss important perspectives to keep in mind along the journey toward advocating for a much-needed transformation and reform of credentialing practices.

Concurrent Session

Presented by:

• Cory Clark, Meazure Learning
• Alana Chamoun, CSBS

Have you ever wondered how test-takers try to cheat on exams? No test – regardless of modality – is free from cheating attempts. Each has its own security concerns that need to be mitigated. And it’s important for credentialing professionals to understand the security landscape of their program so they can confidently collaborate with their partners. Join two leaders in test security as they recount real stories and examples of cheating attempts in test center and online proctoring environments. Some examples are creative, some are funny, but all of them are interesting. In this interactive session, you will learn about test security threats and proven tactics to mitigate those threats, no matter the modality. You’ll also get expert recommendations on how to conduct a health check of your program’s security landscape. You’ll leave the session with a deeper understanding of why test security should be top of mind and actionable steps to take back to your own exam program.

Panel Discussion

Presented by:

• Maria Incrocci, PhD, Meazure Learning
• John Hosterman, PhD, Paradigm Testing, LLC
• Bill West

Innovative credentialing programs consistently review and evaluate the effectiveness of assessment design. Credentialing organizations have developed and implemented innovative assessment strategies, including unique item types, simulations, gaming, performance testing, and multi-modal forms of assessment. Indeed, the industry has witnessed a myriad of technological advances which have evolved test development and delivery to mirror the knowledge, skills, abilities, and attributes of candidates in their job roles. But while these advances are exciting to test developers, they may be problematic for candidates with disabilities. Can we develop innovations in our test design and test delivery, while still accommodating candidates with disabilities who have unique accessibility needs? This session will focus on key areas of consideration for test developers as they pursue new and innovative assessment content and methods for delivery. The presenters will address types of test adaptations that might be problematic for candidates with various types of disabilities. The presenters will emphasize practical considerations for accessible test development, and strategies to avoid potentially costly and time-consuming remediation efforts if your organization develops a test that isn’t accessible.

Panel Discussion

Presented by:

• Moderator: Linda Anguish, Institute for Credentialing Excellence
• Amin Saiar, PhD, Board of Pharmacy Specialties
• Chris M. Beauchamp, PhD, Meazure Learning
• Maria Incrocci, PhD, RPh, Meazure Learning
• Paul E. Jones, Ph D, Pearson VUE

In 2020, the Institute for Credentialing Excellence (I.C.E.) embarked on a process to develop the certified credentialing professional (ICE-CCP) certification program designed for individuals working in the credentialing industry. The program was built from the ground up, with several industry partners collaborating with I.C.E. to design and develop the key components of a credentialing program. This work culminated with the launch of the ICE-CCP pilot examination in November 2021. As members of our credentialing community may expect, this was a program without precedent. Both the subject matter expert volunteers who supported the test development process and the pilot test candidates represented a unique population of individuals: those who are currently employed in the credentialing business representing the variety of roles across governance, operations, and assessment development. In other words, no credentialing examination has ever been taken by such an informed group of test takers. Moderated by Linda Anguish, Director of Accreditation Services at I.C.E., this panel will include measurement professionals from the credentialing industry who contributed to the job analysis, item development, test publishing, test delivery, statistical analysis, standard setting, scoring, and reporting of the inaugural ICE-CCP examination. Hear about their challenges and reflections throughout the process, along with insights and recommendations for future program activities. Attendees are certain to be enlightened and entertained by the several unique and occasionally humorous observations and encounters. Attendees will have the opportunity to ask questions of the presenters and to learn more about the ICE-CCP if they are considering earning the credential themselves.

Panel Discussion

Presented by:

• Moderator: Ashley Norris, PhD, Meazure Learning
• Larry Lynch, National Restaurant Association
• Elizabeth Stewart, CFP Board
• Jennifer Breton, College of Licensed Practical Nurses of Manitoba

One positive outcome of the pandemic has been a renewed focus on equity, diversity, and inclusion in everything we do. And testing and certification are no exception. Credentialing organizations, regulators, and hiring practices are increasingly focused on ensuring assessment fairness across a broad population of test-takers. The organizations responsible for developing and delivering these assessments have been diligently working on attracting a diverse population of test-takers, improving accessibility through hybrid delivery models, and implementing designs that minimize bias. In this session, panelists will discuss challenges and opportunities related to diversity, equity, and inclusion from three unique perspectives, including a large credentialing organization in food safety, a certifying body in the finance industry , and a nursing regulator. The panelists will explore key diversity and inclusion considerations as they build and promote their programs and partners, ensuring equity for all test-takers. They will discuss initiatives they have implemented, including hybrid delivery models, and share positive outcomes and lessons learned.