fact_check

Usability Evaluation Plan

Use this Usability Evaluation Plan template to systematically organize and document your planned usability testing activities, including formative and summative evaluations, for medical devices prior to market release as required by IEC 62366-1:2015 and risk management processes. Complete this plan during the product development phase to ensure all user interface hazards and critical tasks are identified, evaluated, and mitigated, supporting regulatory human factors engineering requirements.
Generate ->

Usability Evaluation Plan

ID: Lorem ipsum dolor sit amet

1. Scope

The Usability Evaluation Plan outlines the planned Usability Evaluation activities and their required resources which will be performed for the device prior to its release. The processes are performed according to SOP Integrated Software Development and the processes within this plan.

2. Definitions

  • Critical Task: A task in a hazard-related use scenario, in which a user error can lead to significant harm.
  • Hazard-Related Use Scenario: A description of a user interacting with the medical device to achieve a certain result under specific conditions of use that can lead to a hazardous situation.
  • Formative Usability Evaluation: Usability interface evaluation conducted with the intent to explore user interface design strengths, weaknesses, and unanticipated use errors.
  • Summative Usability Evaluation: Usability interface evaluation conducted at the end of the user interface development with the intent to obtain objective evidence that the user interface can be used safely.
  • Task: One or more user interactions with a medical device to achieve a desired result.
  • User Error: User action or lack of user action while using the medical device that leads to a different result than that intended by the manufacturer or expected by the user.

3. Use-Related Hazard Scenarios

Hazard-related use scenarios have been defined as part of the risk management process according to ISO 14971. They are captured in the Risk Assessment and listed as being related to Usability. The hazard-related use scenarios along with tasks including critical tasks related to them will be identified in the Usability Evaluation Protocol.

4. Usability Evaluation

Usability evaluation will be performed during the development of the product and prior to product release upon completion of the user interface. The usability testing to be performed will be dependent on the use-related hazard scenarios and the tasks associated with them. A specific outline of the tasks required to be completed as part of the Summative Usability Evaluation will be included in the Usability Evaluation Protocol and the results will be analyzed in the Usability Evaluation Report.

4.1 Formative Usability Evaluation

Formative Usability Evaluation is performed during the development of the device. It is considered part of the development process and can be conducted ad hoc without a formal usability protocol or similar document. The purpose of formative usability evaluation is to proactively improve the product with considerations of human factors and usability during the development of the product. As such, performing formative usability evaluation on portions of, or the entirety of, the user interface ensures that it achieves sufficient quality and increases the likelihood that a summative evaluation (if necessary) of the user interface will be conducted successfully.

Processes that constitute part of the formative usability evaluation can include the following:

  • Review of device mockups or prototypes with medical, patient population, or user subject matter experts to gather feedback for device usability and design.
  • Showing concepts, designs, mockups or prototypes to internal or external UX/UI experts with the goal of gathering feedback on potential usability problems and improvements.
  • Testing users of the intended target user group on the use of the device while observing them and documenting potential user errors, abnormal use, and task completion as well as gathering general feedback. This is performed on a developmental, non-final version of the device with the goal of gathering knowledge for improving the final device.

4.2 Summative Usability Evaluation

Summative Usability Evaluation is performed on the device with its final user interface with the intent of testing hazard-related use scenarios and can be thought of as validation of the use-related safety aspects of the user interface. In the case of a design modification, the summative evaluation can be performed by looking at data obtained from previous summative evaluations for the parts that have not changed.

User tests for the summative evaluation will be based on the hazard-related use scenarios and will be performed according to the Usability Evaluation Protocol. The user tests will consider the following requirements:

  • The users selected to perform the users tests should simulate use with the intended users of the device.
  • Accompanying documentation intended to be supplied with the device should be available to the user during the testing. Training should be provided if required as part of the intended use of the medical device in the intended use environment. If user training is a risk control measure, training needs to be received and an appropriate elapsed time to accommodate for learning decay needs to occur.
  • All Hazard-Related Use Scenarios must be covered by tests performed during the summative evaluation.
  • A sufficient number of patients should be enrolled for testing to ensure adequate validation of all tests. It is suggested that 5 or more users are tested but this can be adjusted based on the product and tasks to be performed.

The Usability Evaluation Protocol will identify use scenarios, tasks, and expected acceptance criteria for a successfully executed task related to the hazard-related use scenarios.

The Usability Evaluation Report will provide an assessment of the results of the execution of the usability evaluation protocol and will summarize the data. The report will assess whether use errors occurred, the users had use difficulties or the users successfully completed the tasks associated with the hazard-related use scenarios (i.e. correct use) that were selected for inclusion in the evaluation. Use difficulty, while not necessarily a use error, can progress to a use error. A use difficulty where a user almost commits a use error while performing a task but recovers in time to avoid said use error can be considered a "close call". Difficulties performing tasks should be included in the usability evaluation report as relevant.

4.3 Results of Usability Evaluation

The results of the usability evaluation will be summarized in the Usability Evaluation Report. The Usability Evaluation Report will provide an assessment of the results of the execution of the usability evaluation protocol and will summarize the data. The report will assess whether use errors occurred, the users had use difficulties or the users successfully completed the tasks associated with the hazard-related use scenarios (i.e. correct use) that were selected for inclusion in the evaluation. Use difficulty, while not necessarily a use error, can progress to a use error. A use difficulty where a user almost commits a use error while performing a task but recovers in time to avoid said use error can be considered a "close call". Difficulties performing tasks should be included in the usability evaluation report as relevant.

Any risks related to use errors or usability that were previously not identified in the risk management file will be added to the risk assessment. Risk controls that were evaluated during the summative testing that were determined to be insufficient should be updated in the risk assessment. User errors that arise out of the usability evaluation testing shall be treated as new risk inputs and undergo the same process of assessment and integration into the system requirements according to SOP Integrated Software Development.

Medical Device Certification Made Easy.