--- title: "Discover Why Traceability Doesn't Have to Be So Hard" type: white-paper publisher: Ketryx source: "https://www.ketryx.com/assets/discover-why-traceability-doesnt-have-to-be-so-hard" content: text extracted from PDF (layout/tables/figures not preserved) --- # Discover Why Traceability Doesn't Have to Be So Hard *Source: [https://www.ketryx.com/assets/discover-why-traceability-doesnt-have-to-be-so-hard](https://www.ketryx.com/assets/discover-why-traceability-doesnt-have-to-be-so-hard)* --- 1 | ketryx.com Executive Summary Software is the leading cause of medical device recalls1. The margin of error for the launch of new medical devices is near zero. Any defects not discovered and corrected during development can result in patient injury or even death. Manufacturers not only face hundreds of millions of dollars in recalls and legal costs1, they may also face significant reputational damage and shareholder harm. With the stakes so high, heavily regulated products such as safety-critical medical devices require that every step of the development process must be traceable with a rigorous process designed to identify potential risk factors and their contributing conditions. This e-book: According to McKinsey2, a single warranty or recall process can cost a manufacturer up to $600 million. This doesn't include the costs of lawsuits and other issues associated with the recall. In fact, the total cost to the medical device industry is up to $5 billion per year. Jul 19, 2021 Explores why traceability has traditionally been so challenging in medical software development and why cloud and AI only add to the complexity ___ 1 Productivity drivers in Medical device and healthcare software development. McKinsey & Company, https://www.mckinsey.com/~/media/McKinsey/McKinsey%20Solutions/Nu metrics/Resources/Insights%20on%20%20Medical%20devices%20Numetr ics.pdf 2 The Business Case for Medical Device Quality. McKinsey Center for Government, 2013, https://www.mckinsey.com/~/media/mckinsey/dotcom/client_service/publ ic%20sector/regulatory%20excellence/the_business_case_for_medical_de vice_quality.ashx Includes an FDA-approved template and guidance on a more integrated approach to traceability to empower manufacturers to develop better, safer software, faster 2 | ketryx.com What is Traceability? Traceability is an established tenet in the regulated software engineering community and is essential for assuring that software not only does what it is supposed to do but is safe for use. A Requirement Traceability Matrix (RTM) provides evidence and assurance that the intended users, use cases, and user requirements have all been realized, implemented, verified, and validated. Regulatory agencies from the FAA to the automotive industries all have similar traceability requirements. The ANSI/AAMI/IEC 62304:2006 standard addresses the requirements for traceability as part of the medical device software development lifecycle processes. Similarly, the US Food and Drug Administration (FDA) states that: “ Traceability analysis must be used to verify that the software design of a medical device implements the specified software requirements, that all aspects of the design are traceable to software requirements, and that all code is linked to established specifications and test procedures. ” "Verification" is the act of showing that the device behaves how you designed it to behave. "Validation" is the act of showing that the device does what the user of the device needs. 3 | ketryx.com A traceability matrix is a visual representation of the relationships and linkages between key areas of your design process, for example: • Your User Needs • Design Inputs (e.g., requirements) • Design Outputs (e.g., software specifications) • Design Verification • Design Validation Figure 1: Key areas of your design process. The goal is to view and analyze all changes made during development — including who made the change, what it was, when it was made and why it occurred in the first place so any issues can easily be isolated and mitigated before they cause problems. In theory, traceability allows teams to map out interdependencies at each phase of development. It ensures all compliance regulations are continually met and any future changes conform from conception to market to post-market surveillance, with meticulous documentation of everything in between, so that nothing is forgotten by accident. When done effectively, a traceability matrix should allow all stakeholders to see a clear path between the physical product and the design history, including evidence of quality controls. Finally, a traceability matrix can be an excellent internal tool for project management purposes. It helps teams track and define complex needs and expected outcomes, introducing much-needed predictability and consistency in managing projects effectively and staying on schedule. 4 | ketryx.com Why Traceability is So Hard Systems are Complex If teams were working on a system with only 10 requirements and 10 specifications they could manage it in an Excel spreadsheet – even if every item were connected to every other item. However, assuming the system was a large, cloud-based system it would have hundreds, even thousands of requirements and each item would be connected to the system in a variety of ways. While the spreadsheet would be easy to set up, efforts to sustain it as the number of items increase wouldn’t scale to support the interdependencies in a clear, functional manner. This is even true for the first version, not to mention the need for future changes, and the regulatory requirement to track that change and ensure end-to- end traceability. Figure 2: Shows why it's so difficult to track interdependencies from thousands of items across different documents and systems. You Have to Trace Across Different Systems Creating a traceability matrix requires a significant amount of coordination between different teams and departments within a company. Manufacturers typically store software source code in one place such as in GitHub and then store requirements and design 5 | ketryx.com controls in another system (or multiple systems!), such as Jira or PTC, and store testing in a third system and complaints in yet another system such as Salesforce or another CRM. You Have to Gather a Lot of Documentation While the traceability matrix itself is just one document, it needs to reference other supporting documents that demonstrate compliance with the requirements. This documentation can include design specifications, test protocols, test reports, and other forms of documentation. For example, the design team will need to provide information on the design specifications while the testing team will need to provide information on the test methods used to verify compliance. Coordination can be strained because the teams may have different priorities or may be working on different parts of the dev process at the same time. New Innovations Add Complexity Combining traditional software development with the cloud and AI introduces additional complexity. One of the biggest challenges with AI is to understand how to trace the output the AI system provided to an input or even back to system requirements. AI and ML are technologies based on statistics, performing billions of calculations in order to arrive at a decision or result. These large data sets make it difficult to trace or diagnose the source of incorrect results in a meaningful way, hence they are validated to certain performance parameters on a given dataset. As a result, having a stable, traceable system, to minimize additional sources of risk and noise is crucial. Most development tools are not designed to support the strict traceability the FDA requires. Building a traceability matrix using Jira and other development tools, can be challenging for several reasons. Jira, for example, is primarily designed as a project management and issue tracking tool for apps and games rather than a dedicated requirements management tool for safety critical applications. It can also be difficult to link requirements, design elements, test cases, and other artifacts, within Jira in a way that effectively demonstrates traceability. In many cases, teams need to use custom fields, tags, or other techniques to link related items which can be time-consuming and error- prone. Additionally, it may be hard to identify the relationship between different requirements and ensure they are properly aligned. 6 | ketryx.com Current State: Nothing is Unified In reality, most traceability processes are not performed as one unified process from the start – rather they exist as duplicate, parallel processes or worse, as an after-thought. Because connecting all of these processes is so difficult, and IT systems aren’t built for it, the default is a manual process, consuming an immense amount of time and overhead that could be better spent on improving the product. According to McKinsey1, approximately 70% of medical products are delivered late to the market, and the average schedule overrun for medical products is 25%. Many teams attempt to do traceability post-hoc – to fill in the blanks after the build with Excel spreadsheets and other documents. Other manufacturers purchase a QMS/document management system and spend time and resources customizing it to manage risk and validation but only add additional complexity and steps to the process. Figure 3: Shows how many MedTech companies attempt to enforce processes by manually reviewing each process step, checking for evidence of compliance and then copying info from the source systems into additional documents and quality systems to generate technical files and maintain records. Each step in this repetitive process introduces substantial product risk, leading to mistakes, compliance gaps, and even patient harm. The FDA frowns on constructing trace links to merely give the appearance of meeting a regulatory expectation2, and when observed in submissions, has commented that this diminishes confidence in the quality of the development process and subsequent safety of the delivered system. Furthermore, performing traceability in an ad hoc, after-the-fact fashion means that organizations incur all the costs of creating trace links without experiencing any of its benefits. ___ 3 Mäder, Patrick, et al. “Strategic Traceability for Safety-Critical Projects.” IEEE Software, vol. 30, no. 3, IEEE Computer Society, May 2013, pp. 58–66. https://doi.org/10.1109/ms.2013.60. 7 | ketryx.com Another approach is to pre-emptively try to design everything in the system with traceability. This process attempts to input all the information, including the requirements, the results, and proper specs and verification and validation tests into one huge spreadsheet ahead of time. The approach fails because, in reality, large complex systems can’t be predefined to such an extent. Until a team is actually developing, it’s unrealistic to expect them to identify all of the potential challenges and issues they may encounter. For example, a team may run into an off-the-shelf software component – often in SOUP or Software of Unknown Providence – that needs to be verified that it is correctly implementing what it is supposed to accomplish. This latter approach of defining everything from the start doesn’t leave room for the human insights that result from developing a product. Moreover, because of the manual efforts and inherent cutting and pasting with these approaches, the process also opens the door to human error that can resurface in a product defect and patient harm. A Better Approach It’s clear that companies building FDA-regulated software will have to use a combination of processes and tools to effectively generate a traceability matrix, and maintain traceability throughout the product lifecycle. Two key processes to make this simpler include: • A structured requirements management process with clearly defined roles and responsibilities, as well as clear guidelines for how requirements will be written, tracked, and approved. This should include using a template for requirements which helps to ensure that all requirements are written in a consistent format and include all the necessary information. • Another important process is to establish a change control process that allows for changes to be tracked, evaluated, and approved. This component must identify the source of changes, documenting the impact of the changes, and revising the traceability matrix as necessary. FDA guidance4 5 argues for embedding traceability as early as possible in a project’s design phases and continuing to trace “incrementally.” This approach establishes continuous ___ 4 Design Control Guidance for Medical Device Manufacturers. FDA, 1997, www.fda.gov/regulatory-information/search-fda- guidance-documents/design-control-guidance-medical-device-manufacturers. 5 General Principles of Software Validation. FDA, 2002, www.fda.gov/regulatory-information/search-fda-guidance- documents/general-principles-software-validation. 8 | ketryx.com traceability across IT platforms and the other myriad of tools and platforms used in day-to- day work. In addition, all project stakeholders can benefit from traceability knowledge throughout the project. Until recently, few tools were available that could actually support this type of functionality. Introducing Ketryx Ketryx is a software solution that overlays the existing tools in many manufacturer environments with computer or machine intelligence to ensure that traceability isn’t an after-thought. Ketryx becomes a living part of the process, empowering all team members to use the tools in which they are comfortable during the development process while remaining compliant, and eliminating the hours spent on hand-offs and approvals. When you create an item in Ketryx, for example, there is already a set of defined other items that items can be traced to – ensuring the process is intuitive and logical. As an example, the Ketryx system would never allow the design output of a requirement to be a complaint because it’s not logical. Change management is also simplified because teams are using one, connected system rather than taking pieces in and out of the system and then having to reconcile them in yet another document. Figure 4: Future state after Ketryx – Effective traceability built into the process is a critical step to ensuring regulatory needs are met and more important, that innovative products can continue to be launched quickly and safely. 9 | ketryx.com With intelligent guardrails, Ketryx doesn’t allow any project to progress unless it is validated and guides users on next steps to stay compliant and move the project forward. Essentially, Ketryx ensures that any software developer can become a regulated developer with minimal training. Developers can focus on coding and product impact rather than endless documentation and repetitive hand-offs that lead to frustration and costly errors. As an added bonus, with increased developer satisfaction, companies reduce attrition and the high costs of developer churn. The attached template helps teams in their traceability planning and design stages. Closing Thoughts Traceability, while simple to understand, is nontrivial to create and maintain. But, it is an essential component of building safety critical systems. About Ketryx Ketryx is the first and only Connected Lifecycle Management software for MedTech that seamlessly creates traceability across the application lifecycle. Compliant by Design, Ketryx embeds guardrails to eliminate the manual, error-prone cutting and pasting from traditional disconnected medical systems to reduce the risk of defects while delivering enhanced patient outcomes. 10 | ketryx.com Appendix I Trace ability Matrix Use Case Requirement (Design Input) Software Specifications (Design Output) Verification Validation SR# SR SRS# SRS TC# Verification Report TE# Validation Report SR = System Requirement | SRS = System Requirement Specification | TC = Test Case | TE = Test Execution Use Case Defines the situation in which the software could be potentially used Requirement (Design Input) Defines what the software should do to meet the use cases Software Specifications (Design Output) Defines the technical functionalities of the software that meets the requirements Verification Checks to see if the developed system meets the design specifications Validation Checks to see if the developed system meets the needs of the stakeholders (i.e. use cases). © Ketryx Corporation, 2023.