---
title: "Why Resubmit When You Can Ship - Accelerating Innovation with PCCPs and CI-CD"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/5i85opyvoy"
content: auto-caption transcript, proper-noun corrected
---

# Why Resubmit When You Can Ship - Accelerating Innovation with PCCPs and CI-CD

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/5i85opyvoy)

---

today, we're going to discuss how teams are accelerating innovation using PCCPs and CICD, and more importantly, how they're doing it without getting stuck in the resubmission cycle. As AI enabled systems become more dynamic, the ability to update, validate, and deploy changes quickly becomes a competitive advantage. But that only works if you can maintain compliance as you move faster. So we'll explore how teams today are leading and using, predetermined change control plans, combining modern development practices to continuously ship updates while maintaining a strong and defensible safety case. So all the lovely things I love doing in quality and regulatory.

So very excite excited to speak to you all today. So why are we here? So PCCV adoption is growing. We actually recently did talk at RAPS about the subject. I loved talking about it.

So more AI enabled devices are getting approved, we're seeing increasing momentum around PCCPs. But despite that, only one in ten cleared AI enabled devices are actually using one. So this is very much, you know, on the rise, but we're still seeing not quite full adoption. So if PCCPs are so powerful, why isn't everyone adopting them? The challenge isn't that the bottleneck just doesn't go away.

It just shifts. So instead of being blocked on submissions, teams end up getting blocked on documentation. So another way to put that is the speed gains of using a PCCP aren't realized if teams are still spending most of the time documenting rather than waiting for approval. So then why would you put in all this extra effort for the PCCP if you can't get that speed gain anyway? So engineers end up spending a significant portion of their time, often around fifty to ninety percent, generating and maintaining documentation just to support the changes.

So as AI systems are increase the rate of frequency and updates, that burden only grows. So while PCCPs unlock faster approvals, they also require a fundamentally different approach to how documentation and evidence is generated. So a little bit about, you know, why who we are, you know, how do we solve this? Ketryx is built to turn your work and your teams into, what turning what they are already doing, to generate continuous assessment ready evidence for PCCP adherence. So one way of saying this, you know, instead of asking teams, hey.

You know, you gotta leave your tools. Now you gotta go do this other thing where you work in this other system, and you need to start filling out this PCCP form. Ketryx will actually directly connect to where the work happens. So imagine you're using GitHub or using you have an Excel spreadsheet or you have a Google Drive or using Jira. We connect to your existing platforms so that you get to stay where the work happens, use that software for what it's great at.

So whether it's system engineering testing or managing your AI pipelines, Ketryx can enforce your processes inside CICD, maintain traceability across requirements, code, tests, risks, and continuously generates the evidence as regulators expect. That means no more switching, you know, between things and, you know, here and there. That's something I certainly did a lot of when I was in quality and regulatory doing this for a medical device company. So as changes happen, whether it be in the code or the models are getting updated or data is changing, Ketryx just keeps up with it, and it has real time compliance. So instead of PCCP shifting the bottleneck to the documentation, the evidence is so continuously generated as a byproduct of development that it enables you to shift faster with those updates.

So I know Ryan gave us a a warm intro there, and I really appreciate, you know, the the things that he highlighted. I would also like to, you know, introduce Adrian who I think will will come off my care and and introduce himself a little bit. So I did come from the medical device world, and I spent the last two years here at Ketryx helping a lot of med tech companies, whether it be startups, medium sized companies, or enterprises, use the Ketryx platform and implement AI workflows for things like quality, regulatory systems engineering, and really help them through whatever goal they're going through. So it could be that they have a really big five ten k, and they're worried about, you know, should we put a PCSP in there? Should we, how do we document this AI model?

And that's what I've been doing for the last couple of years. And Adrian, I'm so excited to have you on the call. Adrian, I think I'm always very impressed by your, you know, your experiences at Boeing, how you think about program management especially, seeing it, you know, scale from a small team to a super large team. So I'll just give you a moment to just introduce yourself. Yeah.

Thank thank you, Jen, and and, yeah, thank you all for for joining online. A a pleasure to be here. And, yeah, a little bit of a a different background there there with Jen where I started my career from a mechanical engineering perspective working through Boeing, really seeing a a very regulated industry there and then jumping to to Novanta more in a program management role. But in that program management persona, you kinda see the the tension or conflict sometimes between teams, primarily quality of engineering where engineers wanna move fast, stay in their tools, and the quality team wants to make sure, hey. We're complying to the right regulations.

So I remember the first time seeing Ketryx and and seeing that demo. My my eyes lit up, and and I got really excited. I'm like, wow. Like, this this is this is awesome and powerful. So really excited to be here today to to show that demo.

What what got me excited about Ketryx, and we'll we'll definitely tailor it and go into PCCPs and validated CICD. But, yeah, really excited to be here. Excited to have you join us. I I think, you know, when I think about the hard problems out there to solve, I'm sure one of the things you definitely saw at Boeing and your previous workplaces is just change management is a huge problem. And what's kind of cool about the PCCP is you kinda have to preload that.

So you almost have to, like, get all your ducks in order ahead of time of what you think that changes. So is that kind of, like, one thing that really resonates with you about, you know, PCCP and getting that change management sort of sorted out? Yeah. Yeah. One hundred percent.

I I think regardless of the the size of of company, whether you're you're large scale over a hundred fifty thousand employees or or on the smaller end of less than a hundred, chain change management is hard. How do you keep everyone aligned as you're going through development? I think program manager's nightmare is, like, you get to the end. It's like, oh, we need we need to change, and the number of releases always add up. So, yeah, it's definitely difficult, and I think PCCPs help shift that that left and think about how your software architecture is broken up, how do you structure it and think ahead, kind of plan your requirements ahead of time, to enable you to move faster, down the road.

So, yeah, a great tool that that we see being adopted here over the last couple years. So I'll quickly walk through our end of what we'll cover today. So first, we'll cover the ground of what a PCCP is and why it's becoming so important for AI enabled systems. Then we'll take a look at why PCCP is often failing practice, kind of maybe touching on what you just said, Adrian, around, you know, that getting getting things planned ahead of time is always a challenge. And we'll from there, we'll cover how teams are using validated CICD to fully leverage PCCPs and continuously ship compliant updates.

And finally, Adrian's gonna walk us through a live demo of how you pull all those pieces together, you know, across the architecture, testing, traceability, and evidence generation in Ketryx. So very excited to touch on those today. Before we get started, though, I'd love to dig a little bit deeper and, you know, hear from the audience here. I wanna get us a quick sense of where everyone is coming from, hoping we can tailor some of our q and a around that. So we're launching a quick poll.

So we'll we'll quickly tackle this first question. So first off, do you have a PCCP in place? So you should have a poll on your screen right now. And if you wanna just go ahead and answer a few of those questions, you know, do you have a PCCP right now? And, you know, if the answer is no, that's totally okay.

We actually love that. It means we have an opportunity to learn a little bit more. And then we'll we'll be sure to provide the results very soon. Yes. Also, for those of our audience watching from, LinkedIn live events, we'd love to hear from you as well.

Please send in your thoughts into the comment section, to this question again of do you have a PCCP? And to our audience, I think we've got most of our answers in, so I'm gonna go ahead and share our results now. Wow. Okay. So it looks like we have quite a few folks who are still trying to figure out, you know, how feasible it is, you know, understanding the requirements.

It's in progress. We'd love to hear about some of the challenges you're facing when it's in progress. And then we have a few that are also, like, not quite considering. So interesting. Really, really excited to dig into some of that.

We have one more poll that we wanna ask kind of, like, to further tailor this. So we'd like to know, you know, is your device AI enabled? So we're actually gonna cover AI enablement in this presentation. Would like to know, yeah, is your device AI enabled? You know, is it production, development, that sort of thing?

Yeah. And I'll chime in here too is, yeah, when we think about PCCPs, kind of the the AI enablement and and retraining models, is a very, u easy use case to think about and see see how you can use it and leverage a PCCP, but, the guidance from the FDA is much broader. There there's a lot of different areas that you can apply to PCCP. It's all about planning in advance and what do you expect those changes to be and very being very explicit. So, yeah, we'll we'll definitely talk about AI enablement because nowadays, everything's everything's moving in that direction.

But just wanted to call that out and and a a distinction as you you read through kind of the FDA's guidance. It's not just all about AI. Alrighty. So I think we have our poll results. So it looks like wow.

So we do have some folks who have an AI enabled product, you know, in production, which is fantastic. That's very exciting. If you think about, you know, where we are are now versus just ten, twenty years ago, it's amazing to see this innovation already hitting the market. And I see we have quite a bit in development still, which is good. Refinement is good or even getting your first product out there.

And then a few early stage end notes. So this is super exciting because PCCP is an AI. They go together like piece in a pod. You know, there's the most amount of information we know about how best to implement a PCCPs all around AI enablement because it's actually a case for making it safer. Right?

You're maintaining the overall safety and effectiveness of this product. So, Adrian, quick question for you. From what you've seen, why is documentation such a burden when it comes to, you know, just regular work? And then when you start adopting PCCPs, how does that just get so much harder? So what is it about evidence generation that, you know, makes this so hard to scale and actually implement a PCCP?

Yeah. Yeah. I mean, especially as as as systems get more and more complex, there's more data that you're pulling in. There's more tests that you run. Teams spend, from from what we see a lot of time documenting everything that they're doing, and and you'll we run into cases where developers are spending more time documenting than actually coding and developing.

And so so Ketryx enables kinda automated document generation by connecting to your tools, and we see that we can sometimes save up to ninety percent of time in documentation and generate that evidence automatically per your your QMS or per how your PCCP is established. So you can set up a Ketryx to, you have your milestones, have your documents that you need, and generate them in a click of a button. So I'll be very excited to show that later. Yeah. I think one of my favorite things about Ketryx is that things are so visible in Ketryx.

Right? And you can kinda see this is the PCCP related test, and, like, that's always bubbled up to the top. So we actually have someone that we've worked with, a partner of ours, where they've done exactly this. So they've actually opera operationalized their PCCPs. So this is the exactly the challenge we're talking about.

You know, PCCs can unlock faster updates, but in practice, teams end up getting kinda, you know, slowed. And, you know, we had one partner where Beacon BioSignals were. They were building an AI powered sleep and brain activity analysis hardware and software product, and they ran into the same bottleneck. So even with a PCCP in place, time to market was still slow because documentation was manual and labor intensive. Requirements and specifications lived in Word and Excel.

Traceability was error prone, and teams were duplicating work that already existed in code. The solution with Ketryx that they found was that they automated traceability across their hardware, software, and machine learning systems. So think about how all those different components how how you can automate the traceability around that. I know sometimes they're kinda, like, running between teams. They integrated directly with their CICD pipelines and started generating evidence continuously from live work rather than, you know, work that was kinda done after the fact.

And what was the results? So they were actually able to release twice as fast, so reduce documentation time by seventy five percent and keeping engineers working in their existing tools and workflows. So instead of PCCP is kinda creating this, like, additional layer of documents that they have to do on top of their existing ones, they were actually able to fully realize it and integrate it into their tooling and their workflows, which I think is amazing. This is the kind of thing that I think the regulators want to see. Right?

This this real time compliance or real time evidence because they want full assurance that this PCCP that is keeping your device safe and even potentially providing more benefit, is still meeting all of the the performance requirements that they wanna see. Alrighty. I feel like that's that was a lot. I'm just gonna take a quick pause here. So before we get into how, we implement this and let's level set on what a PCP actually is.

I mean, I've been saying the acronym really, really fast and very often, so let's walk through what that actually means. So this is why it's harder to operationalize potentially than what it might seem. So at a high level, a PCCP allows you to make defined changes to your device without needing a new premarket submission each time. That means going to the FDA, paying that fee. Instead of going back to the FDA for every update, so even like a software update that just kinda fills that, you know, significant change criteria, you define it in advance the types of changes you're going to make.

So how you'll validate it, you're gonna talk about how you're gonna control those updates that ensure safety and effectiveness. That gives you, sort of a preapproved pathway to ship updates more quickly. You can see that on the right. So without a PCCP, every meaningful modification triggers a full validation and submission cycle potentially. You know, this might be different ways, special five ten k, regular five ten k.

With a PCCP, those changes can happen with an FDA authorized framework. So you can move through those modifications and verifications and ship without resubmitting. You're kind of incorporating it as part of your technological characteristic of your device. So this is especially critical for AI and machine learning systems. Models improve or sometimes maintain safety by having those improvements because we know that there is performance drift over time.

Even the, you know, the population data can change over time. So retraining is not just expected. Potentially, it's also encouraged depending on your intended use and your target patient, patient population. So instead of training every update as a new submission, PCCPs allow you to continuously improve what while maintaining a controlled and validated process. PCCPs really come down to three core sections.

So first is the description of modifications. This is where you define what changes you plan to make over time. It's one of the hardest things to get right because you have to really think about, like, what is your product and how are you planning to update it over time. It's a lot of decision making you have to do, what's in scope, what's out of scope. The FDA is really good.

If you haven't read the guidance yet, they give you specific examples of what would be an acceptable change that isn't, you know, a huge intended use change. So once you have that figured out, then there's a modification protocol. So now that you have an idea of what you wanna do, how are you gonna do it? So this is how you actually implement and control those changes, how the data is managed when retraining happens, how validation is performed, and what your whole release process looks like. This is where a great reference to an SAP does a lot of work for you, does a lot of the heavy lifting.

And finally, impact analysis. This is how you evaluate each change before release, confirming it meets the predefined criteria, and understanding any risk or interaction effects. This is very important. This is where you really wanna get, you know, your regulatory folks, your risk management folks, really experts on the device to look at the data and say, yes. This is safe before you go ahead and release it.

Remember, you're kinda skipping the step where you could have had the FDA look at your data and give you kinda, like, that that extra check there. So you really wanna be sure that you're doing this impact analysis very well. So on paper, very structured, but this is where the challenges start to show up. So once you have a PCC in place, the bottleneck shifts kind of from approval to validation documentation. Right?

So the scope of changes expands to what you need to, you know, trace and test, and you can't easily change some of these things, right, because you kinda got it preapproved. So the protocols often live in documents instead of, like, real development workflows. So execution and documentation still can remain manual. Determining what actually changed across a system can also be very difficult. So if you don't know if you can't explain every single code change or every single retraining what happened and how it potentially impacted the performance of your device, you might be in trouble.

And that's where having a system that has a lot of visibility, you can really trace and show the impacts of any change becomes super, super impactful. And so in conclusion, you know, the problem isn't necessarily defining the PCCP. Now that can be, you know, a challenge in general depending on what your product road map looks like. But once you're over that, operationalize it is also very challenging and something we're really excited to show you how to do today. I'll briefly go over, you know, how PCCPs fit into your submission process.

I know we have a couple folks, you know, especially in the development era where, you know, you might not have yet a regulatory strategy. So I'm hoping that if you have an ML product, you're thinking about what strategy makes sense for you. Just to walk you through it, so first, you submit your device with a PCCP included. Right? So that is a technological characteristic that you keep.

Maybe if you're doing a five ten k, you'll do a substantial equivalence against your predicate device, and one of those changes is likely going to be a PCCP. It's gonna be one of the differences you're gonna discuss. The FDA treats it as, you know, a core characteristic of the device. So it's gonna be in that, like, nice regulatory section where you really have to talk about, you know, why this is this is, you know, doesn't raise any questions of safety and effectiveness. And then the FDA will, you know, give you that, five ten k letter.

And, of course, if you're doing a PMA, this can be a little bit different or de novo. At that point, your PCCP is authorized along with the device. So it's it's an it's a characteristic of your device. It's special about it. It can change.

Right? Your intended use and your indications are kinda, like, locked in. Right? You can't just suddenly, like, add something else. But this is where the value comes in.

You can now ship updates without necessarily resubmitting as long as those changes fall within the preapproved scope of your PCCP. And this is the benefit. Right? So if you need to change, the PCP itself, though, it does require new submissions. So do do keep that in mind.

You can't just, like, make these minor changes over time to the same change form in a way. So if your device stays the same and only the plan changes, the FDA will just wanna review that PCCP. Finally, you can always reference back to the cleared version of the device before any PCCP modifications are applied. So just so you know, this is kinda like the journey. Right?

So make sure that you're always checking back to see you know, as the ship of a Theseus has changed, make sure, you know, it still is the ship of a Theseus. We'll quickly walk through the predetermined change control plan as it applies to AI because I know we have a few AI, folks on the call, and I love that. It's so exciting. We start with an approved model. So this is if you wanna apply your pieces particularly for AI component.

We start with an improved model that's been deployed and is collecting new data from a real world use. Real world data in a and AI is, like, one of the coolest areas of regulatory right now because I think it's there's so much opportunity to really have real time, real world feedback where, historically, it's been a very long process. So our goal is to leverage how you know, this new data that you've collected from the field, leverage it to improve the performance. So the first step is data analysis. You wanna really make sure you understand the quality of that new data.

Think about the machines or whatever thing is acquiring that data, how that can impact your data and the overall quality of it. Verify that it represents the intended population. Right? It might actually be better maybe than what you started with. We then segregate this data, combining it in our different datasets, and create updated training and test datasets.

Now remember, we don't want we wanna avoid overfitting as much possible. That's a risk in the design development process. Next, we move into the retraining phase. So we take our approved model infrastructure and retrain it and combining old data sometimes or new data. This is where you really wanna justify your methods here.

Once you have your new model, we need to validate it as always. Right? So this is the thing that you're gonna call out in your modification impact analysis protocol. Probably, you know, as part of your impact analysis, you're gonna look at your validation report and results and see, is there any, you know, potential questions around, you know, are there any new risks? Is there something that went wrong?

You know, is it clear that this is passing our performance expectations? So if the model passes, that means you can proceed to the release phase, which is very exciting. So this cycle of data collection, analysis, retraining, and validation release allows you to continually improve your AI model while maintaining regulatory compliance. It's structured yet flexible. You you get to choose as, you know, kinda experts in your product.

You are ultimately accountable to the safety of your product, which means you also are empowered to make the right decisions to make it as safe as possible. So very, very exciting exciting thing we get to do. So this is where everything can comes together. So PCBs are powerful, but they can break down without automated compliance. Right?

Chasing after those reports and making sure you're compliant with with the PCB. Most legacy QMS tools from what I've seen out there were primarily built for waterfall development. Right? They probably were built in an era where we thought about supply chain of, like, hardware products for the most part, and they aren't necessarily always tailored for AI workflows. So be very selective about what tools you pick to make sure that you're you're actually able to get all the benefits of APCCPA.

So, as a result, you know, traceability sometimes will become manual if you're using other tools out there that aren't able to keep up with this. Remember that every update still requires assembling compliance evidence after the fact. Just because you have a PCCP doesn't mean, like, free reign. Like, you don't need to document anything ever ever again. Like, you still need to document it.

Right? So you still need to show evidence, apply that quality management system, q the QMSR. Now with ISO thirteen forty five, you need to document everything. And post market monitoring often remains disconnected. This is where you could really leverage post market and real world evidence to inform on whether your PCCP is going according to plan.

So even with the PCCP in place, teams are not always truly operating continuously. Still stitching things together, and that's why PCPs sometimes fail to deliver their full potential. Obviously, this is something I'd love to see changed, but I do think that, you know, the underlying advantage of a PCP is, you know, you get to keep things safer over time. So let's shift to the problem. So, like, more tactics, less so strategy.

How do teams actually solve this? So the example we're gonna run through today is how you can use your continuous integration continuous development, so CICD pipeline, to turn PCCPs into a scalable and repeatable workflow. So what is CICD? So it is basically, a pipeline in which you can continuously integrate your software and continuously deploy it. So it's the process of automatically building, testing, and releasing changes that are made instead of bashing them into large and frequent releases.

This is the thing that we worry about, you know, when you you hear about this website that's gonna launch in, two years. Then when it launches, it's, like, horrible. It's probably because they're not actually using the CICD process and the sort of agile development workflow. Validated CICD takes a step further by embedding your compliance requirements directly into that workflow. So imagine that CICD report tells you, yes, this code passed the test.

By the way, we're a hundred percent sure it's in compliant with our PCCP. So we can get that regulatory feedback on top of our quality feedback, and that's super critical for AI enabled devices. This is how validated CICD master PCCP. So a code change is automatically checked against your pre authorized scope. Pipeline execution enforces your modification protocol.

Test results generate the evidence needed for impact analysis confirming each change meets that predefined criteria, and deployment becomes a compliant release. So it's automatically compliant. Now, obviously, you might wanna add an initial check there, you know, human in the loop. But, you know, you get to do that without resubmission, which is where a lot of the benefit comes from. So instead of documenting compliance after the fact, it's in real time through a validated traceability CICD pipeline, and evidence is generated automatically.

Right? So as part of the CICD pipeline, you're also getting that, you know, known to file that change in the design history file, now called the medical device file. Alright. So quickly looking at, you know, the things you should think about when you're operationalizing a c I c your CICD pipeline and turning it using it for your PCCP. So at a high level, it comes down to building systems where every change is automatically validated, traceable, and release ready.

So you design your architecture, so changes naturally map to your preapproved modification scope. This will involve, you know, some conversations with your lead architect, your software engineer, making sure that you have a nice reflection of, you know, does does the mapping we have, our idea of what the architecture is, actually match reality of what's implemented in the code? You'll embed your testing directly into CICD pipeline so every change is validated automatically. Maintain continuous traceability between your requirements, code tests, and risk. So risk is obviously very important.

I'd love to see some, you know, bias risk in there as well, make sure that's kinda detected. And as the system evolves, you know, the procedures are enforced in a way such that it generates evidence when you go through each, step of the development phase and becomes a byproduct. So we're not gonna go too deep into any of these here. We're gonna show you actually how to do some of these things. And curious to hear your thoughts, especially in the q and a of, like, what you thought or any questions you'd like to raise, you know, things you wanna ask Adrian.

Adrian's gonna walk you through this and how it works in practice. Just one of the many ways you can operationalize your PCCP. Awesome. Thank you so much, Jen. Okay.

Let's get right into the demo. Okay. So I'm now showing a a demo organization here within our Ketryx platform. For this demo, I'll I'll kinda be walking through this in the persona of software development engineer where I'm working on one subsystem of our of our product, and I wanna make a change and and retrain on some new data from the EU and release a new version of that subsystem that the system level can consume. Kinda interspersed through my persona walkthrough, I'll also be doing just, a Ketryx one zero one and overview.

So I'll actually start there. The first thing I wanna show within Ketryx within this org level is actually navigate to our documents. And here, we can use Ketryx as an EDMS tool to store, approve, keep an audit trail of all your important QMS documents. Here, we can see my approved PCCP. And I'll actually show this, and this is a little bit of a preview to a template that we have on our website and something that we'll share at the end of this webinar.

But this is a a very basic template filled out template for a PCCP. Here with Ketryx, I can kinda see records of this PCCP and that I've approved it, but this is something we've worked on with with the FDA and see that version one point one of this PCCP has been approved. A couple things I just wanna show going back to, like, the three main sections of of the PCCP. First, you have your description of modifications, then the modification protocol, and finally, the impact assessment. All all three sections are very critical, in your conversations with with the FDA as you get these approved.

As we kinda have some introduction, a little bit of information about the product itself, and then we get into the description of modifications. So here for this PCCP, we have one modification number modification number one, basically saying that, hey. The modification that we wanna do is is retraining our our m ML enabled algorithm, and and that's those are the changes that we wanna do to improve performance. Then we get into the modification protocol where where we have four different work streams with very specific steps that we will follow to do each of them. So we have something on data management practices, retraining practices, so the procedure will follow to actually retrain the model, performance evaluation, and then updating our our procedures.

And and what's nice about the the template that we have is it ties everything together with the modification, description of modification, and the modification protocol together in one table. So here we can see, again, modification number one and then the modification protocol for each of these work streams as I'm working through this, and this is what the FDA is looking for. The last section is, of course, the impact assessment. So this is a very lightweight impact assessment. It's saying that the the change that we're making will improve the performance of our product, make it more sophisticated, adaptive, improve the accuracy, and this is why we wanna be able to make those changes.

So that's our PCCP template filled out, and we we, again, will be sharing a QR code and a link to to download that at the end of this webinar. I think, Adrian, if we were fully role playing this, this is kinda like me as the right person. I come to you as a software engineer saying, hey. Like, you told me that you wanna change it like this. I got this cleared with the FDA.

Now go execute. And Exactly. Figure out how you wanna execute this in your tooling. Exactly. Exactly.

So navigating back to our organization level within Ketryx, we see three projects that I've kinda highlighted here, and they're connected with a a systems of systems architecture. So going back to one of the important pillars that's needed for a PCTP, it's kinda you need to know your software architecture, your system architecture. So I'm highlighting these three three projects, and Ketryx allows you to view this as well with this graph view. So we can see the irregular rhythm notification system is the top level, and it is connected to two subsystems, one hardware subsystem and one software subsystem, which as a software developer, I'll be jumping into this. We can also see other projects that are within my Ketryx organization but not connected to this system.

The last thing that I wanna show before I dive into the the software subsystem is how these projects are. We've seen how they're connected, but how they're versioned together and independently. So I'll actually dive into the system here, go to our release, and see at the system level, I have two releases, one point o, which is active and has already been released, and two point o, which is one that's in work, and and the systems engineering team, not necessarily the software team, is working to release this. And if I go into the settings of both of these versions, I'll see that the reference projects, so my two subsystems, and what version of that reference project is being consumed by the system level. And this is something that's very important as you start thinking about your your system architecture, how you version these components independently, and and how you link them together.

So here in version one point o, we see, yeah, one point o of the software subsystem is being used and one point one of the hardware system. If I go to version two point o of the of the system level project, I'll see that we're actually calling now software subsystem one point two, which is as the software developer what I'm working on today, and the same version of one point one for the hardware subsystem. Great. So so now I'll jump into the software subsystem. So here, I'm on the all items page showing all the design control items that are being pulled in from my source systems.

So my as a developer, I'm typically working in Jira, and I can see Ketryx is reading in things like requirements, test cases, test executions, risks, software item specifications. And what I'll actually do is I I will click on this source link to open this requirement in Jira. So as a software developer, I like working in Jira, so, typically, I'll be here just calling out a a few key fields for our requirements. We have our description or or your compliant shell or must statement of what what the system needs to do, some specifications. And this is where where we'll see in a little bit inside of Ketryx that we have a new dataset for the EU, and that's that's my chain between version one point one and one point two.

You'll see a status field here. So what state is is the requirement in? Is it still in work? Is it resolved, or is it waiting for approvals? And this is where I'll call out two things that that Ketryx brings to Jira or whatever source system you're working in to import in widgets.

So one is approvals. So Ketryx, to have that complete audit trail, collects approvals through part eleven compliant signatures. And then once all the signatures are created, we'll con create a controlled record in Ketryx that will feed into your documentation. The second is is the traceability view. So on each item, you can open up a a traceability widget and see how your requirement is related to other items within that product.

And this is super important, especially as we we think about AI, and and I can go on in more and more use cases. But once you have that full connected picture of all your design controls and risks, you have a very powerful platform that you can run agents or AI skills on to see if there's any gaps or or what needs to improve. And I'm getting a little too excited, but there there's so much power that's unlocked there. Here in this traceability, we see how our requirement relates to parent requirements, what risks it's controlling. So you see we see here, we see two risks, what software item specifications are being fulfilling this requirement, and what validation test is testing this requirement.

Great. So now I will I will jump back into Ketryx and and show a a powerful feature, and and I foreshadowed it a little bit here. But within Ketryx, you can use a diff function to compare records to one another. So say I I'm working on this requirement, but I wanna see what changed since the last time I had a a controlled version of this requirement. And I can open up my history tab here on the side and then select the version that I wanna compare and click the diff icon for the version I wanna compare it to.

In doing so, we'll bring up red lines or additions that that differ between these two record histories. And in in this case, we can see that, hey. We have a new dataset from the EU that we wanna include in in this requirement. I feel like I spent so much of my career already just, like, comparing two things. Kinda like, what's the difference?

And, like, I love how Catchrix takes all that pain away so you can focus on the more important things. Like, are we following our PCCP and, you know, full visibility into that. Exactly. Exactly. And the last thing I'll just add here is that's kind of the the Jira item, the requirement, and what what's changed.

As Ketryx is that connected system and brings in all the information, we can also see here we're connected to our our GitHub repository. So I can go ahead and click the the source link here for this GitHub item, a software item specification, and see directly in in my repository, yeah, what what information we have here. And if I needed to, I can make an edit, and it'll automatically sync sync to Ketryx when I report my build. Alright. The next thing I wanna wanna talk about here within Ketryx, we see all our items being brought in.

But how do we get that global view of traceability for our entire project? So we saw the traceability for for one item in particular. But with the traceability module within Ketryx, you can see your entire traceability for the whole product. And what's really cool here is this is completely configurable to your q m QMS. So you can add columns to to align to your b model.

And we have these checks that are constantly checking every time a a sync is is made with Ketryx. Are you do you have full coverage between your use cases to your design inputs? Is or do you have full coverage for testing? Have your test executions, passed? Are they in your test plan?

Is everything controlled and approved? And in this case, you see everything is. We've got nice green numbers and checks at the top. And then we can get more interactive and kinda follow that traceability through the traceability matrix where we can see our requirement here that we've edited the the real time data processing is traced to four software items. And then if we look at our sensor module markdown file, we can see that it connects to a test case that's verifying it.

And then a note here and pausing on kind of that continuous integration with our validated CICD pipeline. Here, we can see this test case and the automatic test test executions that get reported to Ketryx every time you report your build. And this is where we we really cut down on that documentation time. As a a software developer, I can go ahead and in my, code make make a change to the test case, and then through my validated CICD pipeline, push that test execution. And here clicking into more details, I can see what what happened with this test execution that it passed.

I can see the bill that it was reported on and the rest of the test and the test executions that came from this build. So Ketryx keeps that full audit log and traceability and updates in real time in your project. The last thing I wanna show here is is when we talk about risks and risk controls. So within Catchrix, you have relations between your design controls and risk items. And here, I'm showing that I have two requirements that are acting as risk controls.

One of them, if I wanted to zoom in, is secure data management, and I could see that it's controlling two separate risks and that this requirement that is a risk control has a test case associated with it that has passed testing and completed our our our QMS requirement of having risk controls, also requirement in in our PCCP. Awesome. I I will now move on into, yeah, releasing this this software subsystem. I see I have my full traceability, and and, again, I wanna release version one point two of of this software subsystem. So this is one of my favorite views within Ketryx, especially coming from that program management background.

Here's where I can see the progress of this subsystem, the project for a PCCP specific, which this version is a release of a a change in in PCCP regulated. We have milestones on top where we can see what needs to be done in what order. And as I'm going through and progressing through my project, I need to approve each of these. So collecting the real world evidence, cleaning that data, then training my model, and finally having a buyers bias report generated. And here, I can see a real time view of my progress and design controls and test executions.

What I wanna show, before I go ahead and approve this this subsystem is actually go to the documents and how do we generate the documentation automatically within Ketryx. And we do that by click of a button. So we have out of the box templates that are available for everyone to use right away. We can also customize these through our templating language to match whatever your requirements are in your QMS. And what I wanna show here is this PCCP conformance report.

So now that I I've changed my requirement, you know, updated my code, ran my test cases, I wanna generate this PCCP conformance report. I wanna go ahead and approve this document, showing that I reviewed it, you know, approved it. Here's where I apply my part eleven compliance signature, which uses, in this case, a biometric signature of my fingerprint. I'll go ahead and approve this document, see that the approval was was executed within Ketryx. And I can go ahead and and download this file either as a a Word doc or if I wanted to as well as a PDF.

I'll show the PDF version here where I can see the traceability from my test cases to those test executions, and did they pass or fail? In this case, all of them passed. And then my risk table, so coming from my risk to risk controls, ended those risk controls past their test cases. At the bottom, Ketryx also amended those approvals that you you just made or I just made. So we see the time stamp from today, minute ago.

I signed off on this on behalf of all these groups, And now I can pull this into my EDMS or any audit and show show my full traceability and compliance to my PCCP all at a at a click of a button. So we're running a little bit short on time. So the last thing that I wanna do is show how I approve this project and kind of signal to to the systems engineers for the system, that we're working on that, you know, the software subsystem's ready to go. And this is another one of my favorite parts. I'll go ahead and click that approve button, apply my part eleven compliant signature, click approve, and then Ketryx will give you some nice confetti here kinda celebrating the the release of the the software system.

So, yeah, that that's, I'll wrap the demo there. I could also jump into the system if if we run dry on q and a questions. But, yeah, really, really excited to to be able to walk you through this. And maybe the one piece that I'll circle back on is is the continuous deployment aspect of of the validated CICD pipeline. We saw the continuous integration where you kinda commit builds and everything six into Ketryx For the the continuous deployment, as a program manager or as a team, we can we can establish our release cadence, say, every two weeks.

And within Ketryx, we can control all of our versions and know that, hey. Every Friday at five PM, we're gonna release. And whether you you get that change done by then, you'll be included in our release train. If not, you'll you'll get into the next release. So, yeah, another super powerful feature that's wanna highlight here.

But, yeah, that that's all I have for today. So so appreciate, kind of the the, the ability to walk through, and and I'll I'll kick it back here to to show our QR code to show how you can download that PCCP template that I showed. We had a filled out version, but this is available on our website. So feel free to scan that QR code and and pull that and and start getting to to work on your PCCPs. And I think now we'll we'll open it up for a little bit of q and a.

Yes. Well, thank you both very much for this very insightful presentation today. As we move into the q and a session, I wanna remind the audience they can also take a look at the handouts module within the GoToWebinar control panel. Where you'll find some additional documentation related to today's presentation to download for some further reading. Also, take a moment, to scan this QR code that I've got up on screen while you continue to send your questions and comments for this q and a portion of the webinar.

If you're attending via GoToWebinar, you can use that questions window. And for those watching on LinkedIn live events, feel free to post your questions in the comments section. I've already received some questions from the audience, though, so we'll get ourselves started with those. The very first question that I have for you here wants to know, if a manufacturer has FDA approved PCCP for an AI enabled medical device and implements a modification in accordance with that PCCP, What postmarked records and controls does the FDA expect to review to confirm compliance with the PCCP? And they give an example.

Are those changes primarily reviewed through inspections or other mechanisms? I love this question. This is a wonderful question that's really thinking ahead, you know, about when are those future events. I I wish I had your name, person who submitted this, but the I wish I had more anecdotal evidence for this. So something to keep in mind is that there's only been about a hundred plus PCCPs approved just kinda starting this year.

So how do they perform an inspections? What gets asked? We don't know quite yet. However, we do have a good sense of what will be asked because the PCCPs, when you submit them, you will, you know, call out the IDs of the protocols that you will be executing in order to get them, you know, your product updated, which means that, you know, the FDA can come in and say, yes. This, you know, protocol six four nine seven, we'd like to see all of the results of this protocol that you executed.

And you don't wanna be in a position where you said, oh, we haven't been running that for the last year, and we've been making changes to our algorithm. So keep that in mind that that is a very normal question that they might ask. And, also, one thing I would do to get ahead of that problem is actually one thing that Adrian showed today, which is that PCCP conformance report. Because if you already have a system where you're automatically generating these reports, it's basically free to be able to have an additional report that's just focused. It's kinda like a filter for all the existing testing that you might do that just focuses in on that testing that you would do for your PCCP and show full compliance to that PCCP that you submitted.

So really excellent question here, and I'm looking forward to seeing how, you know, the feedback that we get from the field, you know, from our partners, how the PCPs are standing up in their audits. Excellent. Very good. Thank you for that answer. I'm sure they appreciated the details.

The next question I have here asks if you can give some examples of the types of changes that fall under a PCCP instead of regular software changes. Yeah. I can take this one unless, Adrian, you wanna take it? Okay. I'm gonna say a nod of, okay.

I'll I'll grab this one. So PCCVs I'll tag tag in at the end, and maybe add a little bit from from, some of the FDA guidance. But, yeah, we'll do. Yeah. Yeah.

So PCCVs can cover extensive preapproved changes more than just like a typical software upgrade. One of the examples that you see is also, you know, adding a new input or output. So for example, you know, they might also allow teams to replace or substantially modify the model architecture. That's something you see in your typical significant change guidance. So not just base minor software tweaks or cybersecurity changes.

As long as that change falls within the approved framework and the team gets to retest it against the agreed criteria, you should be all good. So the key distinction is that change path has already been authorized by the agency. So that makes faster duration possible. Yeah. Then from my perspective, you read through the FDA guidance kind of at the end, they have a bunch of examples of what could be approved descriptions of modification.

And a couple of them were surprising the first time I read them where you can actually, from a a mechanical perspective, change the size of your product as long as that's predetermined and, you know, approved by the FDA. And and if you think through, like, how how would that actually work, like, you if you shrink a product size and the footprint, the the interface that it may have to another product will change. But but I think a very tangible example of that is say that, hey. You have a a product that interfaces with some surgical robot. You know, that surgical robot is gonna go undergo some changes, may maybe reduce, its size and its interface as well.

So that's how you can kinda get those those predetermined change controls approved and and understand your plan, and and it extends far beyond just AI and ML retraining, but you can even go into the mechanical world. Fantastic. Thank you very much. The next question here is, wondering how do you functionally distinguish PCCP changes within your QMS? Drew, do wanna take this Sure.

Yeah. I mean, your QMS and the and the PCCP, work a little bit hand in hand. Like, when the FDA comes in and you start your conversations, they will check that you're in good standing with your your certifications and your QMS and check your audit history. But the PCCP specifically calls out a a set of change or one change that that you agreed to follow and and what the impacts of those changes are. So while while there is some relation there of, like, you're you're following your QMS and and all your policies for development, the PCCP is very focused, just has a very narrowed scope on one product or or one change.

Yeah. And and one of the best ways to kinda, like, narrow that scope is just simply call it on the architecture. Right? You have that software architecture segregation. Perfect.

Alright. Thank you. Now the next question then I have for you here, it asks, how do you handle a third party model as part of a PCCP enabled regulated AIML subsystem? I love this question. So a third party model should be treated like any other subsystem subsystem or software component.

Now I'll have a couple caveats on that one. So it still needs to be brought under, you know, either manufacturer's control and framework. So it has to be qualified, and qualified is the keyword here, and verified through testing to ensure it continues to meet requirements. Now you could bring it into your control in two different avenues. Right?

You could treat it like, you know, your supply chain, like bill of materials. It's a dependency like any other. You could also treat it with your typical supplier controls that you would apply to say, you know, for example, if you were to buy, if you're making a car, you'd have a supply chain for things like wheels and components, and you'd qualify those suppliers and their ability to continuously supply that device. So that's where you might have, like, a quality agreement. You might have, you know, if it's a software component, you might have in terms of service, for example, if they're providing a service.

You might have to license that model. So you wanna really apply both your s bomb controls, your soup controls where you're doing that testing once you have it incorporated into your architecture and keep like, treat it as a component. And you do also wanna apply those supplier controls to make sure you can continuously incorporate as part of your architecture. And you just don't don't have either the rug pulled out from under you. Very good.

Thank you very much. Next up, someone's asking, what is the process for evaluating whether a change is within the scope of the authorized PCCP and checking whether the modification protocol was followed? Yeah. I I can take that one. So there there's a a couple of ways that you can structure this, probably more.

One option is to kinda define the release types that you're using, and the release categories, whether it's a full release of your product or an incremental release. So you can use targeted incremental releases under your PCCP to make those changes, like retrain your model. Another idea or or that you can use here is is to just make sure that every release of that subsystem is actually controlled by your PCCP. So that way, you know, any changes that you're making are pre already preapproved. You don't have to go and and resubmit that software subsystem.

So it's definitely a little bit more advanced. You have to do a lot of that thinking upfront. But if but if you can do it, yeah, a lot of value that you can get get out of that. Very good. Thank you.

Next question then is wondering when in the FDA review process do most companies decide to pursue a PCCP? And as a follow-up, what is, or what if or sorry. Was it during presub or supplemental review? Well, I'd like to say it's a bit of both. So it depends on, you know, where you're thinking your product is going and what your overall strategy is.

You know, you think years and years in advance, not just the submission you're working on today. Companies we've worked with typically pursue PCBs in one of two ways. So at initial submission or even before you you clear a product that, you know, is very new, the PCCP can be included as part of your original premarket review. I'd even go so far as to say, you know, mention it in your QCEP. Like, I always recommend the pre sub process as a way to, you know, align on your V and B strategy, try to find ways to make sure, like, the FDA, you know, says yes or no to some things, you know, really derisk your future submission as much as possible.

One of those things can be, hey. Is this PCCP appropriate given, you know, the type of intended use we're trying to do? And is it you know, would this be something that makes sense? Would it not invite new questions about safety and effectiveness when you look at the technological characteristics? So when you get to that premarket submission, so your five ten k or your de novo or your PMA, you can allow that device to also be cleared with the PCCP.

The other way is in post clearance. So say you already have a product and you're like, wow. This product is doing great. We already have you know, based on all this field feedback from customers, we know how we wanna change it over time. Then you've put in your, you know, your second submission, and you can obviously use your first one as a predicate.

Put in your set second submission, and maybe nothing really changes except that one thing, which is adding a PCCP saying, yes, in the future, you know, next year or the years after, we're gonna change it by updating the machine learning model. A common strategy is to decouple the PCCP from your course submission because, you know, sometimes you wanna reduce the risk of your course submission. There might be a lot of things they're trying to get cleared at the same time. So, you know, consider the risks and rewards there and what your timelines are for commercialization. Perfect.

Thank you very much for that answer and for all of the answers here today, but we have reached the end of our time. Now if you're gonna attend to your questions, know that the team at Ketryx will follow-up with you. Or if you have some further questions, you can visit, the website on the screen. I've also put it into the chat. I wanna thank everyone for participating in today's webinar.

You will be receiving a follow-up email from XTalks with access to the recorded archive for this event. A survey window will be popping up on your screen if you click the exit button, and your participation is appreciated as it helps us to improve our webinars. Additionally, I'm sharing a link to view the recording of today's event in the chat box, which you can also use to share with your colleagues so that they may register for the recording here as well. For those watching via LinkedIn live events, we encourage you to visit x talks dot com to register for this event and then gain access to features such as viewing the recording and more. Now please join me once more in thanking our speakers for their wonderful time here today.

We hope that you found the webinar informative. Have a great day, everyone, and thank you for coming. Thank you so much, everyone. Bye bye.