--- title: "Paving the Way - FDA PCCP Authorization with Beacon Biosignals" type: webinar-transcript publisher: Ketryx source: "https://fast.wistia.net/embed/iframe/2njzswem22" content: auto-caption transcript, proper-noun corrected --- # Paving the Way - FDA PCCP Authorization with Beacon Biosignals *Ketryx webinar — transcript of the recorded session.* [▶ Watch the recording](https://fast.wistia.net/embed/iframe/2njzswem22) --- welcome again to everyone. Before we begin, we'd like to cover a few housekeeping items. First, the webinar is being recorded, and it will be sent out along with the slides following the conclusion. We'll also share other resources in the chat during the webinar. Second, go ahead and put questions in the q and a at any time. Our colleagues, Isken and William are on the line, and we'll be surfacing your questions from the chat to us throughout our presentation. So it looks like many of you on the line are already integrating AI into your product or have plans to, which is really exciting, and we can totally understand why you're here. Alex from our customer base, I think, is as expert as they come in the PCCP process, and I'm sure many of you are interested in how you can apply that to your own regulatory workflows. And so we're excited to have that conversation. We will move forward now. And I think that that is a good lead in, Alex, for us to introduce ourselves. Would you mind introducing yourself to the audience? Absolutely. Yeah. Nice to meet everyone. I'm Alex Chan. I'm VP of analytics and machine learning at Beacon Biosignals, where we're really pushing forward, using machine learning and AI in advancing neurobiomarkers for brain health and and really helping, accelerate therapies in that area. My background really is in, computational neuroscience and medical engineering. I did a lot of time, do really at the intersection between neuroscience and machine learning, applying machine learning to understanding the brain, and also really moving forward towards understanding how we can use machine learning to to help people with various, disorders of the central nervous system. I spent a lot of time in my past career at Apple as well, leading the health technologies algorithms team that delivered, sleep staging for Apple Watch as well as sleep apnea detection. But really, really happy to be here with you, Jake, chat about PCCP and kind of the things we're doing on the machine learning and AI side at at Beacon. Yeah. I'm I'm super excited for the conversation, Alex. And to give you guys a brief introduction to who I am, my name is Jake Stell. I am Ketryx's vice president of client operations. I manage pretty much all aspects of customer experience for Ketryx, and that includes implementation, support, customer success. And my team is very heavily involved in helping our customers achieve kind of the compliance and the operational outcomes that they seek to achieve. I studied at MIT. I got a master's degree in business administration and a master's in systems engineering. While I was in that experience, I co opted at Amazon where I did process engineering for them in one of their fulfillment centers. I then moved on to Amgen where I served in various operational capacities. I was a major deviation investigator in one of the largest biopharmaceutical manufacturing plants in the world, and I also was a manufacturing manager in a drug substance plant before I moved on to do digital project product management, I should say, for a team that built AI models for various aspects of Amgen's process development function. And now I'm at Ketryx doing this and helping to solve a problem that I saw as being critically important while I was at Amgen. Just briefly gonna take you through the agenda. We're gonna do a brief introduction to beacon by beacon biosignals. We are also going to talk about what PCCPs are and why beacon, pursued them. We're also going to talk about how Beacon trains and validates its AI models, and we're also gonna talk about combining PCCPs, agile methodologies, and the right tooling to release faster in a regulated environment. And so to kick us off, I wanna ask you, Alex, can you share a bit with us about Beacon's, mission? Yeah. Yeah. Can you share a bit about with us about your mission? Yeah. So Beacon at Beacon BioBalance, we're really focused on leveraging AI and machine learning to really accelerate therapy development for diseases that affect the brain. This really can include a number of things. You know? Obviously, one big one is in sleep sleep disorders, neurological disease, psychiatric disease. And it really is, you know, anything that touches the brain is is what we at Beacon are are striving to to accelerate therapies for. Our our main focus right now really has been working with pharma companies, who are developing therapies for the brain to to really help accelerate their drug development in the neurospace by helping them incorporate these quantitative neuro biomarkers that really can, in many cases, uniquely extract using machine learning and and AI types of techniques. Incorporate these neuro biomarkers as part of clinical trials, and really help them move these these therapies along to patients much more quickly. And, you know, but they were also we're also starting to think about how we can take our machine learning, our medical devices, and use them more directly to help diagnose and manage, you know, disorders of sleep, neurology, psychiatric disease, those types of things. And so that's really what we're focusing on, and, you know, AI and machine learning really takes, front and center role in in helping us fulfill that mission. Yeah. That's very exciting. Alex, one thing I wanted to make sure I asked you as a part of this is I'm I've worked with you now for, a couple of years, and I know that you are a world class engineer. And I wonder if you could tell all of us what brought you to your current role at Beacon. Like, what brought you to this mission? Yeah. Absolutely. I think, big part of this is is my my background has has been in neuroscience. And so understanding the brain and, you know, trying to help those with diseases that touch your brain has always been something that is really personally interesting to me. And so being able to get back into this area, you know, after spending a lot of time at at Apple, also in the health space, but not not quite in neuroscience or in neurology, was it was very exciting. That was one big piece of it. The other piece is really, you know, coming to a a start up, you know, back to a start up. I had spent some time at a a small start up that I really enjoyed that particular experience because you were working with a a small team that's incredibly passionate about its mission, and you're able to potentially really affect change in a really quick and rapid way, and iterate towards a solution that'll really be able to help people with with unmet need. And so I think those that combination of things, you know, being able to work in the neuroscience space with machine learning, in a in a small startup is really appealing to me. Yeah. It was like a perfect job for you, wasn't it? Yeah. I I think it was it was it was really great to to be able to to find this particular role, and it's just been incredibly fulfilling, over the the now three or more years that I've been at Beacon. Yeah. That's awesome. And it's a great lead in to our next question. Would you mind walking us through the Beacon platform and who it helps? And could you touch on why it's important to you to get your products to market faster? Absolutely. Now let's start with the the latter part of this. You know, I think really important part of this is core to our mission. Right? It's you know, in the end, we wanna be able to help our patients, the patients, patients who have various diseases that touch the brain, and be able to to to be able to help those patients. We need to be able to, you know, rapidly find, and to accelerate therapy development, as I mentioned, with our pharma partners, especially in clinical trials. And be able to do that well means being able to, iterate quickly on the products we provide them, and respond to their needs in a really in a really quick and and reactive way. So at Beacon, what are we building? We're building a platform that really allows us to acquire and analyze brain data. It's right now largely focused on sleep, and it really is built to power these clinical trials for our pharma partners who are developing these types of narrow drugs. And so on the left, really starts with acquisition hardware that really makes EEG or or essentially, measuring of brain signals much more accessible. And we'll talk a little bit about how, you know, the current status quo is is quite it's quite difficult to actually obtain these types of signals. And so we have this wearable hardware, that we call wave band that allows us to to acquire these signals easily in a longitudinal fashion in patients' homes. This data comes to the Beacon Cloud where it goes into the Beacon data store, that allows it to be organized in a really structured way. And on top of that, we have analytics, which really is the core of the machine learning and AI analysis that we run to really analyze this data that a human really is is not well suited to analyzing large amounts of longitudinal data here. Mhmm. We take that, and we display it in what we call the Beacon HQ. It's a front end that allows for, you know, our own data scientists and our partners to be able to explore their data and generate scientific insights from it. And so all of this really enables trialists, researchers, other stakeholders, and clinicians to be able to to draw insight and and more easily leverage brain signal data. And so as part of this, we'll we'll dive a little bit into two specific medical devices, you know, in a bit about that that really leverage and exemplify the AI ML capabilities that we have. Yeah. And it's a great lead into the next slide. And can you tell us, Alex, a bit more about these two products with approved PCCPs? Yeah. So so both these are our two, FDA five ten k cleared medical devices, both both with PCCPs. These are the really the first two, sleep related medical devices with PCCPs authorized at all. The first one, starting on the left, is called Sleep Sage ML. This is a software as a medical device or a SAMD that really, at its core, is a machine learning algorithm that really aims to accelerate the evaluation of traditional, in lab sleep studies. Mhmm. So to say a little bit more about this, you know, these when a patient comes in to get diagnosed with a particular sleep disorder, they usually have to go into a sleep lab, and get hooked up with a bunch of sensors on their head and on their body to monitor their brain activity, their breathing, their movement, all kinds of all kinds of different signals, simultaneously, and they're asked to sleep. This requires a trained technologist to do. And then after they sleep for that night, a human expert would have to review that entire recording, all six to eight hours of it, thirty seconds at a time to determine whether the person was in a particular stage of sleep. Were they awake, light sleep, deep sleep, REM? And it takes enormous amount of time to be able to do this by human experts. Sleep Sage ML essentially is a machine learning model that automates that that analysis of the EEG, the brain signal data to do sleep staging. And this is a service we provide for for, our pharma partners who are doing traditional lab studies in lab studies. The other side of the, this our medical devices is the wave band device. This one's a combination of hardware and software. And so the the core and really important part of this is its headbands that allows a patient to put on put it on themselves and acquire a night of brain signals, the same kind of brain signals you get from a traditional set sleep study, but can be acquired in a patient patient's own home, in their own bed. The machine learning part of this is on the software side where we're, again, analyzing using a machine learning model, the full night of data to estimate sleep stages, at thirty seconds at a time for for that entire night. And so this really allows us to to evaluate sleep in a much more naturalistic environment. Awesome. Could you go into a bit more detail about your aim AI model for sleep staging? And could you talk to us a little bit about your approach for training and validation? Yeah. Absolutely. So both SleepStage ML as well as the the wave bands, medical device have at their core a, really, a a machine learning model and actually a a an algorithmic pipeline that performs this sleep staging. It's very common across it's very, very similar across the two devices. It starts with really, the input signals. In the sleep stage ML case, this is brain signals from a traditional in lab sleep study. In the wave band's case, this is just the brain signals that we get off of our, dry electrode EEG headband. This goes through a number of stages versus signal preprocessing, which does some digital signal processing, filtering, those types of things to the signals, getting them ready for the the machine learning model itself, which performs that sleep staging piece, followed by a post processing piece that, takes those probabilities that are output from a machine learning model and converts them to discrete stages every thirty seconds of the night. And so, really, the core of this is that machine learning, that green machine learning box there. And for both these cases, we train these models on, you know, thousands to tens of thousands of nights, recording with ground truth labels. The models themselves are fixed, and so they don't they don't continuously adapt in the field, which is an important consideration, on the regulatory side. And then the in terms of how we go about validating these particular devices, involves really collecting a representative sample of of, data from the device itself or traditional sleep lab data and demonstrating the performance of these devices, meets our particular requirements and specifications, is able to actually help clinicians in in the review and assessment of sleep. And so those three components are actually important, later on in terms of how we ended up structuring parts of our PCCP, in terms of how how we describe those modifications. That's so interesting. Alex, I have I have a bit of a naive question, scientific question for you. What what is the variance like in this data? Like, how different are people's experiences of sleep? Oh, yeah. It's it's really interesting. It's I think sleep is one of these things where, everybody does it, and so everybody has an opinion. But it it is it's also, like, it is very variable overnight of the the course of nights. And so it's a well known, fact that measuring a single night of sleep, is not is not a great diagnostic test for many things. Like, you know, for example, in sleep apnea, looking at breathing events and these types of things can change from night to night. Individuals, sleep architecture can also change quite a bit from night to night based on your prior day's experience, and many other factors. And even the ground truth of understanding sleep staging is highly variable. When you ask two humans to to stage sleep of a person over over the night or even ask a single person to stage the same recording, you know, separated by several months. There's huge variability there, and I think this is one of the big promises of using machine learning is to allow us to get to more consistent states where we're able to more consistently, you know, stage and analyze these types of data without some of the subjectiveness that, inevitably comes in when humans are looking at it. Yeah. That's so cool. I mean, it's one of those things that's, like, so mysterious. Like, what sleep is about and why. It's it's a really, really interesting problem, Alex. Yeah. It's it's it's just something that I think from our perspective at Beacon is just so interesting, also because sleep is an insight. It's like a a view into the body that is not can reflect so many other things that are not just sleep. Yeah. And so it really is a this really nice window into human physiology that we think is super important to study. Yeah. So cool. We have a question on the line. Steven, if you won't mind, I'm gonna I'm gonna delay your question until we get through kind of the next question, and then we'll come back to it. But, Alex, could you please explain to everyone what a predetermined change control plan is and why is it valuable for AI enabled devices? Right. Yeah. This is great. So, so, predetermined change control plan, this is really allows, a manufacturer of medical device to modify their medical device without a new premarket submission by essentially, you know, prespecifying the methods, the the exact modifications that might be made, and how you go about validating that. And, really, the purpose of it and the goal of it is to allow, for the continued safe and effective and efficient update of medical devices, you know, via these preapproved pathways. And so this diagram on the on the right here has two two to separate size. So it the the left side really shows, what updating, you know, in this specific case, a machine learning based, medical device or AI based medical device might look like without a predetermined change control plan, where you'd go through and modify modify. Maybe you're retraining the model itself with more data. You'd have to go through, validation and verification of that model and the software, as a whole and the medical device as a whole. Then you have to go through this orange box. Right? This is, interacting with the FDA to to get approval or clearance for the the this premarket submission for the changed and updated medical device. Then you can go ahead and and market that and deploy that, and you'd cycle through that loop. The issue is every time you wanna cycle through that loop, you have this place where you have to interact with the regulatory body. And so what a PCCP allows us to do is to essentially front load that effort, to such that once you have, undertaken the efforts to to prespecify and have gotten authorization for how you the specific modifications you might make, how you go about validating and and verifying those modifications. Once you have approval or, authorization for that plan, which is the PCCP, now this orange box, you can continue to iterate without necessarily needing to interact with that regulatory body as long as those modifications fit within that plan, which is a really powerful thing, especially on machine learning. For one, you know, on the machine learning side and and AIML devices, It's really common for, us in this field to want to continue to improve our our models by incorporating more and more training data. And it's a it's something that's commonly done, all across AIML, even in nonregulated spaces. And being able to do that efficiently is really important for us to be able to continue to improve our products. The other piece of this is that these machine learning models are trained based on data, and the data distributions in that we're operating on in the real world can change for a number of reasons. For, you know, sometimes patient populations might change that you're you're using your medical device on. Sometimes clinical standards of care might change that change the data coming in, and being able to address those types of changes quickly, is really important. And so the PCCP framework, I think, is a a really important tool to allow us to be able to to address those types of of, updates to machine learning models that really will allow us to continue delivering effective and safe, medical devices. Yeah. So I think our next question should cover your question, Steven. So let me just advance. Oh, I'm sorry. That's not the slide I wanted. So let's let's pause and answer Steven's question here, Alex. So Steven asks, what changes are allowed under the PCCP, and does the software undergo full VMV with each change? Great question. Yeah. So I think, we we might touch on this a little bit, later as well, but there's there's a number of modifications that that you can make. In most cases, the PCCP can't change the the, intended use or indications of use for the device. But the modifications typically are are one of, kind of three main types of modifications. One is usually modifications whose purpose is to change or improve the performance of a machine learning, or AIML model, especially if we're focusing on that type of, device, AIML enabled device. Two, modifications that are kind of related to the device's inputs or compatibility. And so, for example, you know, if you want your machine learning model to accept, inputs from maybe a a a different accessory or different other medical device that it's analyzing, you know, having a PCCP that allows for that type of modification is another piece. And then one other one is really there are some modifications around the device's use and performance. For example, like within specific subpopulations, that are within your intended use or indications of use, or you could you could, for example, continue to train your model to to perform better on these particular subpopulations or to particular sites or things like that. And so those are the types of purposes. In terms of the actual modifications that can be made, it really depends. I I don't think there's any, I don't think there's any hard and fast rules about what those modifications can be, but we'll talk a little bit more about how to how to navigate some of that, that complexity in terms of the specific technical modifications you can make to your device. Yeah. Awesome. So we're gonna advance into a poll here. And while we're doing that poll, I have another question for you, Alex, which is from from Jess on the line. How does the I think the data that your products collect compare with the detailed sleep data my Galaxy Watch collects? Yeah. It's great. Yeah. So I think, for for a bunch of the, like, the consumer based, sleep monitors, they're usually based on recording accelerometer data. So gross movements, you can see interesting things actually on on those on the accelerometer from risk based wearables that are things like breathing, some crude breathing metrics. Sometimes you can even see heart rate, and those types of things via the optical sensors. And so, you know, a lot of that, you can estimate sleep reasonably well, using a watch based sensor. But, you know, the thing to keep in mind is sleep is defined as a brain state. Mhmm. And so by not measuring the brain itself, you you leave things on the table, especially if you're really, caring about disease populations, which is where Beacon is focused. You really care about these abnormal sleep, you know, architectures and patient populations, which you won't you won't it's really difficult for, risk based, sleep tracking to to accurately, measure in many cases, and there's particular sub stages of sleep that you can't measure with your watch. But, you know, I I do I'm a big fan of of, you know, these types of consumer wearables for for sleep tracking or that type of thing. But I think for our purposes, we do think there's a monitoring the brain itself is really important. Yeah. Yeah. Definitely understand. There's only so much you can do with the device sitting on your wrist. Right? Absolutely. Yeah. Awesome. Thank you so much for the question, Jess. And we are sharing the results of that poll with you. So it seems like the challenges that y'all are facing is there's unclear regulatory guidelines. You're still trying to determine whether the PCCP aligns with, aligns with or requires a significant change assessment. And, I see, incorporating PCCPs into the significant change assessment process, and then there's some tailing here about planning and scheduling PCCP changes. So it's very interesting. Appreciate all of the information, guys. We will move forward with the questions now. Let me so, Alex, could you tell us what are the key regulatory considerations we should be thinking about when preparing a PCCP? Yeah. Absolutely. So I think this this gets back to some of the the previous questions. But, right, PCCPs, I think, really are meant to support changes to medical device that would usually require a new premarket authorization. Right? And so things that would if you go through the the typical flowcharts and FDA guidance would which would not require a new premarket, submission are not things that are really within the scope of of putting into a PCCP. Those should continue to flow through existing, pathways, you know, in terms of submitting letters files if these are, like, things like small bug fixes and through your typical quality system. And so, really, p c p PCCPs are meant to address those those larger changes that would otherwise trigger a new regulatory submission. And in most cases, many, many algorithm updates, if those those algorithms are core to the particular products, and have a particular risk profile often do or often would trigger this, not in all cases, in many cases, which is why it's well suited for for AIML types of, medical devices. I had mentioned before also PCCPs shouldn't change the intended use or in most cases, the indications for use for new medical devices. They can be part of most five ten k's, de novo or PMA submissions. Mhmm. It really does include most of the types of submissions. Really, it it just doesn't it can include PCCPs and some of the submission types that don't require, like, explicit FDA approval or, submissions that have a shorter review cycle. Those are the the the few limitations there. One thing that's important to note is the PCCP itself is considered a technical technological characteristic of the medical device itself. And so what that means is that, when when updating the PCCP itself, it's essentially considered updating a technological characteristic of the device and would typically require re rereview by the FDA for that for the update to the PCCP. And the other piece of this is even in the case where you are adding on a PCCP to an existing medical device, that addition of the PCCP, again, because it's a technological characteristic, would require a a full submission to get that PCCP authorized. And so, you know, it's actually our our case, we we experienced this firsthand, and I'll talk a little bit about that later. And then the final thing to note here is that, when you're using a device that has an authorized PCCP as a predicate, the device itself now can change, right, without needing to go through, additional premarket submissions. But when you're using a device with PCCP as a predicate, you should be using the the the version of that device that was cleared before any PCCP, authorized changes, were made to that device. Maybe we can go to the next slide Definitely. And, talk a little bit about what what a PCCP kind of looks like, in terms of its structure. There's three three main sections to a PCCP, in terms of the document itself, a description of the modifications, the modification protocol, and then the impact analysis. The description of the modifications really is, really is intended to list all of the really specific proposed modifications you're planning to make to the medical device, that defines essentially the scope of that PCCP. And so if any of those modifications are being made, then this PCCP would be the thing that that kicks in to to help, ensure the guidelines for validation verification and how you actually go about implementing it. Those modifications really should each, each of those modifications should be able to be v and ved. The modifications, you need to specify things about the modifications regarding are they intended to be implemented manually or automatically. And I believe, you know, if you if you intend to have automatic, modifications or, like, retraining to your machine learning, often is will likely incur more scrutiny. It's not something that I believe is very common yet, but, it is something that you could potentially explore with the PCCP. You do need to determine whether those modifications are intended to be rolled out globally to all medical devices or just some particular subsets, maybe certain sites, certain patients, and also describe the frequency of those modifications. Then the modification protocol, the second section, really is describing how you go about implementing those modifications and validating and verifying them. And so this really is kind of takes the place of the clinical validation protocol or, as well as the statistical analysis plan where you're describing how you're gonna go about collecting the data to update the device or implement the modification, collect the data to test the device, in terms of its performance, algorithmic performance. You just set the practices and specify the practice practices by which you'll go about retraining your model, how you'll go about validating it. And so you will need very specific acceptance criteria, that can be for each of your modifications to understand whether, modification should be allowed to be to go into production. And, really, you're linking, the modifications themselves to each of the individual components of a protocol, if you have multiple modifications. The final piece of this is impact analysis where you're really kind of, at a high level analyzing the difference between your original device and a potential device with modifications implemented with this PCCP. Need to discuss the risks and benefits of these modifications, as well as, you know, an important part of this is how different modifications might interact with each other, and whether there's any things, any impact of of multiple modifications being made, in terms of devices functioning and safety. And so these are these are really the main components of the PCCP that will have to be in every submission. Thank you for that, Alex. That was awesome. Great introduction. We have a couple of questions from a gentleman named Paul. I think we answered the first one, but I'm gonna give you the second one. When in the FDA review process did you decide to do this? Was it pre sub versus supplemental supplement review? Oh, this is a great question. Yeah. Maybe maybe we can, maybe we can go to the next slide and, like, kind of our history first and then I'll Yeah. Yeah. For sure. From here. Yeah. So, right, so how did how do we go about like, first of all, I think before we get to that question, how do we go about deciding how to do our PCCPs for our two devices? So Sleep Stage ML and Waveband. Right? And so Sleep Stage ML was a new five ten k, that we were planning to submit, at the end of twenty twenty three. The the guidance for PCCPs and AIML enabled devices came early that year. And this was really well suited for us, because at Beacon, you know, we had a growing training dataset. We knew we wanted to continue to improve this this particular model. And so in this particular case, we proceeded with, with including that PCCP with the very first, five ten k submission for this new medical device, together for a number of reasons. I think, you know, ours is a we have a particular set of reasoning for this. I think it'll be different for many other manufacturers. But, in our case, our device is relatively low risk and also demonstrated relatively high performance compared to a bunch of predicates as well as competitors in the space. And, also, I think an important part is we had a very a fairly well established validation pathway in terms of, like, what are the specific acceptance criterias, what are the methodologies. We'd go about collecting data and validating these types of devices because it had been fairly well trodden in the past by by these other devices that came before us. And so because of that, I think we felt confident to go forward with the PCCP here, with this original five ten k. And to answer this question, it really is, we we, in this case, didn't didn't go with, a precept. And part of the risk mitigation here is on this last bullet point. We made sure that the PCCP was decoupled from the rest of the submission, and such that the rest of the submission didn't depend on any specific piece of the PCCP in order to stand by itself such that if we got particular feedback from the FDA that made it, risky, that's the the entire device wouldn't be, in this case, cleared with the PCCP. We could pull that PCCP, at the last minute to to allow the device itself to be cleared. So that was important, consideration for us. Then on the wave band side, this was originally cleared under the name Dream three s. But this was the device itself was clear without a PCCP in twenty twenty three. And here, after our, success with SleepSafe ML, we decided to add a PCCP to this device in twenty twenty four. And so in our communications with the FDA, it was very clear that even though the device itself hadn't changed, adding that new PCCP would require a new five ten k, as I mentioned, because it's a technological characteristic of the device. And so we did we did have to follow-up with a full new five ten k. The the PCCP itself drew a lot heavy inspiration from SleepHQML. And so you know? And mainly because it's very similar device, just included a hardware component. Architecture was very similar. But, you know, we learned a lot from that that first component that, you know, happy to share more here. Awesome. We are we are getting a lot of questions, Alex. Why don't we see if we can knock a few more off? We have a an anonymous attendee who asked, do you think the PCCP process review and approval would be different for premarket approval products versus five, ten k devices? Yeah. I I I can't speak directly from this. This is speculation. I'm also not a regulatory professional, but my guess is my guess is yes. You know, the risk profile of the device certainly, plays into the what a PCCP, how much, scrutiny and and the the amount of, you know, controls that might be in place for PCCP. So I definitely expect this to be very different for, like, a PMA versus a five ten k. You know, I've heard heard, in other conversations that that, you know, there have been people who have been trying to do this on the PMA side and face, you know, additional challenges. And so I think for us, in a relatively low risk device, it has we've had to think very carefully, and we had had to put in a lot of, controls to really satisfy our viewers that we're we're not, you know, for example, proceeding down roads that would lead to overfitting and things like that. And so we've had to put in fairly rigorous controls, but I expect, you know, potentially even more that being necessary for a PMA. Definitely. I I can only echo that. The next question I love. Erwin asks, do we have any awareness of PCCPs in non AIML devices? Good question. I I so this is not, this is not something that I have, firsthand knowledge of. I do know, right, with the FDA recently released the PCCP guidance for for general m I, medical devices, not just AIML, fairly recently. I don't I don't actually off the top of my head know of, of devices that have leveraged this for other other non machine learning changes. I think there are some, but I I don't I can't I don't think I could answer this question specifically regarding, existing devices that might have done this. I am I am also not aware of any devices that are non AIML that have been developed with a PCCP. Awesome. So we have another question here. You mentioned PCCPs for larger known changes that would require a new premarket submission. So would big fixes not be part of a PCCP or updates for cybersecurity vulnerabilities found in a PMA device? It's a good question. Yeah. I I think the the main the thing I have here is really really, if if the existing flows, when you go through the FDA, you know, flowcharts Mhmm. Do not require a premarket submission for that change, then it it really it does the reading of the the guidance really does suggest that it it likely doesn't fit within a PCCP. You know, I'm not a full expert on that. So but that that's my understanding is that, you know, even for large changes that wouldn't require premarket submission, they should they should not they should follow the existing guidances. And it really is things that would would otherwise trigger something. And so if it did, then it, you know, feels like the the latest guidance on general PCCP or general medical device updates feels like it could fit into that, even if it's not a AIML update itself. Awesome. Thanks, Alex. We have another very interesting question from another anonymous attendee. If our software is initially designed to address one infectious disease, is it possible to structure the PCCP in a way that it can later be extended to cover additional infectious diseases without requiring a full new submission? In other words, can a BCCP be written broadly enough to encompass a class like infectious diseases rather than being limited to a single condition? On Yeah. Yeah. I think that's gonna be very, like, it's gonna be very dependent on the the disease area and what you I think that there's probably, like, a technical component of the review of, like, how much the model needs to change in order to encompass the new, the new disease state. But I think that probably challenging. That would be my guess. I I personally haven't heard because I I think the the agency would probably look at that as changing the intended use of the device. That's my my ten second analysis of that situation. Alex, I don't know what you think. I don't know if I have more than that specifically, but I do this is something that, you know, I think we're at Beacon are also interested in. And, you know, like, could we could we leverage this? And I think, you know, when you look at that that restriction to not changing the intended use, really or in most cases, the indications of use, makes it difficult. I there might be ways to structure it in a way that, but it feels like, you know, it's I think this is the going from going from general to specific is is often challenging from what I understand on on the medical device side. If you have, like, a general, indication or intended use, going to say that you specifically treat or are are, focused on particular in, you know, disease states. Within that, it feels tricky, but there may be ways to do it. I don't I don't specifically know. There is this, in the guidance, there's this discussion of improving performance on particular subpopulations, right, within your intended use. And so, this is more speculation, but there may be there may be ways to to leverage that to to accomplish parts of what you're trying to get at. And so, like, but I I I think a lot of it is comes down to how you plan on marketing and labeling the particular device, and so it feels like it likely crosses into a gray zone here. So I know we have more questions. We do still have some content to get through, so I think I'm going to pause on the question answering right now so that Alex and I can move forward. But if we have time at the end, we will pick up the questions again. So, Alex, what role do PCCPs play in helping Beacon achieve its goal to release updates faster? What challenges remain? Yeah. That's a great question. I think so PCPs PCCPs really allowed us to, you know, retrain our AIML algorithms without needing to go through our new five ten k, submissions, which made it really made our iteration and, release cycle, for the AIML updates, the retraining, really more predictable since we didn't have to necessarily interact with this external regulatory body for these types of improvements. But while while they help us streamline that piece, it didn't you know, PCCPs don't directly address the needs of being able to iterate quickly with respect to general software developments, you know, in terms of, you know, following modern agile practices and those types of things. And so, you know, many changes that we make fall outside of PCCPs. Right? They they're these minor improvements or bug fixes that don't need a new submission, and making sure that we could effectively and quickly document those changes. Not something that's directly, addressed by PCCPs. Awesome. Alex, and for the room, can you just tell everyone how often you release? Yeah. So, it depends on our products, but for many of our products, some of them, we try to follow, GCP, and, you know, GXP for a bunch of our software products. We lease lease them on every other week, really in terms of and then following following design controls for them, you know, leveraging leveraging Ketryx, for for our design controls and documentation. Our AIML pieces, don't it would I I I aspire to be able to train our models at such a rapid cadence. So I'm not there yet, but we'll we'll get there eventually. And and how instrumental would you say Ketryx has been in being able to help you achieve that release release cycle? Yeah. I think I think Ketryx has been really important to to be able to allow us to, it's a combination of Ketryx as well as some of our, testing practices. So, like, really leaning in and, and so, like, I think some of this is on our next slide too. But Yeah. Yeah. I think is, really leaning in on the automated testing pieces has been really important. And then leveraging Ketryx's ability to to to, track those test executions, and generate automatically generated documentation that's compliant. That's been really important for us, to be able to to more quickly and be able to to maintain that iteration cycle. I think that that cycle has been really important for us to be able to continuously maintain, within this regular regulated environment. So we were really looking for solutions that that allow us to that wouldn't necessarily slow us down, while maintaining high quality and, software in a way that is compliant with the regulations. Yeah. And I think this is something that we see across our customer base is that it's a it's a combination of tooling processes and and training to to get to a really rapid release cadence. And, it is it's something that we're seeing across our customer base is people who are able to get those three things in harmony are really able to increase the the speed, and the velocity at which they release. Another another thought here, Alex, is, like, my impression from working with our customers is that I think there's some concern in the industry that shipping faster will lead to some degradation in quality. And my perspective from working with our customers now is that every time you improve your release cadence, it actually improves the quality of your your product. I wonder if you have any any thoughts about that. Yeah. I I I think so. Like, I think there's something about, reducing the burden, that comes with making sure documentation for these changes is is as low overhead as possible. But because otherwise, you know, you have this potential disincentive to continue to improve your products, which I think is incredibly important. And for us is at Beacon, it's really, really important for us to be able to continue to address all of these, you know, address the improvements that we wanna be able to continue making, fix the bugs that that we find. You know, all these things are really important for us to to be able to to do. And by moving more quickly, really allows us to continue to deliver these types of products. I I think this really this is why agile development really is a is a thing, and it's important in the software industry is is because it allows us to to more quickly get to to high quality, high quality software, I think, is no different in in the medical space. And just finding that, I think, magic combination of processes, tooling, you know, and kind of training and practices is really important. I, you know, Alex, I I talk to people about this a lot, and I've never heard it framed quite like that, which is that if you have a really burdensome validation process, it can almost disincentivize you to to ship improvements. And I think it's it's a brilliant framing. It's a very good way of thinking about it. So let's let's talk about, why utilizing the PCCP is so hard and, specifically, what advice you would offer other companies considering PCCPs and faster release cycles. Yeah. And so I I think from our perspective, you know, we've gone through the PCCP cycle twice now in terms of, getting this in in two of our medical devices. I think the things we've learned here are really to, you know, define define modifications very carefully. I think the FDA is really looking for, really specific as specific modifications as possible that are as well bounded as possible. And so trying to trying to, you know, to get a successful PCP, I think that the more well bounded these modifications are, the better. Mhmm. The other piece that I think has served us well also is really explicitly describing the differences between the PCP acceptance criteria and the validation, plan within the PCCP with kind of that original device, validation, validation study or whatever whatever it ended up being. And I think in our case, being able to to point to to, really implementing all of the all of the verification software verification validation that we did for the original device, as part of the piece any future PCCP update was really important for us. And, actually, in this case, I think, also, Ketryx helped us here because it it made it actually makes it quite easy. It's the automation plus Ketryx makes it very easy to be able to say we're gonna repeat everything that we did for the original medical device for any PCCP update. Then specifically for AI ML, I think, is really important f our FDA review is really looking for strong controls over mitigating risks of overfitting, or overestimation of performance. And so I think strong technical controls on the, essentially, the test or the final validation datasets is really important, to see to make sure that there is no no potential leakage even, you know, even if you don't actually use that data in training, but derive any type of human insight that could lead to that, you know, that's dataset being overfit to some degree, was something that they cared strongly about. And then, again, more generally in the software development side, investing in really and heavily investing in automated testing, across all the stages of software development and then coupling that with the right tooling, in terms of minimizing the compliance burden for generating documentation, I think, really important for us. And so yeah. So I think those are some of our learnings on the PCCP side that, you know, I think we took even from our first one, Sleep Stage ML to wave band, and and got through successfully. Yeah. And just to speak briefly about this interaction between automation and tooling, I think one of the main things that our our customers communicate to us and one of the reasons that they buy us is that it it's clear that we're building software and we're building systems that are sophisticated enough now that you can't just use one monolithic tool to to manage the validation or the building of those products. And so the the approach that we take is we connect different systems together and we automatically ingest information from them. And so what what Alex and team have been able to achieve is all of their design controls are managed in specific subsidiary systems and their CICD pipeline reports evidence directly into our product. So just to give everyone a little bit of context about the way that our product is being used to support, their PCCP. And then I I think we're gonna get pushed to close here, but maybe we'll take one more question, Alex, before we do that. We have a question from Amy here. Are PCCP submissions automatic, or do they still require review? Let's see. Yeah. So so the I guess okay. I think I think the the question is, like, the original the PCCP itself is the plan that requires, authorization by the FDA. Right? So that has to be part of a premarket submission, requires that authorization that that, document needs to be approved by the FDA as part of your medical device. And and once that's approved, it's part of your medical device. And you can execute that plan really, as long as you're following the guidelines of that the plan that the FDA approved, that no longer needs you know, every time you go through that loop of executing a PCCP, that that plan to introduce a modification, validate it, that doesn't need to go to the FDA. FDA anymore. Right? And so that's that is the thing that really, allows you to to iterate more quickly is, like, if you know what you wanna do and you can get the FDA to to authorize that, then all the subsequent cases that you wanted to make those types of changes, you can you can easily make. So Awesome. Alex, I have to say such an informative hour and really, really appreciate your time and really appreciate you as a customer as well. If anyone in the audience is interested in learning more about Beacon and how they used Ketryx to implement the or support their PCCP, you can go to this case study on our website and learn more about it. For those of you who have questions that we didn't get to, I see our team taking those right now, and we'll definitely follow-up with you. And then I will just put in here that our next webinar is on Thursday, July tenth at eleven AM, and we are going to discuss navigating the FDA's total product lifestyle framework for generative AI devices. Really appreciate the time. And, Alex, I will see you soon. Absolutely. Thank you, Jake, and thank you everybody. Thank you, guys. Bye.